Sign inRegisterCheck connection

Heartbeats for a Weather Website

Heartbeats for a weather website: checking that Cumulus realtime.txt and web-file uploads arrived, with stale-data detection as quality control.

Back to Cumulus weather software guides

A weather website can look alive while the station has been silent for hours. Cumulus uploads realtime.txt and other web files on a timer. If FTP stops, the logger PC sleeps, or the Pi hangs, the last successful file remains on the server. Visitors still see a temperature. The scientifically honest question is not “is the HTTP server up?” It is whether a current observed packet actually arrived.

The historical TNET Cumulus Server Check script existed for that heartbeat. This article treats stale-data detection as quality control. It is not a commercial uptime product, not a status-page pitch, and not a download of the old checker. Modern TNET Weather does not monitor Cumulus sites and does not maintain the script.

Historical context

Personal-weather-station sites of that era typically published from a PC or small always-on host that ran Cumulus, wrote files, and pushed them to shared hosting. The public page was a PHP template that read whatever file was already on disk. HTTP 200 only proves the web server answered. It does not prove the logger wrote a new line.

The Cumulus Wiki TNET scripts catalog lists a “Cumulus Server Check Script” among the named packages (last checked 13 August 2026). Independently, the Cumulus Wiki Toolbox documents a Windows-side check that Cumulus is running and that realtime processing is updating, with email if it fails. Those are two places in the chain: the logger process, and the files that should have appeared on the website. A host-side PHP check watches the second place.

Current Cumulus behavior for the optional realtime file is documented on the realtime.txt wiki page: generation is off by default; when enabled, the file is rebuilt on a realtime interval and may be uploaded on the same interval. A checker that does not know that interval cannot tell late from dead.

What a heartbeat actually tests

Separate the failure modes. They produce the same pretty number if you only look at the temperature field.

Upload absence. The remote realtime.txt (or a processed HTML file) has a modification time older than the expected interval plus a grace window. The page is serving a historical packet. Label it that way.

Clock stall inside a fresh-looking file. Some uploaders rewrite the file on a schedule even when Cumulus stopped changing the contents, or a cache copies old bytes with a new mtime. The packet’s own date and time fields are then the evidence. If the file is new and the packet clock is old, the heartbeat should fail. Trusting mtime alone is not enough; trusting the packet clock alone fails if the station clock is wrong. A robust check uses both and reports which one moved.

Unit and field completeness. A truncated FTP transfer can leave a short line. Parsers that pad missing fields with zero will report 0 mm of rain and call it dry. A heartbeat that only checks timestamp will green-light a damaged packet. Field count and a non-empty unit token are part of quality control, not niceties.

Wrong file. A checker pointed at a sample realtime.txt in a documentation folder, or at a Weather Display ClientRaw, will heartbeat the wrong experiment. Stack identity belongs in the configuration: this path is Cumulus.

None of these tests is a forecast. None is a claim about outdoor air. The output is a derived quality flag: fresh, stale, incomplete, or clock-inconsistent.

Stale data is not calm weather

Still air, a stalled barometer, and a zero rain rate can all be real. They can also be the last line before the USB cable fell out. The difference is the clock.

If the packet time is 14:02 and the visitor’s request is 14:03, a one-minute realtime interval is doing its job. If the packet time is yesterday, the temperature is a historical observation. Publishing it without a “last updated” label converts a quality-control failure into a fake climate event. Storm summaries and heat records have been misread this way on neglected PWS pages. Official warning services do not work like that; neither should a hobby page that still claims “now.”

For hazards, use NWS or the national authority at the site. A green heartbeat is not a safety product.

How to think about thresholds without inventing them

The right stale threshold is a function of the configured realtime interval, the upload path’s usual delay, and a grace period for clock granularity. It is not a universal number, and this article does not prescribe one. Operators who copy a threshold from a forum thread without copying the interval have a checker that either never fires or never stays green.

Document three times in the station notes:

  1. Cumulus realtime interval (seconds), from the software settings.
  2. Observed upload delay (how long after generation the file usually appears), from your own logs—not from a guessed statistic.
  3. The checker’s grace window.

When the heartbeat fails, the first question is “did the file stop or did the clock stop?” The second is “is the logger PC up?” Toolbox-style process checks answer the second. This URL is about the file that the public actually reads.

Heartbeat versus realtime log

A realtime log accumulates snapshots for later analysis. A heartbeat asks whether the latest snapshot is new enough to treat as current. You can have a complete homemade archive of last week and a dead upload today. You can have a fresh packet and no archive. They are different instruments. Rotation of the archive is a third problem, at log rotation.

Practical checklist

  1. Name the file the public page reads. Check that file, not the homepage HTML, unless the HTML is the only published object.
  2. Compare packet clock and filesystem mtime to the server clock. Report both.
  3. Reject truncated lines and missing unit tokens as incomplete, not as zero weather.
  4. Show last-updated on the public page when the check fails. Hiding staleness is the failure.
  5. Do not page a human for a one-interval skip unless your documented policy says so. Gaps happen. Persistent gaps are the QC event.
  6. Keep the checker’s output labeled as derived. A red flag is not a temperature.

Modern relevance

MX sites that publish JSON over HTTP still need the same test: is the document’s valid time recent? A 200 from /api is the new FTP success. Connection-intelligence work that consumes public weather fields has the same requirement: freshness is part of the record. TNET’s public methodology treats source freshness as a quality-control input. It does not publish internal thresholds, and this page does not either.

Raspberry Pi and other small hosts fail in ordinary ways—SD-card wear, power loss, Wi-Fi association drops. Those failures look like meteorology until a heartbeat says the file stopped.

TNET research bridge

Stale-data detection is quality control on an observed feed. TNET’s note on data sources, quality controls, and methodology is the modern discussion of freshness and source identity for records used in connection research. This historical checker is a station-scale illustration of the same idea, not a description of TNET internals.

Related pages: the projects map, the named-package catalog, and realtime log accumulation. The Cumulus legacy hub collects the cluster.

Sources