A realtime archive that only appends will fill the disk. A rotation job that only truncates will throw away the afternoon that held the year’s gust. Log rotation for weather realtime files is therefore a climate-adjacent problem, not only a sysadmin chore: daily or monthly closed files, a stated retention, and a rule that extremes are not deleted in silence.
This URL is a unique dotted path, /projects.realtimelogrotate. It is not a slash twin of realtime log accumulation. Accumulation keeps the next line. This article closes, names, and retains the files those lines become. Modern TNET Weather does not ship a rotator and does not maintain Cumulus.
Historical context
Once operators began appending realtime.txt snapshots—TNET’s RealTimeLog script on the web host, or Toolbox’s realtimelog.txt on the logger PC—the file grew without bound. Toolbox’s own changelog records pruning: an auto and manual prune of realtimelog.txt, plus a fix for non-UK dates in that pruning code (Cumulus Wiki: Toolbox, last checked 13 August 2026). That history is the proof that rotation was a recognized failure mode, not an aesthetic.
Cumulus already rotates some vendor files by month: standard log files start a new file for each month. The homemade realtime archive is not that vendor log. It needs its own naming and retention because it was an extra series invented for graphs and analysis. If you rotate it as if it were /var/log/syslog—keep seven days, delete the rest—you may still have a pretty 24-hour JPGraph and no July.
The TNET project name on this dotted path marks that retention job. This hostname does not rehost the script.
What rotation must preserve
Weather logs store extrema in ordinary rows. Unlike a database with a separate records table, a high-frequency append file is the extreme: the gust is a line at 15:41, not a trophy in another cabinet. Deleting that line deletes the evidence. dayfile.txt may still hold the daily maximum if Cumulus was running and the dayfile was intact. The homemade realtime log is often the only place a sub-daily shape survives. Destroying it because /var was at 95% is an unlogged climate edit.
A defensible rotator therefore does three things in order:
- Close the current file at a documented boundary (end of UTC day, end of station meteorological day, or end of calendar month—pick one and write it down).
- Name the closed file so a human can find
realtimelog-2026-07.txtwithout reading every byte. - Retain closed files for a stated period or until a stated disk budget, and emit a warning before deletion. Silent
rmof last summer is not rotation. It is loss.
If disk pressure requires dropping files, drop the oldest closed month after copying it off-box, or drop the high-frequency series only after confirming the vendor dayfile and monthly logs still hold the daily extrema you claim to keep. Do not guess. The dayfile appendix is the daily climate of record for Cumulus; the realtime archive is the sub-daily shape. They back different claims.
Daily files versus monthly files
Daily closed files are easy to ship, easy to compress, and easy to lose as a pile of 365 names. They match a 24-hour graph pack: yesterday’s file is a natural input. They also match backup tools that expect one object per day.
Monthly closed files match Cumulus’s own monthly-log habit and keep a directory listing readable. They are awkward for a script that wants “the last 24 hours” near month-end: that window spans two files. The reader must concatenate.
Neither choice is mandatory. Mixing them without a manifest—some days, some months, some files named realtimelog.txt.1 in syslog style—is how archives become unreadable. Syslog-style numbered rotation (file, file.1, file.2.gz) also erases identity: file.3 is not “March.” Weather files need dates in the filename.
Timezone of the boundary must match the log’s clock policy. Closing “at midnight” in PHP’s default zone while packets are stamped in station local time splits a meteorological day across two files. That split is survivable if documented. It is poison if a monthly rainfall sum later assumes one file equals one civil month.
Disk is a sampling parameter
Retention is part of the experiment. A station that keeps 24 hours of one-second samples has a different climate memory than a station that keeps five years of five-minute samples. State the product:
- interval (from the accumulator);
- rotation boundary;
- retention (count of closed files, or months, or a disk budget);
- off-box copy (yes/no, where).
Cumulus MX’s own backup folders for vendor files keep a limited number of dated copies on the same device—the dayfile wiki notes on the order of several restart backups, not a decade of history. Homemade realtime archives should not pretend they are safer than that unless an off-box copy exists. Same-disk rotation is still better than one ever-growing file, because a corrupt truncate then damages one period, not the whole series.
Extremes: do not prune the peak by accident
Toolbox’s prune feature existed because realtime logs get large. Prune-by-age inside a single file (delete lines older than n days, keep the file name) is operationally convenient and scientifically dangerous if n is shorter than the phenomena you later quote. A “weekly prune” will not contain the monthly gust. A graph pack that reads only the pruned file will then disagree with the dayfile, and operators will trust the prettier line.
Rules that respect extrema:
- Prefer file-level rotation over deleting lines inside the current file.
- If you must prune in place, copy out any row that is a running monthly max/min before deletion, or accept that the pruned file is no longer a climate source—only a 24-hour plot buffer.
- Never fill pruned holes by repeating the last kept value. That draws fake calm, the same figure crime described in the JPGraph article.
This is quality control, not nostalgia. A derived archive that silently shortens is a different dataset than the one you started.
Distinct from vendor rotation and from accumulation
Cumulus monthly logs, extra sensor logs, and dayfile appends have vendor rules. Do not run a generic logrotate config on data/dayfile.txt. This article is only about the homemade realtime series.
Accumulation (/projects/realtimelog) may run every minute. Rotation may run once a day. They should not share a lock that causes the append to skip at midnight. Close-then-open is the usual pattern: rename current to dated name, create a new empty current file, then let the accumulator write the next line. Document what happens to a line that arrives during the rename.
Practical checklist
- Date-stamp closed files. Do not rely on
realtimelog.txt.1. - Pick a boundary timezone and record it next to the accumulator’s clock policy.
- Keep daily extrema in the dayfile as the climate appendix; do not make the pruned realtime file carry that burden unless retention is long enough.
- Copy closed months off the logger disk if you quote them in public summaries.
- Alert before delete. A heartbeat can watch disk free space the same way it watches packet age; see server check for the freshness analogue. Disk is another stale-data cousin.
- Do not run recovered prune PHP that takes a path from the query string.
Modern relevance
SD cards in Raspberry Pi loggers still die from unbounded append. Object storage still needs prefixes that include dates. Browser charts that fetch “the last 24 hours” still need yesterday’s file at 00:10. The dotted historical path is awkward to read; the job it names is ordinary.
Connection-research archives that keep high-frequency series face the same retention arithmetic: completeness versus cost, and no silent deletion of extrema. TNET’s public methodology discusses source completeness without publishing internal storage layouts. This page stays on the station-scale rotator.
TNET research bridge
Retention policy is part of provenance. A series whose oldest month vanished without a note is not the series a later reader thinks it is. TNET’s data sources, quality controls, and methodology note is the modern counterpart of that discipline. This rotator article does not describe TNET internals.
Related pages: realtime log accumulation, the projects map, JPGraph figures, and the dayfile appendix. The Cumulus legacy hub lists the cluster.