Sign inRegisterCheck connection

Parsing Cumulus Station XML Safely

How to parse a Cumulus station XML export without XXE, unbounded files, missing elements, or silent unit mistakes. Distinct from the XML format article.

Back to Cumulus weather software guides

A Cumulus XML export is useful only after a program reads it. The historical path /parse_xml.php was that reader on the Cumulus host: PHP that opened a station XML file and turned named elements into page values or a downstream packet. This article is about safe parsing—what can go wrong when the file is large, incomplete, or hostile—and how a consumer should treat missing elements and unit attributes.

It is not a description of the export schema. Named elements versus web tags versus realtime.txt belong on Cumulus XML as a structured export. It is not a one-shot file migrator; converting units and date formats between station-software families belongs on convert1. The original PHP is not rehosted.

Historical context

Personal-weather websites in the Cumulus era often ran on inexpensive shared hosting. An extra template produced cumulusxml.xml (or a similar name) on a timer. A PHP script on the same host then called simplexml_load_file or DOMDocument and printed a temperature. That loop worked until the file was missing, the encoding was not what the parser assumed, a unit element was empty, or the XML contained a document type declaration the operator never intended to honor.

Community documentation for the dump itself is on the Cumulus Wiki XML webtags page. Parser behavior is defined by the XML specification and by the library you actually use—on historical TNET hosts, that was typically PHP’s libxml bindings.

Well-formed is not sufficient

XML 1.0 requires a well-formed document: one root, matching tags, a declared encoding that matches the bytes. A weather consumer needs more.

Scientific completeness. An observation record needs a value, a unit, and a valid time. A well-formed file that omits <unit> on pressure, or that puts the station clock in a free-text <description>, is incomplete. Completeness is a data-quality rule, not a parser error.

Encoding. The documented community dump used ISO-8859-1 in its XML declaration. Degree signs, non-breaking spaces inside CDATA unit fields, and later UTF-8 templates do not mix. If the HTTP Content-Type charset disagrees with the declaration, strict parsers may refuse the file; lax parsers will silently corrupt units. Prefer UTF-8 when you control the template, and require agreement between header and declaration when you do not.

DTDs are not part of the weather contract. Station XML does not need a document type declaration. If one is present, treat it as unexpected input, not as a schema to fetch.

External entities and resource exhaustion

XML allows a document to define entities that expand to text or that point at external resources. That feature is unrelated to weather. On a public parser it is a vulnerability class known as XML External Entity (XXE) processing: a crafted file can cause the parser to read local files, open network connections, or expand entities until memory is exhausted. The OWASP XXE prevention cheat sheet states the defensive rule directly: disable DTDs and external entities; if a DTD cannot be disabled, disable external entity resolution and external DTD loading.

For PHP’s libxml-based parsers (last checked 13 August 2026 against OWASP and current PHP docs): do not pass flags that expand entities or load DTDs (LIBXML_NOENT, LIBXML_DTDLOAD, LIBXML_DTDATTR, LIBXML_DTDVALID). PHP 8.0+, built against libxml 2.9+, refuses external entity loading by default; older PHP did not. libxml_disable_entity_loader() is deprecated as of PHP 8.0—prefer parser flags and, where needed, libxml_set_external_entity_loader() to refuse external loads. Disable network access during parse (LIBXML_NONET where available). A 2009 weather script is not safe by subject matter.

Do not ship an XXE payload as a “test.” The honest test is that a DOCTYPE is rejected or ignored and that parse time and memory stay bounded.

Oversized files are a separate denial-of-service. The wiki’s webtag dump was already tens of kilobytes. A file that has grown to megabytes of repeated elements, or that never ends, should fail a size check before a DOM is built. Cap the byte length to something larger than a legitimate Cumulus dump and far smaller than PHP’s memory limit. Stream with XMLReader if you must inspect a large file; do not load an unbounded string into SimpleXML “just this once.”

Missing elements and unit attributes

Cumulus XML items are records, not a rigid SQL row. Templates differ. MX adds tags. An operator can delete a section. The parser’s job is to fail loudly or to mark a field missing—not to invent a number.

Rules that keep the science honest:

  1. Absent node ≠ zero. Skip the field or emit an explicit missing token. Zero is a legal rain total and an illegal relative humidity.
  2. Read the unit with the value. In the community dump, units live in a sibling <unit> element, sometimes wrapped in CDATA with a leading space. In realtime.txt, units are separate positional fields (realtime.txt). If the unit is empty, the value is not comparable to another station.
  3. Do not unit-convert inside the XML parser unless that is a separately tested, labeled step. Conversion of C/F, hPa/inHg, mm/in belongs in a migrator with fixtures, which is the convert1 problem, not a side effect of simplexml_load_string.
  4. Time is an element, not the HTTP date. Date and time tags in the dump can be locale-shaped (dd/mm/yy, 17:40 on 05 September 2009 in the wiki example). Parse them with an explicit format. If the timezone is missing, the observation is not comparable to a METAR (UTC) or to another station.
  5. Unknown names are unknown. Do not map <item name="temp"> to a Weather Display variable by guesswork. Keep the Cumulus name until a documented map exists.

Leftover web tags (<#temp>) mean Cumulus did not process the template. That is not a parse error in XML terms if the characters are escaped; it is a publication error. Detect the <# pattern in text nodes and refuse to treat the file as observations.

A bounded ingest procedure

A consumer that wants current conditions from a Cumulus XML file can follow a short, boring procedure:

  1. Check that the path is a local file you intended to read, or an HTTPS URL you allow-listed. Do not accept a user-supplied filename.
  2. Reject the object if it exceeds a size cap or if it is older than the station’s maximum upload gap.
  3. Parse with DTDs and external entities disabled; collect libxml errors instead of printing them into HTML.
  4. Require a known root (for the community dump, weatherdata) and a version token if the template provides one.
  5. Extract a small allow-list of names: outside temperature, humidity, pressure, rain, wind speed, wind units, observation time. Ignore the rest until you need it.
  6. For each extracted value, store unit, raw text, and a numeric conversion only after the unit is recognized.
  7. Compare file mtime, the time tags inside the document, and wall-clock. If they disagree by more than the upload interval, label the record stale, not current.

That procedure is quality control. It is the same discipline TNET describes for public weather records on data sources, quality controls, and methodology: freshness, native units, and schema identity before any later use. How TNET later distinguishes observed and derived information in connection research is conceptual on how the service works; this page does not describe that assembly.

What not to restore

Do not copy a 2008 parse_xml.php onto a public host. Historical scripts often printed warnings into the output, loaded XML from a query parameter, or fetched remote URLs with no timeout. Those are hosting accidents, not Cumulus features. Prefer a file Cumulus already wrote, parsed with the bounds above, or MX’s documented JSON/HTTP interfaces. The Cumulus hub indexes related pages without offering the old PHP as a download.

Sources