The ingestion ledger

Why publish pipeline failures?

Because a run log that only shows successes is marketing. Every dated figure on this site is a claim that a pipeline ran; this page is where that claim can be checked. A commercial data vendor cannot publish its own failure record without undermining what it sells. We are not selling reliability, so we can, and a log with visible failures is the only kind worth trusting.

What does a failed run mean for the data I see?

Usually nothing visible. The house rule is that a failed ingest refuses to write, so pages keep serving the previous good snapshot with its original date. A failure here plus a stale date on a page is the system working as designed: showing you older data honestly rather than newer data invented.

Why do some datasets appear only in the freshness table?

Run-level recording arrived in stages, and some older pipelines write their dataset without writing a run record yet. Freshness is observed from the data itself, so the freshness table has no coverage gap; the run log covers the pipelines that record runs, and the gap between the two lists is stated rather than hidden.

What is not on this page?

Cache warmers, email and alert deliveries, and AI generation jobs. This ledger covers pipelines that write the datasets pages read. It also is not a service-level promise: it is a record of what happened, not a guarantee of what will.