What the parser does with real scheduling filings
A date on the earnings calendar marked confirmed was read from the company’s own filing by a parser, not taken from a vendor. This page publishes the latest stored run of the eval that feeds that parser real filings and counts what it decided. The eval is not rerun when you load this page: it is printed from the record of the run on 2026-07-22.
The population: 8-K filings from the 21 days before the run that match the three phrases the production sweep searches EDGAR full-text for ("conference call to discuss", "will report" with "financial results", "will release" with "financial results"). A confirmed share here is a share of filings the search found, not of companies on the calendar; the calendar figure is below and has a different denominator.
Every decision, by reason
| Decision | Filings | Share | What it means |
|---|---|---|---|
| Confirmed | 22 | 27.5% | a future date inside the window, within 240 characters of scheduling words such as "will report" or "conference call", in a filing that names earnings or results |
| All dates past | 35 | 43.8% | no date in the filing is a future date inside the window; most of these hold only the cover-page date, so they are the next recall work, not errors |
| Date not near scheduling language | 8 | 10.0% | the nearest date inside the window sits more than 240 characters from any scheduling words, too far to be trusted as the event |
| Availability window date | 6 | 7.5% | the only date inside the window is when a replay or archive stays available, not when the company reports (the E-003 rule) |
| No earnings keyword | 5 | 6.3% | none of the words financial results, quarterly results, earnings or results of operations appear |
| Future date outside the window | 4 | 5.0% | a future date appears, but outside the window of 3 days before the run to 150 days after it |
| Total | 80 | 100.0% |
The false confirmation it found, and the fix
E-003: a replay availability date ("available through July 27") was being confirmed as the earnings date. On the same filings, with only the parser changed, availability dates among the confirmations went from 3 to 0, confirmations went from 25 to 22, and the filings whose dates were all past stayed at 35 (from 35), so the fix took out false confirmations without losing a real one. The rule can only refuse a date, never newly accept one, which is why it could ship without a case by case review: it cannot add a new false confirmation.
On the calendar
At the sweep of 2026-07-22, 53 of 142 upcoming earnings events (37.3%) carried a date confirmed from a filing. The rest carry an estimate from the company’s own reporting history, labelled as an estimate, and the confidence backtest measures how close those estimates land.
How the eval works
- Discovery uses the same EDGAR full-text search the production sweep uses, so "did discovery find it" and "did the parser accept it" are measured on one population.
- Each document runs through the instrumented parser (lib/earnings diagnoseAnnouncement), which returns either a confirmation with the sentence it matched or an abstention with the rule that refused it.
- It reads and reports only; it writes nothing and changes no calendar state. It runs on request behind the cron secret because each document is an SEC fetch.
Not measured: the parser’s overall precision and recall against a hand-checked answer for every document
This page prints no precision or recall figure, because none has been measured. The reason: the eval counts decisions; it does not hold a truth label per filing, so it can show that the one known false-confirm class went to zero, not that every confirmation is right. What it needs: a hand-labelled set of scheduling filings with the true report date, scored by the same harness, with the sample size printed beside each rate.
Where this result is recorded
The run is written down in frozen baselines (docs/research/baselines.md) and finding E-003 (docs/research/findings-from-site/E-003-availability-date-filter.md) in the project’s repository, and the figures above are copied from there; a new run replaces them, a hand edit never does. The eval itself is the gated route /api/earnings-parser-eval?docs=80.
Every validation test we run, including the ones that found nothing, is listed in the file drawer. The fair value band has its own stored audit. Back to methodology.