Oct 5, 2026 · No measurable effect
Do resistance areas stop price?
- Tested
- Whether price that reached the nearest resistance area above the close, built from past swing highs only, then failed to close above it more often than chance.
- Control
- Random bands of the same width at a similar distance, overlapping no real area.
- Sample
- 136 tickers, daily bars 2024-07-01 to 2026-10-02, evaluated every 25 sessions from session 60: 1,277 areas reached, 1,227 random bands reached.
- Variants
- One specification, fixed before the run and not tuned on results. Re-run the same day after the display cap rose from 10 to 25 areas: identical results.
- Result
- Held (no close above within 5 sessions of reaching it inside 20) 33.4%, 427 of 1,277, against 33.3%, 408 of 1,227, for random bands: +0.2 points, 95% bootstrap range over tickers -3.2 to +3.5.
- Then
- Shipped as a map of where price turned before, with this result printed on every resistance page; nothing on those pages says an area holds.
- Record
- site/src/lib/resistance-core.ts:31-49 · docs/research/resistance-areas-research-2026-10-05.md:23-32
Where it shows: Support and resistance
Oct 21, 2026 · Did worse than its baseline
Does max pain land nearer the settle than no change?
- Tested
- Whether the max pain strike read before expiry lands within 0.5, 1 and 2 percent of the settlement price more often than a forecast of no change.
- Control
- The previous close: the spot at the last reading before expiry day.
- Sample
- 613 settlements on 20 tickers, about 258 independent once settlements on the same expiry date are grouped.
- Variants
- Three tolerances, 0.5, 1 and 2 percent, all reported.
- Result
- The previous close landed within the tolerance 30, 52 and 77 percent of the time against 23, 44 and 66 for max pain, and the 95 percent interval on each gap excludes zero.
- Then
- Max pain is still published as a calculation, and its accuracy page prints the no-change baseline beside it.
- Record
- commit c596c60f · commit 4ccd8816
Where it shows: Does max pain work?
Oct 21, 2026 · Could not tell it from its control
Does price reach liquidation clusters more often than mirrors?
- Tested
- Whether price reaches a liquidation cluster on our map within a fortnight more often than a mirror band at the same distance on the other side of the price.
- Control
- A mirror band at the same distance on the opposite side, graded with each claim.
- Sample
- 1,306 claims on 55 coins, grouped into 103 coin-fortnight blocks, because a coin’s daily claims share 13 of their 14 days.
- Variants
- Not recorded in the run note.
- Result
- Clusters minus mirrors: +3.4 points, 95% interval -1.1 to +8.3. The interval spans zero.
- Then
- The map still publishes; its pages say the record cannot yet tell clusters from mirrors, and the register keeps grading.
- Record
- commit 68c310a1
Where it shows: The liquidation map
Sep 20, 2026 · Changed what we test
Can a claim about the largest cluster be graded at all?
- Tested
- Whether the largest cluster on each map sits close enough to the price to be reached inside the grading window.
- Control
- How far BTC actually travels in a fortnight, over the venue’s own candles.
- Sample
- The register’s first 67 claims; 900 days of candles.
- Variants
- One check.
- Result
- The largest cluster sat a median 37.5% from the mark and only 2 of 67 were closer than 15%. BTC travels 30% inside a fortnight 2% of the time.
- Then
- A register of largest clusters would have spent almost its whole sample on claims that could not resolve, so the nearest cluster above and below the price were registered as separate claims.
- Record
- site/src/lib/liq-cluster-core.ts:44-58
Where it shows: The liquidation map
Jul 22, 2026 · Measured, with no outside baseline
How often does the projected earnings date land?
- Tested
- The earnings-date projection walked forward: each report projected only from the reports before it, by the same function the site runs.
- Control
- None. It was measured against the page’s own earlier, unmeasured claim of about 50% on the day and about 90% within a week.
- Sample
- 4,713 predictions across 131 tickers.
- Variants
- One run of the shipped estimator.
- Result
- Exact day 46.8%, within 1 day 61.6%, within 3 days 68.6%, within 7 days 91.3%; median error 0 days, mean signed bias +1.08 days.
- Then
- The page copy was rewritten to the measured figures in place of the unmeasured ones.
- Record
- docs/research/baselines.md:16-37 · collab/DECISIONS.md:176 · collab/DECISIONS.md:182
Jul 22, 2026 · Made results worse; not shipped
Would a flat +1 day fix the projection’s bias?
- Tested
- Whether adding one day to every projection, to cancel the measured +1.08-day mean bias, would raise exact-day accuracy.
- Control
- The same projections, their signed error split by the weekday projected.
- Sample
- 4,713 projections: Monday 424, Tuesday 1,220, Wednesday 1,010, Thursday 1,531, Friday 528.
- Variants
- One candidate. The offset itself was never run, because the split ruled it out.
- Result
- Mean signed error by projected weekday: Monday -2.29, Tuesday +0.56, Wednesday +3.23, Thursday +0.79, Friday +1.70. A flat +1 day would have made Monday worse; the average was carried by Wednesday.
- Then
- Rejected before shipping.
- Record
- docs/research/findings-from-site/E-001-global-plus-one-day-offset.md:10-33 · collab/DECISIONS.md:177
Jul 22, 2026 · Made results worse; not shipped
Quarter-aware week choice, first version
- Tested
- Replacing the weekday snap with a score that prefers the gap that followed the same quarter in earlier years, to fix projections that land a week early.
- Control
- The shipped estimator on the same predictions.
- Sample
- The walk-forward predictions of the estimator baseline (4,713).
- Variants
- One version; a second is the next entry.
- Result
- The +5 to +8 day cluster fell from 18.4% to 13.5%, but a -5 to -8 day cluster rose from 4.1% to 14.8%. Exact day fell from 46.8% to 40.0%, within 1 day from 61.6% to 53.4%, while the mean bias went from +1.08 to -0.01.
- Then
- Not shipped. The finding kept: a mean bias near zero can come with a worse model.
- Record
- docs/research/findings-from-site/E-002-quarter-week-selection.md:12-15 · collab/DECISIONS.md:179-180
Jul 22, 2026 · Made results worse; not shipped
Quarter-aware week choice, second version
- Tested
- A rare override to the later week, firing only when a company’s same-quarter history was large, tight and disagreed with the projection by exactly 7 days.
- Control
- The shipped estimator on the same predictions.
- Sample
- 4,713 predictions.
- Variants
- Four parameter settings, all reported.
- Result
- It fired on 5 to 11 predictions (0.11% to 0.23%); precision among the changed outcomes was 17% to 38%, below 50% at every setting, and the net exact-day change was negative at all four (-2 to -5).
- Then
- Not shipped. Two attempts bounded what reporting history alone can fix, and the work moved to reading the company’s own filings.
- Record
- docs/research/findings-from-site/E-002-quarter-week-selection.md:17-29 · collab/DECISIONS.md:180
Jul 22, 2026 · Beat the method it replaced; shipped
Can the confirmation parser stop confirming replay dates?
- Tested
- A filter that rejects a date attached to a replay, archive or transcript, or to "available through", before the earnings confirmation parser can confirm it.
- Control
- The same harness and the same documents before the change; only the parser changed.
- Sample
- 80 discovered scheduling documents.
- Variants
- One rule, tried on 8 sentences first, including a month-range trap.
- Result
- Replay or availability dates among confirmed dates fell from at least 1 to 0; 6 documents moved to a new availability-window bucket; confirmations went from 25 to 22, and the genuine all-dates-past count stayed at 35.
- Then
- Shipped. It can only reject a date, never newly confirm one. The parser’s precision against the dates companies actually reported has not been measured.
- Record
- docs/research/findings-from-site/E-003-availability-date-filter.md:14-30 · docs/research/baselines.md:41-61
Oct 12, 2026 · Beat the method it replaced; shipped
Is every earnings 8-K the quarter’s report?
- Tested
- A classifier that anchors each Item 2.02 filing on the company’s own 10-Q or 10-K, so a pre-announcement no longer counts as the quarter’s report.
- Control
- The previous engine, walked forward on the same actual report dates.
- Sample
- The 142-ticker universe and 1,518 more calendar filers; 60 randomly drawn filings read by hand; 58,843 points compared with the prototype.
- Variants
- One rule, revised once: the hand check found 3 delinquent-filer releases wrong in the first cut, and they were fixed.
- Result
- Universe: within 7 days 87.4% to 90.9%, misses over 30 days 361 to 192, 178 projections improved and 5 harmed. Other filers: 85.2% to 89.0%, misses over 30 days 4,660 to 2,562, 2,573 improved and 105 harmed. Hand check: 60 of 60 correct after that fix.
- Then
- Shipped.
- Record
- commit abdca87a · site/src/lib/earnings.ts:493-497
Where it shows: Earnings calendar
Oct 12, 2026 · Our published figure was overstated
Our published calibration figure was too high
- Tested
- The within-7-days rate on the calibration page, re-scored with an honest walk-forward: only filings known before each report.
- Control
- The published figure, scored on histories with pre-announcements removed in hindsight.
- Sample
- 4,754 points as published; 4,725 points re-scored, 343 filings set aside.
- Variants
- One re-score.
- Result
- Published: 91.4% within 7 days. Honest walk-forward: 87.4% for the engine that figure described, 90.9% for the replacement.
- Then
- The page moves to the honest figures as stored filing histories arrive.
- Record
- commit abdca87a
Where it shows: The confidence backtest
Aug 20, 2026 · Measured, with no outside baseline
Do our high, medium and low labels on earnings dates mean anything?
- Tested
- Whether projections labelled high land nearer the real date than those labelled medium or low, walked forward.
- Control
- The tiers against each other; no outside baseline.
- Sample
- 4,778 scored points across 136 companies, 2005 to 2026: high 4,662, medium 116, low none.
- Variants
- One rule, the one the engine runs; it agreed with the live engine on 139 tickers, 0 disagreed.
- Result
- High: 69.0% within 3 days, 91.5% within 7. Medium: 62.9% and 87.9%. Low: no scored points, because scoring starts at a company’s fifth report. 97.6% of scored points carried a single label.
- Then
- Shipped saying the labels ordered correctly, with the single-label caveat printed as prominently. Withdrawn in part on 2026-10-21 (below).
- Record
- ARCHITECTURE.md:1709-1736 · collab/DECISIONS.md:242
Where it shows: The confidence backtest
Oct 21, 2026 · Claim withdrawn
Does the high label beat medium once companies are resampled?
- Tested
- Whether high landing nearer than medium survives a resample of companies, with the claim allowed only if it holds in 95% of resamples.
- Control
- The medium tier, under 2,000 seeded resamples of companies.
- Sample
- 4,718 reports, about 1,312 independent once a company’s reports are grouped.
- Variants
- One test.
- Result
- High 90.9% against medium 89.2% within 7 days; the ordering held in 71% of 2,000 resamples.
- Then
- The page now says it cannot claim the labels order correctly.
- Record
- commit f465778d · commit 4ccd8816
Where it shows: The confidence backtest
Oct 21, 2026 · Measured, with no outside baseline
Live earnings windows, first grade
- Tested
- The projection standing the day before each report, logged as it was published from 2026-08-19, graded against the report.
- Control
- None.
- Sample
- 28 reports on 28 companies, past the gate of 20 reports on 20 companies.
- Variants
- One rule.
- Result
- 75.0% landed inside the published window, interval 57.1 to 89.3%.
- Then
- Published on the confidence page and graded as reports settle.
- Record
- commit f465778d
Where it shows: The confidence backtest
Nov 2, 2026 · Measured, with no outside baseline
Is the journal’s luck band as wide as it says?
- Tested
- Whether a 90% band drawn from a pool of 20 trades covers what the trader’s whole record produces, and how often real stretches of 20 fall outside it.
- Control
- Draws from the whole log, and its real 20-trade stretches.
- Sample
- One real log of 444 trades (dollar results, no stops); 22 stretches of 20.
- Variants
- One method.
- Result
- A band drawn from 20 trades held a median 63% of what the whole log produced, not 90%; real stretches fell outside it 4 times in 22, about twice what independent draws give.
- Then
- Shipped with the caveat printed in the panel, which leads with the middle half and says the band runs narrow.
- Record
- site/src/components/journal/JournalLuck.tsx:9-14 · site/scripts/verify-journal-luck.mts:100-104
Where it shows: Trading journal