Sundog · Augury

Do the gamblers beat the bureau?

Every day, thousands of dollars trade on tomorrow's high temperature in a dozen cities. Weather forecasting has a mature, publicly funded model stack — physics ensembles, statistical blends, the works. So here is a fair question: does the betting crowd's price carry any weather information the models miss? We scored the market against the model stack at matched information cutoffs, over three and a half years. It does — and the surprise is which forecaster turns out redundant.

The edge is in the final hours

Market forecast skill minus the National Blend of Models, by hours before the daily high. Lower is a bigger market advantage. Pooled over 7 cities, 2023–2026.

The market wins by a hair at long lead and roughly four times as much in the final hours — exactly where "it is already 88° at 2pm" bounds the day's high in a way the morning model run cannot. The crossover is the finding, not the average.

The augur reads tomorrow off the price. At the margin, the augur beats the bureau — but only where the bureau has stopped looking.

A town kept two weather-tellers. One was the Bureau, which each dawn nailed a careful notice to the well and did not change it. The other was the Market — a knot of traders by the well who re-shouted their odds every hour as the day wore on. At sunrise the two agreed almost exactly, for the Market mostly read the Bureau's notice aloud. But by mid-afternoon, with the heat already in the air and the notice hours stale, the traders knew something the dawn notice never could: today was already what it was. The Bureau was not wrong. It had simply stopped speaking while the Market kept listening. A parable, not a proof. The chart above is the real measurement.

The question, made fair

Prediction markets like Kalshi settle each city's daily-high contract on the National Weather Service's official climate report. So the market is not competing to measure the temperature — it competes to forecast it, against the same models everyone else has. The honest way to ask "does the crowd know more?" is to freeze the clock at a given minute and compare, at that instant, the market's implied odds against the freshest model guidance available at the very same minute. Snapshot both at 6am; snapshot both at 2pm. Anything less measures staleness, not skill.

We did that at every hour across the run-up to each day's high, in seven cities, from February 2023 through June 2026 — roughly 3,300 city-days. Ground truth is the settling climate report; scoring is done at exactly the temperature thresholds the market trades, so nobody gets a home-field metric.

The pantheon, and the one that folds

A single "market beats one model" result is weak — a lossy echo of that model produces it by luck. The real test puts the whole forecaster pantheon in one regression and asks which members are non-redundant: whose information survives once everyone else's is already accounted for. We ran the market against a physics ensemble (the GEFS), the operational statistical blend (the National Blend of Models), and a second independent model (the ECMWF).

Who is still in the room?

Each forecaster's marginal contribution with all others present (encompassing coefficient, 95% interval). Above zero = carries information no one else has.

the aggregate (market) a model, still non-redundant redundant / screened out

The minimal set of forecasters that survives is { National Blend, ECMWF, market }. The one member that folds is the raw physics ensemble — because the National Blend already post-processes exactly that kind of physics, so the raw ensemble adds nothing on top of it. When the whole model stack is in the room, it is a model that proves redundant, not the market.

Market and model, at opposite ends of the day

Splitting the same test by lead time sharpens it. The market is non-redundant at both ends but strongest in the final hours — its advantage is real-time observation the stale model cycle cannot carry. The ECMWF is the mirror image: it earns its place at long lead, where genuine model skill separates the forecasters, and fades in the final hours where the market's live reading takes over. The two non-blend members of the set carry their information at opposite ends of the horizon. That is an information-access result, not a curiosity: each forecaster wins exactly where it has access the others lack.

Where does the edge come from?

Two deflating explanations come free with the result above, and both are testable. One: the traders are just reading the current thermometer — by early afternoon the day's high is nearly in, and a price that tracks the latest observation would "beat" a morning model without knowing anything. Two: the traders are just front-running the models — anticipating where the next run will land, so their "extra information" is model information arriving a few hours early. We ran both as kill tests against the same ladder.

Two ways to explain the market away — neither works

The market's marginal contribution (encompassing coefficient, 95% interval) with each would-be explanation added to the regression. Zero = fully explained away.

Top: at short lead, a nowcast built from the latest official observation plus the climatological remaining rise joins the regression — if the market were a thermometer reader, its bar would collapse to zero. It stays at 0.72. Bottom: at long lead, the dawn market is scored against the blend's final same-day issuance, published hours later — if the market were front-running the models, the future model would absorb its edge. It stays at 0.66.

So the price is neither a thermometer reader nor a front-runner: it aggregates information that is genuinely independent of both the current observation and the model stack's best eventual answer. And the access pattern stays consistent when the nowcast joins the pantheon — the observation rung earns a seat only in the final hours, the independent model only at long lead, the market at both. Directionally, where a forecaster's information can enter is what decides whether it matters, more than which forecaster it is.

What this is and is not

It is an information claim, scored on raw market midpoints against operational model guidance at matched information cutoffs, with ground truth from the settling climate report.
It is not a trading strategy and not investment advice. Fees and the bid-ask spread eat the tails; a "tradable edge" is a separate, weaker claim this page does not make.
The comparators are the models' mean-and-spread rendered as a normal distribution, and the ECMWF rung in the headline analysis is its deterministic run. A follow-up sensitivity with the full 21-member ECMWF ensemble reproduced the same determining set, the same market result, and the same long-lead-only ECMWF membership. The robust backbone is the encompassing survival, not any single model's calibrated score.
The "human forecaster" rung (station MOS) is absent — no reliable historical same-day source was available — so the pantheon here is physics-ensemble, statistical-blend, and second-model, plus the market.
The horizon crossover is the headline; the exact per-hour map is a supporting exploratory view, not a separately adjudicated claim.
The provenance tests are the same information-claim tier: the observation nowcast is a named approximation (latest official observation plus a persistence-and-climatology remaining rise, fit on 2015–2019 data), and "the models' final word" means the blend's freshest same-day issuance. The "access decides membership" reading is directional — a five-forecaster pantheon is too thin to prove it as a law.

Why it is not obvious

The efficient guess is that the market is just a fast mirror of the models — arbitrage bots reprice within seconds of every model cycle, so the price should be a lossy shadow of the best public model, minus fees. That guess is mostly right, and it is exactly why the residual matters: after the models are fully accounted for, a non-zero, horizon-shaped signal remains. Richard Roll noticed the same shape forty years ago in orange-juice futures, which predicted errors in the government's Florida temperature forecasts. The market was reading the weather off the price. Here we can say where: at the margin, and in the final hours.

Sources & further reading

R. Roll, "Orange Juice and Weather", American Economic Review (1984) — the founding observation.
Forecast encompassing / matched-cutoff scoring in the prediction-market setting: cf. the FluSight-vs-market comparison (arXiv:2605.11220), the method template adapted here to weather; R. Gürkaynak & J. Wolfers, Macroeconomic Derivatives (2005) — market-based forecasts encompassing surveys, the same test in macro.
Model guidance: NOAA's National Blend of Models and GEFS; ECMWF open data. Ground truth: NWS Daily Climate Report via the Iowa Environmental Mesonet.
Sundog write-up: the Augury ledger and its pre-registration (internal until this page).

A Sundog evidence page. Real data, real ground truth, honest fences. Not investment advice.