Benchmark
How we know a version really predicts instead of just describing the past. Versions are frozen in code, settings are tuned on older seasons only, and the ranking uses the holdout season that nothing was tuned on. The forward test is the final judge: picks saved before tip-off, never recomputed.
Forward test · picks logged before tip-off
| # | Version | Settled picks | Hit rate | Bets won | Brier | Waiting | Logging since |
|---|---|---|---|---|---|---|---|
| 1 | Elo on top of roster strength v9-roster-strength | 81 | 60.5% ± 10.6% | 67.3% of 55 | 0.221 | 1119 | 20 Oct 2026, 12:03 |
| 2 | v7 with a lighter injury weight v8-lighter-injuries | 81 | 64.2% ± 10.4% | 66.7% of 54 | 0.226 | 1119 | 20 Oct 2026, 12:03 |
| 3 | v6 with a smaller home court v7-smaller-home-court | 81 | 64.2% ± 10.4% | 66.7% of 54 | 0.227 | 1119 | 20 Oct 2026, 12:03 |
| 4 | Elo + rest + missing players v3-elo-players | 81 | 63.0% ± 10.5% | 64.6% of 65 | 0.237 | 1119 | 20 Oct 2026, 12:03 |
| 5 | Elo + back-to-backs v2-elo-rest | 81 | 60.5% ± 10.6% | 64.3% of 70 | 0.238 | 1119 | 20 Oct 2026, 12:03 |
| 6 | Elo + rest + travel + injury report v5-elo-injury-report | 81 | 61.7% ± 10.6% | 66.2% of 65 | 0.239 | 1119 | 20 Oct 2026, 12:03 |
| 7 | Elo v1-elo | 81 | 58.0% ± 10.7% | 65.2% of 69 | 0.239 | 1119 | 20 Oct 2026, 12:03 |
| 8 | Elo + rest + travel v4-elo-travel | 81 | 60.5% ± 10.6% | 65.2% of 69 | 0.240 | 1119 | 20 Oct 2026, 12:03 |
| 9 | Elo + rest + injury report (forecast only) v6-elo-report-forecast-only | 81 | 63.0% ± 10.5% | 65.7% of 70 | 0.241 | 1119 | 20 Oct 2026, 12:03 |
| 10 | Better record baseline-record | 81 | 60.5% ± 10.6% | 61.1% of 36 | 0.244 | 1119 | 20 Oct 2026, 12:03 |
| 11 | Always home team baseline-home | 81 | 51.9% ± 10.9% | — | 0.251 | 1119 | 20 Oct 2026, 12:03 |
Every sync logs the next games for every version. Results fill in as games finish.
Forward test · against the bookmakers
| Version | Our Brier | Bookmakers' Brier | Our bets ROI | Edge bets ROI | Games |
|---|---|---|---|---|---|
| Elo on top of roster strength v9-roster-strength | 0.237 | 0.202 | -14.0% on 30 | -42.8% on 20 | 46 |
| v7 with a lighter injury weight v8-lighter-injuries | 0.240 | 0.202 | -13.4% on 29 | -31.6% on 23 | 46 |
| v6 with a smaller home court v7-smaller-home-court | 0.241 | 0.202 | -10.3% on 28 | -31.6% on 23 | 46 |
| Elo + rest + missing players v3-elo-players | 0.245 | 0.202 | -13.0% on 35 | -21.1% on 33 | 46 |
| Elo + back-to-backs v2-elo-rest | 0.247 | 0.202 | -10.2% on 39 | -22.1% on 37 | 46 |
| Elo + rest + travel + injury report v5-elo-injury-report | 0.244 | 0.202 | -10.5% on 34 | -15.0% on 37 | 46 |
| Elo v1-elo | 0.248 | 0.202 | -10.2% on 39 | -24.2% on 36 | 46 |
| Elo + rest + travel v4-elo-travel | 0.247 | 0.202 | -7.8% on 38 | -21.5% on 36 | 46 |
| Elo + rest + injury report (forecast only) v6-elo-report-forecast-only | 0.250 | 0.202 | -7.8% on 38 | -19.1% on 37 | 46 |
| Better record baseline-record | 0.230 | 0.202 | +16.1% on 16 | +12.8% on 15 | 46 |
| Always home team baseline-home | 0.248 | 0.202 | — | -22.1% on 18 | 46 |
Bookmakers' Brier uses closing odds with the margin removed. ROI = profit per 1€ staked at closing odds. Edge bets = games where we give our pick at least 3 points more chance than the bookmakers do.
Holdout benchmark
Runs are started from the development machine with npm run benchmark.
Latest run · #3
| # | Version | Tuning hit rate | Holdout hit rate | Holdout Brier | Back-to-back games | Versus next |
|---|---|---|---|---|---|---|
| 1 | Elo v1-elo | 64.6% ± 2.6% | 67.5% ± 2.5% 1322 games | 0.211 | 64.1% ± 5.4% 301 games | too close to call vs v6-elo-report-forecast-only |
| 2 | Elo + rest + injury report (forecast only) v6-elo-report-forecast-only | 67.7% ± 2.5% | 66.5% ± 2.5% 1322 games | 0.211 | 61.8% ± 5.5% 301 games | too close to call vs v2-elo-rest |
| 3 | Elo + back-to-backsLIVE v2-elo-rest | 64.8% ± 2.6% | 67.8% ± 2.5% 1322 games | 0.211 | 65.1% ± 5.4% 301 games | too close to call vs v4-elo-travel |
| 4 | Elo + rest + travel v4-elo-travel | 64.0% ± 2.6% | 67.2% ± 2.5% 1322 games | 0.211 | 64.5% ± 5.4% 301 games | too close to call vs v5-elo-injury-report |
| 5 | Elo + rest + travel + injury report v5-elo-injury-report | 66.6% ± 2.5% | 66.0% ± 2.6% 1322 games | 0.214 | 60.8% ± 5.5% 301 games | too close to call vs v3-oracle-lineups |
| 6 | Perfect injury reportBACKTEST ONLY v3-oracle-lineups | 66.7% ± 2.5% | 66.9% ± 2.5% 1322 games | 0.214 | 65.8% ± 5.4% 301 games | too close to call vs v3-elo-players |
| 7 | Elo + rest + missing players v3-elo-players | 64.5% ± 2.6% | 66.3% ± 2.5% 1322 games | 0.218 | 63.5% ± 5.4% 301 games | too close to call vs baseline-record |
| 8 | Better record baseline-record | 64.0% ± 2.6% | 66.7% ± 2.5% 1322 games | 0.221 | 66.4% ± 5.3% 301 games | clearly better than baseline-home |
| 9 | Always home team baseline-home | 54.5% ± 2.7% | 55.5% ± 2.7% 1322 games | 0.247 | 51.5% ± 5.6% 301 games | — |
Ranked by holdout Brier score. "Clearly better" means the gap holds on the same games with 95% confidence; "too close to call" means the difference could be luck.
Latest run · against the bookmakers
No closing odds saved for these games yet. Add a free key from the-odds-api.com to .env and sync daily.
History
#318 Oct 2026, 21:15Run 6: after injury report fixv1-elo · 67.5% · too close to call
Tuning 2024-10-01 → 2025-10-01 · Holdout 2025-10-01 → 2026-07-31
| # | Version | Tuning hit rate | Holdout hit rate | Holdout Brier | Back-to-back games | Versus next |
|---|---|---|---|---|---|---|
| 1 | Elo v1-elo | 64.6% ± 2.6% | 67.5% ± 2.5% 1322 games | 0.211 | 64.1% ± 5.4% 301 games | too close to call vs v6-elo-report-forecast-only |
| 2 | Elo + rest + injury report (forecast only) v6-elo-report-forecast-only | 67.7% ± 2.5% | 66.5% ± 2.5% 1322 games | 0.211 | 61.8% ± 5.5% 301 games | too close to call vs v2-elo-rest |
| 3 | Elo + back-to-backsLIVE v2-elo-rest | 64.8% ± 2.6% | 67.8% ± 2.5% 1322 games | 0.211 | 65.1% ± 5.4% 301 games | too close to call vs v4-elo-travel |
| 4 | Elo + rest + travel v4-elo-travel | 64.0% ± 2.6% | 67.2% ± 2.5% 1322 games | 0.211 | 64.5% ± 5.4% 301 games | too close to call vs v5-elo-injury-report |
| 5 | Elo + rest + travel + injury report v5-elo-injury-report | 66.6% ± 2.5% | 66.0% ± 2.6% 1322 games | 0.214 | 60.8% ± 5.5% 301 games | too close to call vs v3-oracle-lineups |
| 6 | Perfect injury reportBACKTEST ONLY v3-oracle-lineups | 66.7% ± 2.5% | 66.9% ± 2.5% 1322 games | 0.214 | 65.8% ± 5.4% 301 games | too close to call vs v3-elo-players |
| 7 | Elo + rest + missing players v3-elo-players | 64.5% ± 2.6% | 66.3% ± 2.5% 1322 games | 0.218 | 63.5% ± 5.4% 301 games | too close to call vs baseline-record |
| 8 | Better record baseline-record | 64.0% ± 2.6% | 66.7% ± 2.5% 1322 games | 0.221 | 66.4% ± 5.3% 301 games | clearly better than baseline-home |
| 9 | Always home team baseline-home | 54.5% ± 2.7% | 55.5% ± 2.7% 1322 games | 0.247 | 51.5% ± 5.6% 301 games | — |
#218 Oct 2026, 21:02Run 5: v6 forecast-only injury reportv1-elo · 67.5% · too close to call
Tuning 2024-10-01 → 2025-10-01 · Holdout 2025-10-01 → 2026-07-31
| # | Version | Tuning hit rate | Holdout hit rate | Holdout Brier | Back-to-back games | Versus next |
|---|---|---|---|---|---|---|
| 1 | Elo v1-elo | 64.6% ± 2.6% | 67.5% ± 2.5% 1322 games | 0.211 | 64.1% ± 5.4% 301 games | too close to call vs v2-elo-rest |
| 2 | Elo + back-to-backsLIVE v2-elo-rest | 64.8% ± 2.6% | 67.8% ± 2.5% 1322 games | 0.211 | 65.1% ± 5.4% 301 games | too close to call vs v4-elo-travel |
| 3 | Elo + rest + travel v4-elo-travel | 64.0% ± 2.6% | 67.2% ± 2.5% 1322 games | 0.211 | 64.5% ± 5.4% 301 games | too close to call vs v6-elo-report-forecast-only |
| 4 | Elo + rest + injury report (forecast only) v6-elo-report-forecast-only | 67.7% ± 2.5% | 66.0% ± 2.6% 1322 games | 0.214 | 63.1% ± 5.5% 301 games | too close to call vs v3-oracle-lineups |
| 5 | Perfect injury reportBACKTEST ONLY v3-oracle-lineups | 66.7% ± 2.5% | 66.9% ± 2.5% 1322 games | 0.214 | 65.8% ± 5.4% 301 games | too close to call vs v3-elo-players |
| 6 | Elo + rest + missing players v3-elo-players | 64.5% ± 2.6% | 66.3% ± 2.5% 1322 games | 0.218 | 63.5% ± 5.4% 301 games | too close to call vs v5-elo-injury-report |
| 7 | Elo + rest + travel + injury report v5-elo-injury-report | 66.2% ± 2.6% | 64.9% ± 2.6% 1322 games | 0.220 | 63.5% ± 5.4% 301 games | too close to call vs baseline-record |
| 8 | Better record baseline-record | 64.0% ± 2.6% | 66.7% ± 2.5% 1322 games | 0.221 | 66.4% ± 5.3% 301 games | clearly better than baseline-home |
| 9 | Always home team baseline-home | 54.5% ± 2.7% | 55.5% ± 2.7% 1322 games | 0.247 | 51.5% ± 5.6% 301 games | — |
#118 Oct 2026, 20:38First real NBA datav1-elo · 67.5% · too close to call
Tuning 2024-10-01 → 2025-10-01 · Holdout 2025-10-01 → 2026-07-31
| # | Version | Tuning hit rate | Holdout hit rate | Holdout Brier | Back-to-back games | Versus next |
|---|---|---|---|---|---|---|
| 1 | Elo v1-elo | 64.6% ± 2.6% | 67.5% ± 2.5% 1322 games | 0.211 | 64.1% ± 5.4% 301 games | too close to call vs v2-elo-rest |
| 2 | Elo + back-to-backsLIVE v2-elo-rest | 64.8% ± 2.6% | 67.8% ± 2.5% 1322 games | 0.211 | 65.1% ± 5.4% 301 games | too close to call vs v4-elo-travel |
| 3 | Elo + rest + travel v4-elo-travel | 64.0% ± 2.6% | 67.2% ± 2.5% 1322 games | 0.211 | 64.5% ± 5.4% 301 games | too close to call vs v3-oracle-lineups |
| 4 | Perfect injury reportBACKTEST ONLY v3-oracle-lineups | 66.7% ± 2.5% | 66.9% ± 2.5% 1322 games | 0.214 | 65.8% ± 5.4% 301 games | too close to call vs v3-elo-players |
| 5 | Elo + rest + missing players v3-elo-players | 64.5% ± 2.6% | 66.3% ± 2.5% 1322 games | 0.218 | 63.5% ± 5.4% 301 games | too close to call vs v5-elo-injury-report |
| 6 | Elo + rest + travel + injury report v5-elo-injury-report | 66.2% ± 2.6% | 64.9% ± 2.6% 1322 games | 0.220 | 63.5% ± 5.4% 301 games | too close to call vs baseline-record |
| 7 | Better record baseline-record | 64.0% ± 2.6% | 66.7% ± 2.5% 1322 games | 0.221 | 66.4% ± 5.3% 301 games | clearly better than baseline-home |
| 8 | Always home team baseline-home | 54.5% ± 2.7% | 55.5% ± 2.7% 1322 games | 0.247 | 51.5% ± 5.6% 301 games | — |
Frozen versions
Picks the home team every game. Anything worth using must beat this.
Picks the team with the better win rate this season.
Elo ratings with home court and margin of victory, no rest days.
v1 plus a 45 Elo point penalty for a team playing the day after a game.
v2 minus the value of regulars who sat out the team’s previous game (NBA.com box scores).
Same as v3 but knows exactly who played. Backtest only: the most an injury feed could add.
v2 plus distance since the last game, time zones crossed, 3 games in 4 nights and altitude.
v4 plus missing players from the official NBA injury report published before tip-off, falling back to v3’s guess when no report is usable.
v2 ratings learn as usual; the injury report adjustment only shifts the prediction, at 7.5 Elo per Game Score point (weight picked on the 2024-25 tuning season with npm run diagnose:players).
Same as v6, with 35 Elo of home court instead of 100. Picked on the 2019-20 warm-up season only (best Brier between 30 and 40, home teams won 55.2%), after npm run diagnose:market showed we gave home teams 62% while they won 55%.
Same as v7, with 5 Elo per Game Score point of missing players instead of 7.5. Picked on 2016-17 to 2019-20 from data/tuning.sqlite only (npm run tune): 5 was best with both actual lineups and last-game lineups, 7.5 was clearly worse with last-game lineups.
Each team also gets the summed value of its 8 most used current players (Game Score above replacement, slow-moving form), at 5 Elo per point; Elo learns only what the roster does not explain, with K 10. Missing players count 3 Elo per point. Trades and signings move a team as soon as the player has played once. Picked on 2016-17 to 2019-20 from data/tuning.sqlite only (npm run tune).