Final placement: 89th of 4,013 forecasters (top ~2%) · peaked #11 · 3,410.67 points · beat the crowd on 67% of 1,196 markets — an average RBP gap the platform ranks "better than 81% of forecasters". A fully reproducible statistical forecasting system + live API bot for Jump Trading's Probability Cup — a Brier‑scored probability competition over the 2026 World Cup on SportsPredict.com, where the #1 finisher wins a paid 10‑week Fellowship at Jump Trading, Chicago.
This repository is the whole thing, in the open: the statistical models, the live REST‑API bot (manual lane + two de‑correlated "scenario" bots), the de‑vig‑the‑odds → add‑alpha → calibrate pipeline, a per‑game record of forecast‑vs‑crowd‑vs‑outcome for all 104 games, and an honest post‑mortem with per‑category P&L — including the one market family where the crowd beat me all tournament.
It is also a story about a climb and a comeback. I rose to #11 after the group stage, then a rough Round of 32 (two −200 games) and a steadily growing, sharpening field pushed my relative rank down to 157th — before the knockout run clawed it back to 89th of 4,013, gaining +17 places on the ×3‑weighted final alone, where the honest under‑priced read of a cagey 1‑0 game landed the best single‑lane result of the entire run (+433.77).
- The campaign at a glance
- What worked, and the one leak
- Are the probabilities honest? (calibration)
- The two‑bot "win‑one‑game" strategy
- Repository map
- Methodology
- Architecture
- Reproducing the results
- Tools I built
- Competition facts
- FAQ
- Author
- Acknowledgements
- Licence and citation
| Stage | Games | Multiplier | Stage RBP | Rank at stage end |
|---|---|---|---|---|
| Group | 1–72 | 1× | +1,841 | ~12 / 2,500 |
| Round of 32 | 73–88 | 1× | +112 | 129 |
| Round of 16 | 89–96 | 1× | +577 | 133 |
| Quarter‑finals | 97–100 | 1× | +429 | 115 / 413 |
| Semi‑finals | 101–102 | 2× | +142 | 111 / 439 |
| 3rd place + Final | 103–104 | up to 3× | +435 | 89 / 4,013 |
| Total | 104 | +3,536 | 89 / 4,013 |
Stage RBP = summed per‑question Relative Brier Points (our computed attribution). The official leaderboard score is 3,410.67 — the two differ slightly by platform timing/metric. Ranks are the global cumulative leaderboard position at each stage's end.
The Round‑of‑32 dip (+112 only) was two games — g077 Côte d'Ivoire‑Norway (−233) and g082
USA‑Bosnia (−189) — both the same mistake: over‑pricing high‑bar team shot volume. Strip those two
and the knockout run was strongly positive. The cumulative RBP curve never turned down.
The official SportsPredict standings and "My Performance" pages (the numbers above are all reproducible
from results/, but here's the receipts):
- 89th of 4,013 on the global leaderboard, 3,410.67 points (handle Dilbert).
- RBP gap vs crowd +2.9 — "better than 81% of forecasters", +3% confidence bias, over 1,157 settled forecasts on the manual lane.
- Boldest call that paid: goal before the first hydration break (you 74 / crowd 40 → YES, +64.4). Best match: the ×3 final, +314.6. Roughest match: Côte d'Ivoire‑Norway −232.8 — the same game the retrospective flags.
- The platform's own category view independently confirms the leak: Set Pieces & Possessions is my
only negative category, while Total shots on target is a top‑3 edge flagged −7% timid — exactly
what
results/RETROSPECTIVE.mdconcluded from the raw data.
Net Relative Brier Points by market family across all 1,196 markets:
| Market family | n | Net RBP | Beat‑crowd | Read |
|---|---|---|---|---|
| Player props (score/assist, SoT) | 233 | +957 | 71% | biggest edge — confident fades of rotated/benched players caught via live lineups |
| Shots on target | 10 | +230 | 90% | our sharpest category — we were even too timid on it |
| Goals combos / BTTS | 56 | +274 | 73% | crowd under‑prices multi‑condition YES |
| Territorial (corners/shots) | 167 | +430 | 62% | corner totals a real edge; high‑bar team‑SoT the tail risk |
| Goals totals / team‑scores | 162 | +628 | ~69% | de‑vig + model |
| Cards & discipline | 67 | +213 | 67% | pen‑or‑red over‑pricing fade works |
| Match result / winner | 119 | +366 | ~62% | de‑vig baseline |
| Offsides | 75 | −31 | 59% | |
| Fouls | 57 | −125 | 35% |
The honest bit: fouls beat the crowd only 35% of the time. Comparing‑fouls and offsides are the
one place I should have submitted the model price and never hand‑leaned — a rule I broke as late as the
×3 final (offsides 60 vs crowd 46 → −32 to −39 across all lanes). The fix is enforced in code by
scripts/set_pieces_audit.py --lint.
Reliability diagram over 1,159 forecasts with known outcomes: the points hug the diagonal (expected calibration error 0.036). The crowd is a near‑efficient baseline — its Brier (0.221) is essentially identical to mine (0.223). The edge is not in absolute accuracy; it's in relative scoring — out‑forecasting the field on the majority of markets is what Relative Brier Points reward — a 3,410.67‑point, 89th‑of‑4,013 finish with an average per‑forecast RBP gap the platform ranks "better than 81% of forecasters."
The platform's own calibration view agrees — stated confidence tracks actual outcomes across every probability bucket:
Each account may run two bots. Instead of pointing both at expected value, I ran them as two de‑correlated extreme‑scenario stacks — a favourite‑control/under lane and an upset/over lane — to maximise the chance that at least one lane nails each game's branch (a max‑order‑statistic play). The honest manual lane stayed model‑calibrated.
On the ×3 final (Spain 1‑0 Argentina, a.e.t.) the three lanes diverged hard and the Spain‑control / under bot hit the cagey‑final scenario for +433.77 — the best single‑lane result of the tournament. The same de‑correlation delivered in‑game top‑10 finishes earlier (g044 manual 2nd, g050 bots 2nd/4th, g061 bots 2nd/3rd) and caught upsets others missed (g099 +195, g102 +178, g103 +225).
probabilitycup-jumptrading/
├── README.md ← you are here
├── LICENSE ← MIT
├── CITATION.cff ← machine-readable citation
├── pyproject.toml ← installable; python -m bot run --offline --today
│
├── bot/ ← the forecasting bot
│ ├── sportspredict/ ← REST API client + config
│ ├── markets/ ← market-question parser (novel-type detection)
│ ├── models / pricer.py ← Dixon-Coles goals · Poisson/NB + Skellam counts · discipline · props
│ ├── data/ ← pluggable data layer (base rates, team stats, player SoT)
│ └── cli.py ← dry-run-by-default CLI
│
├── docs/ ← the playbook
│ ├── COMPETITION.md ← rules, scoring, prizes, API
│ ├── STRATEGY_AND_CALCULATIONS.md ← Brier/RBP math, de-vig, multipliers, checklist
│ ├── FORECASTING_MODEL.md ← the end-to-end pipeline
│ ├── DATA_SOURCES.md ← data-gap analysis + source taxonomy
│ └── plots/ · results/ ← the charts + hero/social images in this README
│
├── games/ 001–104 ← per-match research (H2H, form, squads, betting stats) + live questions
├── teams/ group-a…l ← per-nation source libraries + stats
├── referees/ 52 officials ← per-official tendency profiles (cards/fouls/pens) + assignments
├── stadiums/ 16 venues ← weather/roof/altitude profiles
│
├── results/ ← the truth ledger
│ ├── games/<NNN>.json ← forecast vs crowd vs outcome + RBP, all 3 lanes, per game
│ ├── lessons/ · LESSONS.md ← auto weak-point analysis
│ ├── leaderboard.json ← rank/points/gap tracker (104 games)
│ └── RETROSPECTIVE.md ← the full post-mortem
│
├── scripts/ ← tooling (charts, calibration, set-pieces lint, lessons)
└── workflows/ ← repeatable runbooks (score / enrich / referees / baseline)
Start here: results/RETROSPECTIVE.md (the story + the data) ·
docs/STRATEGY_AND_CALCULATIONS.md (the math) ·
bot/README.md (the code).
Scoring is a proper scoring rule (Brier), so the winning move is to report honest probabilities — you earn Relative Brier Points for being closer to the truth than the hidden crowd. Every market went through the same loop:
- De‑vig sharp market prices (Pinnacle / Betfair / a multi‑book consensus) and submit them as a crowd‑beating baseline on every market.
- Run the statistical models where they add signal — Dixon–Coles for goals, Poisson/Negative‑ Binomial + Skellam for counts (corners, cards, shots), dedicated handlers for discipline and player props.
- Add alpha only where I had real information the crowd lagged — confirmed lineups, injuries, referee tendencies, stadium weather, benched/rotated stars.
- Answer everything (ranking is cumulative), clamp to 1–99, and weight effort toward the knockouts (2×) and the final (3×).
- Settle → attribute → learn. After each game,
build_lessons.pyandset_pieces_audit.pyattribute the P&L by market family and flag any lean that fought the model — a calibration loop that turned the fouls/offsides leak into a hard rule.
The single most useful artifact for anyone entering a future Probability Cup is
results/RETROSPECTIVE.md + results/LESSONS.md:
every recurring mistake, with the RBP number that taught it.
flowchart LR
O["Sharp odds<br/>Pinnacle · Betfair · multi-book"] -->|de-vig| B["Market backbone<br/>(crowd-beating baseline)"]
M["Statistical models<br/>Dixon-Coles · Poisson/NB · Skellam · props"] --> B
I["Info edge<br/>lineups · injuries · weather · referees"] --> B
B --> E{"Ensemble +<br/>calibration"}
E --> L["Live REST-API bot<br/>manual + 2 de-correlated lanes"]
L --> R["Settled results<br/>results/games/*.json"]
R -->|build_lessons · set_pieces_audit| K["Lessons + category P&L"]
K -.->|feeds back| M
No API key required — the bot scores from local data in offline mode.
git clone https://github.com/DataAthleteChamp/probabilitycup-jumptrading
cd probabilitycup-jumptrading
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt # or: pip install -e .
# Score today's games from local data (no key, dry-run):
PYTHONPATH=. python -m bot run --offline --today
# Regenerate every chart in this README from results/ data:
PYTHONPATH=. python scripts/make_portfolio_charts.py
PYTHONPATH=. python scripts/make_hero.py
# Calibration / Brier report, and the set-pieces board lint:
PYTHONPATH=. python scripts/calibration_report.py
PYTHONPATH=. python scripts/set_pieces_audit.py --lint predictions/baselines/<board>.jsonLive submission is opt‑in and dry‑run by default — it only fires with an explicit --submit
flag and a real SPORTSPREDICT_API_KEY in bot/.env (see bot/.env.example).
scripts/make_portfolio_charts.py— every chart in this README, generated fromresults/(rank, cumulative RBP, category P&L, calibration, stage, bot lanes).scripts/set_pieces_audit.py— per‑category RBP attribution and a--lintthat flags over‑leaned set‑piece markets before submission (the fix for the one leak).scripts/build_lessons.py— re‑prices every settled game with the current model and auto‑writes weak‑point lessons →results/LESSONS.md.scripts/calibration_report.py— Brier / reliability report.scripts/backtest.py— replays the model over all settled games.- Plus the enrichment/readiness runbook tooling in
workflows/.
- Free to enter; ran 2026‑06‑11 → 2026‑07‑19; 104 matches, 1,196 binary markets scored.
- Predictions are integers 1–99; the crowd price is hidden; scoring is relative Brier.
- Up to 2 bots per account via REST or MCP; knockouts count 2×, the final 3×.
- #1 wins a paid 10‑week Fellowship at Jump Trading (Chicago). Official site: https://sportspredict.com.
Why did the rank fall from #11 to 157th while points kept rising?
Two reasons: a rough Round of 32 — two −200 games (g077, g082) where I over‑priced high‑bar team
shot volume — and a growing, sharpening field (entrants kept joining and climbing). My absolute
points never stopped rising; the relative rank compressed, then the knockout run climbed it back to
89th of 4,013. See the two charts above.
What's the one thing you'd fix first next time?
The set‑pieces leak. fouls (−125, 35% beat‑crowd) and offsides (−31) are the only net‑negative
families — and the platform's own category view independently flags Set Pieces & Possessions as my
only negative category. The rule: submit the model price on foul/offside markets, never hand‑lean —
now enforced by set_pieces_audit.py --lint.
Is 89th of 4,013 good? Top ~2% of the field. It's not the Fellowship, but the system beat the crowd on 67% of markets with genuine calibration (the platform ranks my average RBP gap "better than 81% of forecasters") — which is the point of the repo.
How is the "beat the crowd" claim not just luck? 1,196 markets is a large sample, the reliability diagram shows honest calibration, and the edge is concentrated in structurally‑explained families (player props, shots on target) rather than spread noise. The fouls leak — where the crowd beat me — is shown just as prominently as the wins.
Did you really submit live to an API?
Yes — the bot fired to the SportsPredict REST API across the tournament (manual lane hand‑entered, two
bots via update_prediction/submit_batch), always dry‑run by default and verified 15/15 or 20/20
before each lock. The client is in bot/sportspredict/.
Jakub Piotrowski — @DataAthleteChamp · LinkedIn
Sibling quant repo: dtu-quant-lab-imc-prosperity-4 (28th of 18,803 in IMC Prosperity 4).
To Jump Trading and SportsPredict for running the Probability Cup, and to the open forecasting
and sports‑analytics community whose de‑vigging, Dixon–Coles and calibration write‑ups shaped the
pipeline. Third‑party odds feeds and the competition's proprietary data are not redistributed here —
only my own forecasts, code, and derived analysis. See docs/COMPETITION_TERMS.md.
Released under the MIT licence. If you reference this work, the recommended citation is in
CITATION.cff.
Peaked at 11. Fell to 157. Climbed back to 89 on the game that counted triple. The crowd is hard to beat — but it is beatable, one honest probability at a time.