The 60% Mirage: How Look-Ahead Bias Almost Tricked Us Into Promoting SP+
Analysis · by Rick's Picks Analytics
This is a story about the moment our methodology saved us from publishing a result that would have blown up in our faces -- and in the face of anyone who trusted it.
A few weeks ago we were testing whether Bill Connelly's SP+ ratings (one of the most respected publicly-available CFB power ratings) could be used to beat the spread. We built the hypothesis file, ran the backtest, and stared at the numbers: at high disagreement thresholds -- games where SP+ and the closing line differed by 10 or more points -- the SP+ side was hitting 60-62% against the spread on the test set.
Those numbers are absurd. A 60% ATS hit rate at -110 odds returns roughly +15% ROI. On out-of-sample data. That would be among the biggest betting edges ever published from public information.
It wasn't.
What Actually Happened
SP+ for a given season is computed FROM the games played in that season. The 2024 SP+ rating for Ohio State reflects all of Ohio State's 2024 results. When we joined SP+ data to games BY SEASON, we were effectively saying "given the team's full-season SP+ rating, did they cover Game X?"
But Game X is part of how the SP+ rating got computed. We were predicting the past with information from the future.
This is look-ahead bias, and it's the single most common way data scientists fool themselves into thinking they've found edge.
The Fix
We shifted the SP+ rating by one season. Now a team's 2022 SP+ rating predicts their 2023 games -- no information from the current season is used.
Here are the same tests after the fix (test window 2023-2025):
| SP+ Variant | Train Hit % | Test Hit % |
|---|---|---|
| Overall (prior-year) | 49.5% | 49.5% |
| Offense (prior-year) | 49.5% | 48.3% |
| Defense (prior-year) | 50.8% | 49.0% |
| Special teams (prior-year) | 50.1% | 53.4% |
The "edge" vanished completely. SP+ is right at coin-flip when used honestly. The closing line already prices it in.
(The special-teams row is the one test-only curiosity in the table -- it drifted up again in 2025 -- but its train hit rate is a dead-flat 50.1%, so under our rules it's a watch item, not a signal. A test-only trend with no train support is exactly the divergence pattern the methodology exists to filter.)
We then re-ran the same analysis with FPI (ESPN's rating, shifted to prior year): 49.7% / 50.0%. Same story.
Where Else Look-Ahead Hides
Season-level ratings are the most dramatic example, but look-ahead bias creeps into sports backtests through quieter doors, and we've had to seal each one explicitly:
- Conference assignments. College football realignment means a team's conference is a function of the season. If you tag 2016 games with 2024 conference membership (say, Texas as an SEC team), every conference-based angle inherits information from the future. All of our conference lookups run through a realignment-aware mapping keyed on team AND season.
- Rankings. "Was the favorite ranked?" must mean ranked going into that game -- the pregame AP poll -- not the team's final ranking or a rating computed after the season. Our backtests use only the pregame rank stored on the game row.
- Team stats. Any current-season aggregate (points per game, turnover margin, pace) computed over the full season leaks the future into every early-season game. Stats used as predictors must be prior-season or strictly game-windowed.
None of these produce a 62% mirage on their own, but they each add a fraction of a point of fake edge -- and fake edge compounds, because it survives every test except the one that matters: the future.
Why This Matters
If we'd missed the look-ahead, we would have:
- Promoted SP+ overall to the engine as a "+10 point edge" signal
- Built picks around it
- Watched users go 50% over the season instead of the promised 60%
- Quietly removed it later, hoping nobody noticed
Instead, the methodology caught it. Our promotion rules require sample sizes, train-test alignment, and -- critically -- documented joins. As soon as we flagged "SP+ for season Y joined to game in season Y," the look-ahead became visible.
We logged both the original (leaky) and corrected (honest) results in our integration log. The honest result is: all major team-rating systems are priced by the market. SP+, FPI, SRS, our own ELO walk, talent composite, recruiting -- they all hit ~50% on test ATS after look-ahead is removed.
The Lesson
Anyone who tells you SP+ (or FPI, or any other public rating system) beats the spread without showing you the join logic is either:
- Using look-ahead bias and either doesn't know it or isn't telling you
- Cherry-picking a sub-slice that worked retrospectively
- Selling you something
The 60% "edge" we initially saw isn't a Rick's Picks-specific result. It's reproducible by anyone who joins SP+ to games naively. Just be sure you also reproduce the test with prior-year ratings before claiming you've found something.
How Rick's Picks Guards Against This
Every hypothesis file in our data analysis layer has explicit documentation of: - Which seasons' data feed each feature - Whether features are shifted forward (no look-ahead) - Train (2015-2022) vs Test (2023-2025) usage -- the test window grows as each season completes (2025 was folded in as fresh holdout in July 2026; the SP+ and FPI coin-flip results held), and 2026 will be the next unseen holdout
We post our methodology publicly, we keep an audit log of every promote/reject decision, and we publish the test-set numbers for every signal we use.
We don't claim 60% ATS edges. We claim what the data actually supports -- which is much smaller, much rarer, and much more honest.
Rick's Picks publishes statistical analysis of college football for informational and entertainment purposes. Nothing here is betting advice. 21+. If gambling is affecting you or someone you know, call 1-800-GAMBLER.