How Rick's Picks Decides What to Include in the Algorithm
Analysis · by Rick's Picks Analytics
There's a problem in sports betting analytics: everyone has an angle, almost nobody has proof. "Fade home favorites." "Back teams off a bye." "ACC games go under." These claims float around message boards and YouTube channels with no real backing.
At Rick's Picks, we've built a discipline around answering one question before any new signal enters the algorithm: does this edge actually exist, or am I fooling myself?
This post explains the principles, without giving away the recipe.
We Test Every Idea Against a Decade of Data
Rick's Picks uses 11 seasons of FBS-vs-FBS game data (2015-2025) as its testing ground. Every hypothesis -- "this stat predicts ATS outcomes," "this situation creates an edge," "this weather pattern moves totals" -- gets the same treatment: run it through the full historical dataset, see whether it produces a real, statistically-meaningful pattern, and decide if that pattern is large enough to overcome the bookmaker's vig.
The Standard: Beat the Vig, Not Just the Coin Flip
Standard sportsbook odds (-110) require you to hit 52.4% just to break even. Anything below that is a losing position over time, no matter how good it feels. Our internal bar is higher: a signal has to clear that threshold in our historical testing AND survive a separate set of seasons that the analysis never saw during development. If it can't hold up on unseen data, it doesn't go in the engine.
And the test never ends. Each time a season completes, we re-score every signal -- the ones we promoted and the ones we rejected -- against the new games. When the 2025 season went into the books, every live signal survived the re-check, and we still trimmed the parts of our own numbers that didn't replicate. Nothing is grandfathered in.
The Wall Between Past and Future
The single most important structural choice in our testing is the temporal split. Every hypothesis is developed and fit exclusively on the 2015-2022 seasons -- the training window. It is then scored, untouched, on 2023 through 2025 -- the test window. The two windows never mix, and the split is by time rather than by random sampling, because a random split lets a model quietly learn October's results and "predict" September's.
Why does this matter so much? Because with eleven seasons of data and unlimited retries, you can find a pattern that fits the past perfectly and means nothing. The training window is where ideas are allowed to look good. The test window is where they have to stay good, on games that did not exist when the idea was formed. Most don't. A signal that hits 55% in training and 49% in test isn't "half right" -- it's a warning that the 55% was memorized noise.
Significance, Effect Size, and the Multiple-Comparisons Tax
Three statistical disciplines run under everything we publish:
- A significance test appropriate to the question -- comparing scoring averages, cover rates, or category counts each gets the right tool, not a one-size-fits-all number.
- Effect size alongside every p-value. With thousands of games, an effect can be statistically "significant" and still worth a quarter of a point -- far too small to overcome the vig. Significance without effect size is noise with a diploma.
- A correction for how many things we tried. Test twenty angles at the usual significance threshold and one will pass by pure luck. When a research file tests multiple related hypotheses, the bar tightens proportionally (the Bonferroni correction). This single rule has killed more tempting-looking angles than any other.
An Example of the Filter Working
Two weather hypotheses entered the same testing gauntlet with the same prior: bad weather suppresses scoring. On training data, precipitation looked like the stronger signal -- a bigger effect on totals and a smaller p-value than cold temperature. If we shipped signals based on how the training window looked, rain would be in our engine today.
Then the test window voted. The cold-weather effect held its direction and most of its size on unseen seasons, and got stronger again when 2025 was added. The precipitation effect flipped sign entirely -- games in the rain scored more than the market expected in the test years. One of those two windows was lying, and there's no way to know which, so the signal died. Cold went live; rain contributes exactly nothing to any pick. Same file, same prior, same math -- opposite verdicts. That's what the wall between past and future is for.
Why So Few Signals Make It
Most "edges" in football betting either: - Were random noise that looked like a pattern in a small sample - Were real once but the market has long since adjusted - Are mathematically real but too small to overcome the vig
We've tested a lot of ideas. The ones that survive are rare. When you see a signal driving a Rick's Picks recommendation, it's because the math actually backed it up over a full decade of college football -- not because someone read a tweet.
The Ledger: Keeping Score on Ourselves
Every decision -- promote, reject, defer -- goes into a permanent integration log with the numbers as they stood at decision time. When a signal goes live, its test-set performance is recorded at promotion, and it gets re-scored against every new season afterward. The ledger is also where we log the embarrassing stuff: the backtest that looked spectacular until we found the data leak, the legacy engine constants we audited and zeroed out, the near-misses that sit on a watch list because they look great in a small recent sample and show nothing in the large older one.
The ledger exists because memory is the enemy of honest analytics. Without a written record, every analyst slowly becomes a highlight reel of their own wins. With one, the failures stay visible, and the standard that killed them stays enforceable -- including against our own published findings.
What This Means for You
When Rick's Picks gives you a confident recommendation, there's specific historical evidence behind it. When the algorithm says "no play," it's because the math doesn't see an edge worth chasing. We'd rather sit out a game than push a pick we can't defend with data.
And to be explicit about what we are not saying: none of this is a promise of profit. Betting markets are brutally efficient, the edges that survive testing are small, and variance can swamp a real edge for months at a time. What we can promise is process -- documented tests, out-of-sample validation, and a public record of every verdict, negative results included.
That discipline -- saying no to ideas that don't survive the testing -- is the difference between a real analytics platform and a confident-sounding guess.
Rick's Picks publishes statistical analysis of college football for informational and entertainment purposes. Nothing here is betting advice. 21+. If gambling is affecting you or someone you know, call 1-800-GAMBLER.