1. I have purchased similiar products like this in the past with just average results. What makes your product so outstanding that I should buy it?
First of all, we believe that no other NCAA tournament analysis product or information service offers the same level of power, flexibility, and interactivity as BracketBrains, especially for the price.
Sure, you can go out and buy a 15-page long stat sheet that lists every number you can imagine regarding a matchup in question. All we can say is, good luck trying to process all of that data simultaneously and make a confident final decision before the NCAA tournament is over!
Likewise, various online tools can tell you that only 1.2% of #3 seeds from southwest Kansas with a bad coach have ever beaten a #15 seed from Dubuque that has three guards from Lithuania who eat ravioli for breakfast. We've all heard those types of statistical proclamations before.
Unfortunately, this kind of analytical approach is often completely unsound for predicting outcomes with a high level of accuracy. In most cases, the matchup circumstances to which they allude have only occurred a few times in all of NCAA tournament basketball history -- not nearly a sample size big enough to draw meaningful conclusions as to what will happen in the future under the same circumstances.
BracketBrains is much more intelligent. The technology behind it finds the delicate but critical balance between defining and identifying similar historical game scenarios and ensuring that there is enough relevant data available to make a confident and statistically sound prediction of game outcomes. Plus, the interface is so simple and easy to use that you can make smarter, more educated bracket picks in seconds.
Back to question list
2. What percentage of winners did BracketBrains produce last year?
Ah, the ever popular question. Sometimes it makes our blood boil, but we totally understand where this question comes from. After all, at the end of the day, we all want to win our bracket pools. Given that goal, is it worth it to invest your hard-earned cash in BracketBrains? Here is our perspective, and if you are seriously considering buying BracketBrains, we strongly encourage you to read it in full.
First, it's critical to understand what BracketBrains is and what it isn't. BracketBrains is not the world's best handicapping service. We can assure you that if we stumbled across a way to predict NCAA Tournament game winners with 80%+ accuracy, we would not be making a web site to sell it to you for twenty bucks.
In fact, we would not be making web sites or selling it to you at all; we'd be on the Amalfi Coast eating osso buco every night and donating billions to charity. Here is a good time to allude to the case of Bob Stoll, an analytically-minded handicapper recently featured in the Wall Street Journal, who is more than happy if he picks 58-59% of games correctly...and that means 58-59% of the small percentage of games for which he actually decides to issue recommendations!
With that said, there is no question that adding powerful data-driven analysis to your bracket picking process should increase your effectiveness at picking games. For example, the now-famous Billy Beane and his Oakland A's consistently outperform expectations over the long term by applying data-driven strategies to running their team. With BracketBrains, you can apply similar concepts to making your bracket picks.
However, just as the A's don't win the World Series every year, please don't buy BracketBrains and then send us hate mail if you don't win your 2008 bracket pool. The long term is measured in years, not games. Just keep in mind the odds. If you are in a bracket pool of 20 people, all things being equal, your odds of winning that pool are 5%. Repeat: All things being equal, you are expected to win that NCAA bracket pool once every 20 years. If you first win it in college, you shouldn't complain if you don't win it again until you're around 40 years old. Kind of sobering when you think about it, isn't it?
Even if BracketBrains triples the average person's effectiveness at making winning bracket picks, that person should expect to win a 20-person pool only about once every 6 or 7 years. So what you need to decide is how much the added edge that BracketBrains offers is worth, given what's at stake for you: the potential prize money, the bragging rights, your entire sense of self-esteem, etc.
At the end of the day, even with BracketBrains, you'll still need to make difficult calls on some closely matched teams, and the outcome of those games (and your luck at picking the winner) certainly will effect your chances at winning your 2008 NCAA bracket pool.
For instance, there will be games for which BracketBrains calculates Team A's odds to win at, say, 51.3% and Team B's odds to win at 48.7%. So is Team A the hands-down favorite? No way. Will you be throwing chairs around the room if you chose Team A as the winner based on what BracketBrains said now your entire bracket is crushed because they lost and you had them in the final game? If yes, please don't buy BracketBrains; you're not using your head. Probabilities are probabilities because they are not certainties.
The fact is, what BracketBrains is telling you is that it expects Team B, technically the underdog, still to win almost 49 out of every 100 games against the "favored" Team A. Some people simply refuse to believe that two teams can really be evenly matched -- that there is not some arcane form of mathematical sorcery out there that surely will reveal one team as the clear favorite. Newsflash: there isn't.
BracketBrains is not going to rock your world on every single bracket matchup by identifying a 65% or even a 55% odds winner, so please don't expect that. And we're not going to muck with the science behind it just to produce some sexy but statistically unsound predictions. (We'll leave that to most handicappers out there.)
Just remember one key facet of NCAA Tournament game picks, especially regarding bracket pools. Most handicappers out there look at ten or twenty games and then recommend two or three of those games as "opportunities." In bracket pools, you need to make a decision on every single game, even when two teams facing off are truly evenly matched. There is absolutely no way any sane, ethical person could sell a product that claimed to consistently pick the winners of every NCAA Tournament game, and some bad luck in the early round toss-ups easily can crush your chances of winning your 2008 pool.
In conclusion, winning any given year's tournament pool still requires luck, whether you use BracketBrains or not. But it takes a lot less luck to win when your odds are 1:7 versus 1:20.
But we digress; back to the question at hand. What percentage of winners did BracketBrains produce last year? It's impossible to say. Our users each generate their own predictions and then make their own calls on final bracket picks. We just design simple and easy ways to allow them to incorporate powerful data-driven analysis into the process. Of course, you're always welcome to read what our customers say.
Back to question list
3. Which BracketBrains factor settings had the most 'predictive power' in last year's NCAA Tournament?
The default pre-loaded settings are a rough recommendation, but it also depends on the specific matchup in question. In many cases, certain factors really stand out as being important. Here are a few examples:
- A #1 seed is playing a #16 seed. In this case, overall power rating, margin of victory power rating, and probably the seed trump everything else. Do other factors, such how far each team is traveling, matter too? Perhaps, but they're probably not the defining difference between the two teams in question. The bottom line is that a #1 seed and a #16 seed are almost certainly unevenly matched teams, and factors like overall power rating and seed are measures of exacly how big this mismatch is.
- A #8 seed from the Pac-10 is playing a #9 seed from the Big East in Philadelphia. In this case, we'd tend to weight highly both power ratings differences (overall and margin of victory) and distance traveled. Seed in this case doesn't really matter -- the Selection Committee does a decent job distinguishing a #1 seed from a #16 seed (such genius...), but an awful job distinguishing between 7, 8, 9, and 10 seeds. However, the fact that the Pac-10 team has to travel 2,000 to 3,000 miles more than their opponent, and may be playing in front of a hostile crowd as well, is likely to impact their play. (Historical data generally confirms this fact.)
- A very 'hot' and streaking #10 seed is playing a comparatively 'cold' #7 seed, in a location roughly halfway between each team's home court. Here, we'd likely want to combine power ratings analysis with momentum ratings. Weighting these settings highly would analyze how past teams (a) with similar statistical profiles to the streaking team and (b) that also have been on tears coming into the NCAA Tournament have fared against this specific type of opponent. Distance traveled (a wash) and seed (close enough) probably don't make much of a difference.
The magic of BracketBrains is that it automatically gathers the necessary data and performs all the complex calculations underlying targeted, relevant, and powerful analyses such as those outlined in the examples above. Just click a few buttons and BracketBrains does the rest!
Back to question list
4. Why don't you just figure out which factors matter the most in specific rounds/games (i.e. correlation coefficients) and use those to properly weight the different factors?
Fair question. We'd love to create a statistical model for that does exactly what you describe, but it's actually pretty difficult. We've built models that work really well for small datasets, and others that work very well for large datasets. With large datasets, there's a lot of room for different techniques that find the interplay between different variables. With small datasets, that's not possible. You really can only build something simpler, otherwise, you are bound to "overfit" the results.
The information collected on historical NCAA Tournament games definitely qualifies as a small dataset; hundreds, not millions, of past games have taken place. Trying to find small-scale trends using this dataset is consequently difficult or useless. For example, our less statistically savvy friends over at ESPN actually charged readers money for a "statistical" analysis that advised them to follow three basic rules for picking a winner in a scenario where two #1 seeds face each other in the Final Four. Well, it turns out that such a case has only occurred 9 times in NCAA Tournament history, so each "rule" has a historical sample of three.
As any statistician worth his/her salt will tell you, it's not difficult to find trends in a sample of three if you look hard enough. Just pick the team whose name starts with a consonant from the first half of the alphabet, and they'll probably have won those types of matchups 100% of the time.
Another statistically more responsible technique is a regression. Regression would assume that certain factors effect all teams the same way in all games. For example, traveling more than 500 miles will effect every team in the same way, regardless of when the game is played, what type of team it's played against, etc. Regression is a better method than trend-spotting, but it's still limited.
The technology behind BracketBrains is designed to leverage the best aspects of trend-spotting and regression while avoiding their major disadvantages. For example, BracketBrains can identify micro-trends (e.g., among teams with big power rating advantages that have to travel long distances) while simultaneously casting its data net at a level where it can draw meaningful conclusions.
So why don't we just do everything automatically? First of all, it would be a lot less interactive, informative, and fun for our users. In that case, our customers would simply be purchasing a completed bracket from us. More importantly, user-level configurability enables our customers to adjust the predictive algorithms to account for any additional information that they want to incorporate into their bracket decisions.
For example, even though a team may be playing a game 3,000 miles from home, you may know that they've actually been staying at a nearby hotel for the last five days. If you strongly believe that this fact will nullify any effects of long distance travel, then you can set BracketBrains to ignore distance traveled as a predictive factor. Or maybe that team's previously injured superstar has just returned to the squad, providing an emotional spark that you believe will decrease the relevance of that team's uncharacteristically poor performance in recent games.
These are the types of things that are all but impossible to quantify and model, or for which rich historical data is not available.
Back to question list
5. If win/loss performance in close games doesn't really matter all that much, as you stated elsewhere, then why is it included as a BracketBrains modeling factor?
On the whole, we haven't found win/loss performance in close games to be terribly important in predicting future outcomes. But there could exist certain types of matchups that feature a peculiar characteristic for which we have little or no historical data (e.g. if a heavily favored team has suddenly lost its best player to injury) that relates to this factor. Such a case could throw into question the relevance of the general conclusion that win/loss record in close games doesn't really matter.
If and when these cases come up, BracketBrains users can employ human judgment to decide whether to take into account win/loss performance in close games -- say, by increasing its importance if they think a close game may now be far more likely to occur given the favored team's unexpectedly injured superstar. But it's entirely up to you whether or not to have BracketBrains consider this factor.
Back to question list
6. Do you post your own bracket picks from past years so I can see how you did?
Sure. As it turns out, we've both benefited from using BracketBrains and its predecessor products. Disclaimer: Past performance is no guarantee of future results!
In 2005, Mike annihilated the competition in his 51-person NCAA Tournament Pool, recording the highest score ever in the pool's 13-year history. Last year, he placed second in a pool of 30+ people. Tom, on the other hand, has placed 2nd in his 15-person bracket pool for the past two years. We both used BracketBrains independently to come up with unique NCAA brackets. You can check out our past brackets here.
Please note that subscribers to BracketBrains 2008 Ultra, Pro, and Standard versions will receive access to our personal 2008 bracket picks, which we usually complete and post a day or two after Selection Sunday.
Back to question list
7. The TeamRankings.com 2006 end of season power rankings had UConn at #1. In BracketBrains, I tried every combination of factor weightings imaginable to generate a result that had UConn favored to win a matchup against Duke. However, BracketBrains predicted Duke as the favorite in every single scenario. In fact, even though Duke was rated #3 in your 2006 power rankings, according to BracketBrains, no team in the entire 2006 NCAA Tournament field would ever be favored to beat them. How can that be?
This is a fantastic question and kudos to 2006 BracketBrainiac RM who took us to the task and hammered on the product. In truth, several things can happen that explain this seemingly contradictory situation:
- Due to the mathematics involved, the power ratings that are used for BracketBrains are slightly different than what we post on the main TeamRankings.com site.
- Other BracketBrains factors rely heavily on historical NCAA tournament results, while our published in-season power rankings only take into context current season results. Statistically speaking, teams with Duke 2006's statistical profile have fared slightly better in the NCAA Tournament than teams that looked and played like UConn 2006 (although the difference between the two teams was small). As a result, Duke had a slight edge over UConn and a relatively larger edge over many other teams.
- The 2006 Duke team still had a better predictive power rating (different than the overall power rating) than UConn. That's very important. (You can read specific factor definitions AT A TBD LINK.)
Back to question list
8. Does the BracketBrains data set get updated during the NCAA Tournament? In other words, will later-round matchup comparisons reflect the actual outcomes of earlier-round games?
Regarding 2008 season related factors like overall power rating, the answer is no. In the context of more than a decade of historical data, the actual impact of adding another 32 games in real-time would be minimal, despite how alluring it sounds. Of course, the round of each game, the location where it will be played, and other such contextual factors are taken into account on a round-by-round basis. And recent performance quickly becomes a non-differentiator as the NCAA Tournament progresses; if Gonzaga and Texas are meeting in the Elite 8, obviously they have both won their last three games.
Current Vegas point spread information is updated multiple times per day in BracketBrains Ultra and Pro versions. However, the point spread factor is only taken into consideration for actual upcoming games. BracketBrains ignores this factor if no current point spread information for a matchup is publicly available.
Back to question list
9. Sometimes BracketBrains Ultra and Pro show seemingly conflicting results. How can a team have higher win odds if the expected margin of victory favors their opponent?
Although admittedly confusing, such a result is 100% legitimate and explainable from a statistical standpoint. First, BracketBrains independently computes the predictions for win odds and expected margin of victory. As a result, even though one team is predicted to have less than 50% odds of winning a game, historical data could show that if that team were to win the game, it is likely that they would do so by a wide margin of victory.
For example, say Ohio State plays Richmond three times. Richmond wins the first two games, one by 2 points and the other by 1 point. Then O-State drops the hammer and wins the third game by 21. If these three games represented a large enough data set to draw meaningful conclusions (of course, they don't...), then Richmond would have a predicted 66.7% odds to win a hypothetical fourth matchup, but the expected margin of victory would be Ohio State by 6 points. That's a highly simplified example, but you get the point.
ADD ADD ADD ADD does this mean upset potential???
The same logic holds for point spreads. Cases will occasionally arise where a team’s AVERAGE result is going to be worse than the spread, but their odds of beating the spread are less than 50%......
HOW DO I PICK UPSETS? MIKE WE SHOULD ADDRESS UPSETS
Back to question list
|
