Statistical models are created by identifying influential random variables and then assigning characteristics to those variables. In the case of football, how many rushing yards the 49ers will gain is a random variable. So is the number of interceptions the Ravens will make. The formulation of a statistical model involves identifying the most important of these random variables and assigning probability distributions to them, complete with specified means and standard deviations.
Selecting the most important random variables is a reasonable, even if artful, undertaking. The next step, assigning distribution characteristics to these variables, involves further judgement. These characteristics are often assigned according to historical patterns. History doesn’t always repeat itself, and certainly not at just the time we predict it will do so. As frequently disclaimed in investment materials, “Past performance is not a guarantee of future results.”
Limited data can be a problem. When simulating possible Super Bowl outcomes, the challenge is that a professional football team plays only sixteen regular season games against various opponents. One might be tempted to draw upon data from prior seasons, but one must consider that teams and individual players change over time. Financial data tends to be far more plentiful. Exceptions arise in the case of new securities or unique investment strategies.
Additionally, judgement arises in how to weight data points. In the case of the Super Bowl, should more recent games be weighted more heavily than early-season games? Should an early season game against a more comparable opponent be given additional weight?
There is also the possibility of a surprise factor. Among those who predicted the outcome of Notre Dame v. Alabama in the recent college football championship game, who considered the possibility that Manti Te’o’s dead girlfriend was a complete hoax and that his sudden realization of this could significantly affect his performance? Of course, no one. The possibility of a surprise factor that changes everything, even though unlikely, should be considered. In finance, disasters are certainly possible: major earthquakes or hurricanes, recessions, and wars could substantially affect an investment portfolio.
All these factors are behind assumptions that go into a Monte Carlo style simulation model. When your financial advisor wants to impress you with a “computer model” that he ran 50,000 times, you should smile. Then dig into the assumptions.
