May 01, 2012
Post Hoc Reasoning
Here's why I'm becoming a Bayesian.
Let's say I collect the winning lottery numbers from the last 10 years. I sped a couple months trying to find some pattern---any pattern---that generates numbers that would have won a few times. It's even possible I might come up with a model that is statistically significant and accounts for some decent percentage of the variance in lottery numbers.
Have I found a way to predict future lottery draws? Will it make me rich? Why not? But it's statistically significant and predicts an acceptable percentage of variance. Why isn't that enough?
In contrast, if I approach the problem from a Bayesian standpoint, the first thing I have to do is defend the notion that such a model can be made: that is my prior probability and I can't run an analysis without it. In other words, the plausibility of the research is part of the research.
There's a downside to this, of course. If people don't believe a hypothesis is likely to be true, the insightful researcher who suspects it is will have a harder time using a Bayesian approach than a traditional approach.
But that's because Bayesian's don't let you get away with post hoc reasoning. And that's a pretty neat trick.
Posted by rbrent at May 1, 2012 06:05 AM