Statistics… suffice to say- not my strong suit. But one thing I know about debating them is that they are usually BS. The way I often teach students to think about statistics is that they are generally calculated in social sciences using a 95% confidence interval- which in shorthand is a way of saying “we are 95% sure this is correct” (obviously not the technical definition).
If we conduct 1,000 studies and publish the results, and 95% of them are likely to be correct, that means we will have 50 studies that are probably incorrect. These studies make up the oddball news items like “drinking soda makes you live longer” and other weird things you see that have “statistical” support.(Credit to Sklansky on this one)
Those 50 also probably make up a disproportionately large percentage of statistics used in debate. The reason is obvious- they are crazy and debaters gravitate towards crazy arguments. As people are taught more and more to rely on peer reviewed quality sources it should be remembered that just because something has some math behind it doesn’t necessarily mean its “true”.
In that vein here is an interesting link
At long last, regressions were run and… no result. No relationship between price shocks and conflict, even in the most generous scenarios. I shrugged and thought, “Well, so much for that.” My committee said, “Huh, what about that child soldiering project we told you not to do?” And off I went on my career as micro-conflict man.
In the meantime, lots of papers that did see an impact of economic shocks on conflict or instability did get published. The conventional wisdom grew: Rising incomes made the state more attractive to rebels as a prize, and falling incomes made it easier to recruit rebels. No matter that these two ideas ran in apparently opposite directions.
Meanwhile, I met other academics that had run the same regression as me. Famous ones you have probably heard of. Their reaction was the same as mine: “Oh, I found that result,” several said, “but I’m worried there’s nothing there because my data have problems, and the specification wasn’t quite right. So I left it out of the paper. I’ve been meaning to get back to that.”
Let’s follow a simple decision rule: run your regressions with inevitably imperfect data and models. If you get the theoretically predicted result (any of them), publish. If not, wait and look into your data and empirical strategy more.
The result? As in the natural sciences, most published research findings are probably false.