The idea here is that it’s not about the number of heads, it’s that we’ve found even one. A statistical test is a bit like flipping a coin; if it’s heads, it’s significant, if it’s tails, it isn’t. If all we care about is getting a ‘significant’ result, we’ll just keep on flipping until we get a heads.

What the Bonferroni test does is essentially what you are saying; it makes the level required for significance higher proportional to the number of tests we do so that we can’t just keep flipping coins to get the result that we want.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store