A significance level of α = 0.05 in hypothesis testing means:
- AThe probability the null hypothesis is true is 5%.
- BThere is a 5% chance of rejecting a null hypothesis that is true.Correct
- COnly 5% of experiments conducted at this level will produce correct results.
- DThe study has a 95% probability of detecting a real effect if one exists.
Why B is right
The significance level α is the probability of committing a Type I error — rejecting the null hypothesis when it is actually true. Setting α = 0.05 means the researcher accepts a 5% chance of a false positive (incorrectly concluding there is an effect when there is none).
Why the other 3 choices miss
- A
- The significance level is not a statement about the probability that H0 is true. That would require prior probabilities and Bayesian reasoning. Alpha is the long-run false positive rate under the frequentist framework.
- C
- The significance level has nothing to do with the proportion of experiments producing correct results. It specifically controls the false positive rate.
- D
- The probability of detecting a real effect is statistical power (1 - β), not α. Power depends on α, the effect size, and the sample size, but α alone does not determine power.