Skip to content
AP Statistics · Inference for Quantitative Data: Means

Statistical vs practical significance

Statistical vs practical significance appears in 4 reviewed AP Statistics questions and sits in Inference for Quantitative Data: Means. Below are worked examples showing exactly how the exam tests it, with the answer marked and every distractor explained.

How Statistical vs practical significance is tested

Example 1Harder
A researcher finds a statistically significant chi-square test for independence (p < 0.001) between gender and career preference in a sample of 10,000 adults. However, Cramer's V (a measure of effect size for categorical variables) is only 0.04. This result illustrates that
  1. Athe sample size of 10,000 is too large for a valid chi-square test of independence
  2. Bthe chi-square test must be incorrect because the reported effect size is so small
  3. Cwith large samples, trivial associations can still be statistically significantCorrect
  4. Dgender and career preference are strongly related because the p-value is below 0.001

Why C is right

With very large samples, chi-square tests have enormous power and can detect even tiny departures from independence. A Cramer's V of 0.04 indicates an extremely weak association that, while statistically significant (unlikely due to chance), has virtually no practical importance. This demonstrates the critical distinction between statistical significance and practical significance.
Why the other 3 choices miss
A
Incorrect. There is no upper limit on sample size for chi-square tests. The test is valid; the issue is interpreting a significant but practically meaningless result.
B
Incorrect. The test is not incorrect. The tiny effect truly exists in the sample; it is just too small to be meaningful. Statistical significance correctly identifies a non-zero association.
D
Incorrect. A very small p-value indicates that the association is unlikely to be due to chance, but it says nothing about the strength of the association. Cramer's V = 0.04 shows the association is very weak.

Studied alongside

Other concepts from Inference for Quantitative Data: Means.