Skip to content
AP U.S. Government and Politics · American Political Ideologies and Beliefs

Polling methodology

Polling methodology appears in 3 reviewed AP U.S. Government and Politics questions and sits in American Political Ideologies and Beliefs. Below are worked examples showing exactly how the exam tests it, with the answer marked and every distractor explained.

Polling methodology is the set of choices that decide whether a poll is accurate. These include how the sample is drawn, how questions are worded, whether likely voters are screened, and how the margin of error and confidence level are reported. A 95 percent confidence level means that if the same poll were repeated many times, about 95 percent of the resulting ranges would contain the true value.

Why it matters

A poll can be large and still be wrong if its method is flawed, so you need to know which errors a bigger sample fixes and which it does not. That judgment is what separates a trustworthy poll from a misleading one.

How the exam tests it

Passages describe a poll that missed the result and ask you to pick the cause. Check the size of the miss against the margin of error, then look for clues about who agreed to answer, which points to nonresponse bias.

Don’t confuse it with

Students confuse random sampling error, which shrinks as the sample grows and is captured by the margin of error, with systematic bias such as nonresponse bias, which a larger sample cannot fix.

Polling methodology in real questions

Example 1Medium
Read the following practice passage. A polling firm set out to forecast a statewide Senate race. Over two weeks, its interviewers called 30,000 randomly selected phone numbers and completed 2,400 interviews, giving the poll a reported margin of error of plus or minus 2 percentage points. Every respondent heard the same neutrally worded question, and every adult who finished the interview was counted, whether or not the person said he or she planned to vote. After the election, researchers found that the people who agreed to be interviewed were far more likely than those who hung up to follow politics closely and to hold strongly partisan views. The poll missed the final margin by 7 points. Based on the passage, which of the following best explains why the poll was inaccurate?
  1. AThe number of completed interviews was too small to keep random sampling error below the gap between the candidates
  2. BThe survey question was worded in a way that nudged respondents toward one of the candidates
  3. CPeople who took part in the poll differed systematically from those who declinedCorrect
  4. DThe firm's likely voter screen dropped many people who went on to vote in the election

Why C is right

The passage reports that the people who agreed to be interviewed were much more politically engaged and partisan than the people who hung up. That is nonresponse bias: the sample did not represent the electorate, even though it was large and randomly dialed. The passage also rules out the other explanations. The margin of error (plus or minus 2) is much smaller than the 7-point miss, the question wording was neutral, and no likely voter screen was used. A large sample reduces random error but cannot correct this kind of systematic bias.
Why the other 3 choices miss
A
Small samples do produce sampling error, but the passage reports 2,400 completed interviews and a margin of error of plus or minus 2 points, far smaller than the 7-point miss, so random sampling error cannot explain it.
B
Biased question wording is a real source of polling error, but the passage states that every respondent heard the same neutrally worded question.
D
Likely voter screens can exclude people who end up voting, but the passage says every adult who finished the interview was counted regardless of whether they planned to vote, so no screen was applied.
Example 2Harder
When pollsters report that a survey has a '95% confidence level,' they mean that
  1. A95% of the eligible population was contacted and offered the opportunity to participate in the survey
  2. B95% of respondents gave accurate and honest answers, and 5% are estimated to have provided false responses
  3. Cthe pollster is 95% certain that no sampling bias occurred in the respondent selection process
  4. Dif the same survey were conducted many times using the same sampling method, 95% of the resulting confidence intervals would contain the true population valueCorrect

Why D is right

A 95% confidence level is a frequentist statistical concept: if the survey procedure were repeated many times on independent random samples from the same population, approximately 95% of the calculated confidence intervals would contain the true population parameter. It does not mean the pollster is 95% certain their specific interval is correct — that probability is either 0 or 1 (the true value either is or is not in the interval). This distinction is frequently misunderstood by journalists and the public, who often interpret it as a probability about the specific poll's interval.
Why the other 3 choices miss
A
Incorrect because no poll contacts 95% of the eligible population; polls contact small samples (typically 600-2,000 respondents) and the confidence level refers to the sampling procedure's long-run accuracy, not the fraction of the population surveyed.
B
Incorrect because the 95% confidence level is a statement about the sampling procedure's long-run properties, not about respondent honesty rates; dishonesty or social desirability bias is a separate measurement concern addressed differently.
C
Incorrect because a 95% confidence level says nothing about sampling bias; confidence intervals assume random sampling and are invalid (regardless of stated confidence level) if systematic bias exists in sample selection.

Studied alongside

Other concepts from American Political Ideologies and Beliefs.