Public confidence in political polling tends to move in cycles: broad trust, a high-profile miss, sweeping declarations that polling is broken, then a quiet return to citing it. A more stable position is available, but it requires being specific about what a poll is actually measuring.
The sampling problem, restated
A poll estimates a population value from a sample. If the sample were drawn at random and everyone contacted responded, the only error would be sampling error — the familiar margin of error, which for a sample of a thousand is roughly three percentage points at conventional confidence levels.
Related: what first past the post and proportional representation each reward
Response rates to telephone polls have fallen dramatically over recent decades, in many cases into the low single digits. This does not automatically bias results, but it means the sample is no longer random in any meaningful sense. It is a sample of people willing to answer, and the question becomes whether willingness to answer correlates with political preference. When it does, the error is systematic and no amount of increasing sample size fixes it.
Weighting, and its limits
Pollsters correct for known imbalances by weighting: if the raw sample under-represents a demographic group relative to census figures, responses from that group count for more. Modern weighting schemes adjust on many variables simultaneously — age, sex, region, education, past vote, and sometimes attitudinal measures.
Weighting works well for characteristics that are measured and known. It cannot correct for an imbalance on an unmeasured characteristic. If people with a particular political disposition are systematically less likely to respond, and that disposition does not map cleanly onto any demographic the pollster weights by, the correction will not catch it. Several of the most notable polling misses trace to exactly this situation.
Related: how inflation is actually measured
The turnout model is doing most of the work
In an election poll, the raw responses are only half the exercise. The other half is deciding who will actually vote. Pollsters use various approaches — self-reported likelihood, past voting behaviour, modelled propensity — and these choices can move a headline figure by several points from identical underlying data.
This is why two polls fielded in the same week can differ substantially without either being wrong in any simple sense. They are answering slightly different questions about slightly different populations. It is also why the spread between pollsters is itself informative: a wide spread indicates genuine uncertainty about the composition of the electorate, not merely noise.
What polls do well
Polls are reliable at measuring change over time using consistent methodology. If the same pollster asks the same question the same way and the number moves eight points, something real has almost certainly happened, even if the absolute level is off.
They are good at identifying the relative salience of issues, at showing how opinion differs between groups, and at detecting large and stable majorities. They are considerably better at these tasks than at predicting a close election result, which is the task they are most often judged on.
What polls do badly
Close races are genuinely hard. When the true margin is inside the total error of the exercise — sampling error plus turnout modelling uncertainty plus non-response bias — a poll cannot resolve the outcome, and reporting that presents a two-point lead as meaningful is overstating the instrument.
Polls also struggle with questions where respondents have no pre-existing opinion. Asked about an obscure policy, many people will supply an answer rather than admit unfamiliarity, and the resulting distribution can be almost entirely an artefact of question wording. Small changes in phrasing on such questions produce large changes in results, which is a reliable sign that the underlying opinion is not firmly held.
Reading a poll properly
Several things are worth checking before taking a number seriously. Who commissioned it, since sponsors sometimes have an interest in the result and question wording is where that interest shows. What the exact question was, in full. When it was in the field, since events move numbers. How many people were sampled, and among whom. And whether the reported figures include or exclude undecided respondents, which alone can shift a headline by several points.
The single most useful habit is to look at averages across multiple pollsters rather than any individual result. Averaging reduces the influence of any one house’s methodological choices, and the trend in an average is considerably more informative than the level of any single poll.
The honest summary
Polling is a measurement technology with known and quantifiable weaknesses, operating in conditions considerably harder than when its methods were developed. It remains the best available tool for understanding public opinion at scale, and it is systematically misused when treated as a forecast of a close outcome rather than a measurement with error bars.
More from PressBBC
- First Past the Post vs Proportional Representation: What Each System Rewards
- How a Bill Becomes Law in the UK Parliament
- Understanding Inflation: CPI, Core, and Why Your Experience Differs
- Browse everything in our Politics section.
Sources and further reading
- Pew Research Center — Methods — how modern surveys are designed and weighted
- British Polling Council — the disclosure rules reputable UK pollsters sign up to
