Great piece -- an elegant, concise summary of the hypotheses that polling experts and campaigns are mulling over. I find the "low social trust leads to low response rates" hypothesis most compelling (though we shall see). Do we have any evidence that polling is off in similar ways in other countries? The most recent Australian national election comes to mind, when the Conservatives surprised poll watchers and won a majority.
Thanks for the insightful write up. Interested in your thoughts about Zeynep Tufekci‘s ideas on the inherent weaknesses of models, especially as they can become participants in the events they are predicting, thus influencing the events.
Why are we doing polls when every business can access aggregate and more nuanced targeted search history data to find look alike customers. The Republicans have formed an unusual alliance by finding look alike customers in certain policies and conspiracy theories and then feeding the aspect that pertains to their policies/lies to grow polarization. Polls are more passive and are slower to react to the fast changing trend and non centralized media market... that happens when people randomly search for something for the first time or suddenly get a new ad or suggested video on YouTube that sends them down a rabbit hole.
I should have made this point more clear in the piece, but the Republicans were just as shocked at the results as the Dems in '16 and '20. So they don't have any secret sauce either. Lots of work to do to figure how to accurately measure public opinion
Great insight, Dan. I’m very much interested in the theory that low social trust correlates to low poll responses, and no doubt the conspiracy mindset of qanon has increased that. I can’t help but wonder how many Trump voters misled pollsters to ‘own the libs’, but either way - if the margin of error for republican voters is reliably in one direction only, then perhaps it’s time to forever assume republican approval ratings are 5 (or so) points higher than what any poll claims. More of a permanent ‘red enigma’, less of an Election Day ‘mirage’.
There is a distinction between Trump voters being unwilling to answer the phone and answering the phone and lying about their vote. Lots of evidence of the former, less evidence for the latter.
Millions spent on polling and we could take the temperature on the local social media, Nextdoor. The Trumpers used all the memes and tropes they were spoon fed, socialism, communism, take away their rights. Large number of Trumper Hispanics posted.
1- is there an over-reliance on polls due to a relatively low level of ongoing voter engagement. I seem to remember that the Obama models were based on combining voter contact information with polling information. Everything keeps pointing to the need for a stronger ongoing organisation where voters are engaged outside of elections.
2- polls are great at modelling based on past observations but are generally poor (at least in terms of getting things within a reasonable margin of error) when there are new dynamics. Romney thought turnout would revert to the historic pattern re youth turnout. Polls this year assumed, at best, 2016 levels of enthusiasm for Trump in his base.
And one question:
-Why did Ann Seltzer nail Iowa? On the basis that she applied standard methodologies in getting respondents, she clearly applied local knowledge to shape the questions she asked and the weighting assumptions.
1. This year in particular there was less and different voter contact info because of the shift to online organizing/text v. door to door.
2. correct. This helps explain 2016, but raises questions about why the adjustments made in 2020 didnt work
3. Great question. She is certainly one of the best pollsters and has more experience polling Iowa than any pollster has in polling any state, but her methodology was similar to others. It's possible it was a broken clock being right situation, because some of the underlying numbers were off. Hard to say
Like Ed, I find the "low trust in institutions" hypothesis pretty compelling. My big question/worry is, if that's what's going on, how reversible is that? Eroding faith in government/institutions generally has been the GOP's strategy for 40 years and counting, although it's reached another level in the last decade. But I wonder how many people who've crossed over into another level of mistrust could be won back with a president/(hopefully) Congress that actually seeks to rebuild trust in institutions and delivers some meaningful results, vs. how many have fallen off the deep end and will be hard to get back anytime soon. E.g., let's say Biden nails the COVID response from here on out. Will that fundamentally change levels of trust in government, or is it way more of a long-term problem?
While I respect Nate Cohn's opinion, this piece https://www.washingtonpost.com/opinions/2020/11/19/why-were-polls-wrong-ignore-calls/ by David Hill seems to be far more insightful based on my own experience of not answering calls from unknown numbers and from trying to call voters for my adopted state of AZ where I got one answer in a hundred. I am not sure how pollsters and politicians in general are going to tackle this in order to find out what voters want but some "outside the box" thinking needs to happen.
My concern is that polling can no longer be relied upon to provide reliable information. If so, perhaps it is time to devise an alternative method of analysis.
A really solid piece. This piece summarizes nearly everything on my mind about polling in 2020. The entire political industry on both sides runs off of perceptions, grounded in data, of what the American public thinks about issues and how they are likely to vote. What moves them to an issue or a particular candidate. And also what will move them away from a candidate or issue outcome. Well done. I am hoping for solutions to be offered and presented so we can get a better sense of the electorate in 2022.
Really helpful writeup. I would like to learn more about why the Biden campaign polling was better - how did they do their polling? Also, anecdotally, it does seem as if other candidates had more accurate internal polling. For example, Gina Ortiz Jones's race (TX-23) was consistently rated as promising - even likely - for Democrats, but her internal polling evidently showed her down 5. Would love to hear more about what they and the Biden team were doing differently. Also would love to hear more about herding among pollsters, which I believe Zeynep Tufekci mentioned in her piece.
Great piece Dan! One area I'd like to hear more about is how polling isn't just a measure of attitudes, it actively influences people's behaviors and decision making. Media poll reporting is a player in a race (something journalists are loathe to admit, I think).
I think the Presidential Election outcomes seem more in line with the polls than some of the Senate races. How did the polls get Maine or North Carolina or even Iowa/Montana so wrong? Polls said we'd easily win Maine and likely win North Carolina and were 50-50 in Iowa/Montana.
Great insight – terrifying to think about the size of the silent Trump vote/QAnon clan (one and the same) and how polling is yet another victim of the erosion of trust in institutions. He poisons minds in his and the Republican Party's quest to eviscerate the very idea of objective reality.
One thing I have wondered, does it reveal anything that Biden seems likely to come with an +.5 to -1.3 of his vote share in most places with a lot of polling, including his national average and Trump's disapproval number? His final popular vote share might end up exactly at his 538 average. Seems also set to win every state he polled above 50 as well. Even the generic down ballot polling doesn't look it's going to overestimate democrats, just underestimate GOP vote share.
Great piece -- an elegant, concise summary of the hypotheses that polling experts and campaigns are mulling over. I find the "low social trust leads to low response rates" hypothesis most compelling (though we shall see). Do we have any evidence that polling is off in similar ways in other countries? The most recent Australian national election comes to mind, when the Conservatives surprised poll watchers and won a majority.
I need to dig into this more, but it certainly happened with Brexit in the UK
Thanks for the insightful write up. Interested in your thoughts about Zeynep Tufekci‘s ideas on the inherent weaknesses of models, especially as they can become participants in the events they are predicting, thus influencing the events.
Why are we doing polls when every business can access aggregate and more nuanced targeted search history data to find look alike customers. The Republicans have formed an unusual alliance by finding look alike customers in certain policies and conspiracy theories and then feeding the aspect that pertains to their policies/lies to grow polarization. Polls are more passive and are slower to react to the fast changing trend and non centralized media market... that happens when people randomly search for something for the first time or suddenly get a new ad or suggested video on YouTube that sends them down a rabbit hole.
I should have made this point more clear in the piece, but the Republicans were just as shocked at the results as the Dems in '16 and '20. So they don't have any secret sauce either. Lots of work to do to figure how to accurately measure public opinion
Great insight, Dan. I’m very much interested in the theory that low social trust correlates to low poll responses, and no doubt the conspiracy mindset of qanon has increased that. I can’t help but wonder how many Trump voters misled pollsters to ‘own the libs’, but either way - if the margin of error for republican voters is reliably in one direction only, then perhaps it’s time to forever assume republican approval ratings are 5 (or so) points higher than what any poll claims. More of a permanent ‘red enigma’, less of an Election Day ‘mirage’.
There is a distinction between Trump voters being unwilling to answer the phone and answering the phone and lying about their vote. Lots of evidence of the former, less evidence for the latter.
Millions spent on polling and we could take the temperature on the local social media, Nextdoor. The Trumpers used all the memes and tropes they were spoon fed, socialism, communism, take away their rights. Large number of Trumper Hispanics posted.
V intelligent, reflective piece. 2 observations:
1- is there an over-reliance on polls due to a relatively low level of ongoing voter engagement. I seem to remember that the Obama models were based on combining voter contact information with polling information. Everything keeps pointing to the need for a stronger ongoing organisation where voters are engaged outside of elections.
2- polls are great at modelling based on past observations but are generally poor (at least in terms of getting things within a reasonable margin of error) when there are new dynamics. Romney thought turnout would revert to the historic pattern re youth turnout. Polls this year assumed, at best, 2016 levels of enthusiasm for Trump in his base.
And one question:
-Why did Ann Seltzer nail Iowa? On the basis that she applied standard methodologies in getting respondents, she clearly applied local knowledge to shape the questions she asked and the weighting assumptions.
1. This year in particular there was less and different voter contact info because of the shift to online organizing/text v. door to door.
2. correct. This helps explain 2016, but raises questions about why the adjustments made in 2020 didnt work
3. Great question. She is certainly one of the best pollsters and has more experience polling Iowa than any pollster has in polling any state, but her methodology was similar to others. It's possible it was a broken clock being right situation, because some of the underlying numbers were off. Hard to say
Like Ed, I find the "low trust in institutions" hypothesis pretty compelling. My big question/worry is, if that's what's going on, how reversible is that? Eroding faith in government/institutions generally has been the GOP's strategy for 40 years and counting, although it's reached another level in the last decade. But I wonder how many people who've crossed over into another level of mistrust could be won back with a president/(hopefully) Congress that actually seeks to rebuild trust in institutions and delivers some meaningful results, vs. how many have fallen off the deep end and will be hard to get back anytime soon. E.g., let's say Biden nails the COVID response from here on out. Will that fundamentally change levels of trust in government, or is it way more of a long-term problem?
While I respect Nate Cohn's opinion, this piece https://www.washingtonpost.com/opinions/2020/11/19/why-were-polls-wrong-ignore-calls/ by David Hill seems to be far more insightful based on my own experience of not answering calls from unknown numbers and from trying to call voters for my adopted state of AZ where I got one answer in a hundred. I am not sure how pollsters and politicians in general are going to tackle this in order to find out what voters want but some "outside the box" thinking needs to happen.
My concern is that polling can no longer be relied upon to provide reliable information. If so, perhaps it is time to devise an alternative method of analysis.
A really solid piece. This piece summarizes nearly everything on my mind about polling in 2020. The entire political industry on both sides runs off of perceptions, grounded in data, of what the American public thinks about issues and how they are likely to vote. What moves them to an issue or a particular candidate. And also what will move them away from a candidate or issue outcome. Well done. I am hoping for solutions to be offered and presented so we can get a better sense of the electorate in 2022.
Really helpful writeup. I would like to learn more about why the Biden campaign polling was better - how did they do their polling? Also, anecdotally, it does seem as if other candidates had more accurate internal polling. For example, Gina Ortiz Jones's race (TX-23) was consistently rated as promising - even likely - for Democrats, but her internal polling evidently showed her down 5. Would love to hear more about what they and the Biden team were doing differently. Also would love to hear more about herding among pollsters, which I believe Zeynep Tufekci mentioned in her piece.
Great piece Dan! One area I'd like to hear more about is how polling isn't just a measure of attitudes, it actively influences people's behaviors and decision making. Media poll reporting is a player in a race (something journalists are loathe to admit, I think).
I think the Presidential Election outcomes seem more in line with the polls than some of the Senate races. How did the polls get Maine or North Carolina or even Iowa/Montana so wrong? Polls said we'd easily win Maine and likely win North Carolina and were 50-50 in Iowa/Montana.
Great insight – terrifying to think about the size of the silent Trump vote/QAnon clan (one and the same) and how polling is yet another victim of the erosion of trust in institutions. He poisons minds in his and the Republican Party's quest to eviscerate the very idea of objective reality.
Real consequences as we chart our course forward, you say. Tell us more about that, please.
One thing I have wondered, does it reveal anything that Biden seems likely to come with an +.5 to -1.3 of his vote share in most places with a lot of polling, including his national average and Trump's disapproval number? His final popular vote share might end up exactly at his 538 average. Seems also set to win every state he polled above 50 as well. Even the generic down ballot polling doesn't look it's going to overestimate democrats, just underestimate GOP vote share.
What if we should all just move to the Nordic countries?
https://worldhappiness.report/ed/2020/the-nordic-exceptionalism-what-explains-why-the-nordic-countries-are-constantly-among-the-happiest-in-the-world/
Sorry, I've read way too much Twitter today.