Political scientist Brian Schaffner writes:
If you happen to’ve paid any consideration in any respect to the information lately, you’ve got most likely seen various tales about how the financial system is weighing down President Biden’s reelection hopes. Many of those tales are primarily based on knowledge from polls that ask individuals about their very own financial conditions and in addition ask them what they consider Biden or how they plan to vote in 2024.
Such an evaluation could look one thing like the next graph, which plots responses to a query we requested on the Cooperative Election Survey about how every individual’s family earnings has modified over the previous yr and their approval ranking of President Biden. . . .
However we additionally see a possible pink flag once we have a look at this knowledge: extra Individuals reported that their incomes decreased quite than elevated over the previous yr even supposing authorities knowledge signifies that wages and salaries are on the rise. . . .
Republicans have been more likely to report that their incomes declined in the course of the earlier yr in comparison with Democrats. 35% of Republicans reported a decline in family earnings in comparison with 19% of Democrats.
Schaffner continues:
Is it actually true that Republicans are struggling considerably greater than Democrats in terms of their family incomes? Or is that this one other instance of a sample that survey researchers name “expressive responding” — a phenomenon the place people strategically present dishonest solutions to survey questions in an try to make their occasion look good or the opposite occasion look dangerous?
A Republican answering our survey may think about saying they’re doing worse economically than is definitely true as a approach of supporting the narrative that the financial system is struggling below the Biden presidency. Likewise, Democrats could report that they’re doing higher economically than they’re to undermine that very same narrative.
What can the info say? Right here’s Schaffner:
It’s typically arduous to detect survey respondents who’re partaking in such expressive responding as a result of we don’t truly know when somebody is giving dishonest survey responses. (Although see here for work I did with Sam Luks the place it was fairly clear). However within the case of earnings change on the 2022 CES, we truly do have knowledge that permits us to get a very good sense of this as a result of it simply so occurs that 11,000 of our respondents have been people we had beforehand interviewed again in 2020. Every time we interview a respondent, we ask them to report what their family earnings truly is. So for every respondent we all know what they stated their family earnings was in 2020 and what they stated it was in 2022. Since these questions present exact response classes and are buried amongst different demographic questions, it’s unlikely that respondents would assume to be dishonest when answering them in the identical approach that they may for the extra obscure and politically related query about earnings change.
So what can we see once we discover what individuals reported their earnings was in 2022 in comparison with what they reported it was in 2020?
Within the 2022 survey, “17% of Individuals reported that their earnings decreased considerably previously yr whereas one other 10% stated that their earnings decreased rather a lot. Solely 19% of Individuals reported a rise of their earnings.”
However when evaluating to the previous survey: “solely 18% of Individuals gave a decrease family earnings in 2022 than they did in 2020, whereas 35% reported the next earnings stage. Moreover, the partisan variations on this metric are a lot smaller — 21% of Republicans reported decrease incomes in 2022 in comparison with 2020 whereas 18% of Democrats reported the identical.”
Shaffner continues, “these two questions should not fully comparable because the self-reported change query asks concerning the previous yr however this second method compares incomes reported over a two-year interval. However, it’s uncertain that too many individuals may have suffered a lower in earnings from 2021 to 2022 if their earnings had elevated from 2020 to 2022.”
After evaluating responses of Democrats and Republicans, Shaffner states:
The unlucky impact of this sample is to exacerbate the sort of relationship we noticed within the first graph, making it appear like earnings change is having a significant impact on Biden’s approval ranking when the truth is it’s simply as doubtless that how any person feels about Biden is affecting how they reply the query about earnings change. I [Shaffner] can present this most clearly by recreating the primary plot, however this time utilizing the measure of how every respondent’s self-reported earnings truly modified between 2020 and 2022 quite than how they stated it had modified in the course of the earlier yr. Utilizing this method, it seems that the connection between earnings change and Biden approval nearly fully disappears.
He concludes:
This isn’t to say that the financial image is totally irrelevant to Biden’s comparatively low approval ranking or his reelection probabilities subsequent yr. It’s cheap to suspect that some swing voters are being persuaded by excessive inflation. However what this evaluation exhibits is that merely asking individuals how inflation is affecting them after which evaluating that to how they may vote in 2024 just isn’t a great way to ascertain an correct image of that relationship. Finally, we can’t at all times take at face worth what individuals inform us in polls to help a story that’s circulating within the information.
Good level. I’ve nothing so as to add.
P.S. Schaffner and his colleagues additionally have a post on the declare that liberals are happier than conservatives, or the opposite approach round. Concerning such statistics, let me level you to this post from a few years ago with further context here. It’s not simply survey respondents who say issues that aren’t true however match their political ideologies.