Wednesday, March 16, 2022

Too High, Too Low, Mostly Wrong

Americans don't know squat about their fellow countrypeople, as shown in this graph from YouGov:

(Click to enlarge.)

There's a lot of eyeroll-worthy doofiness in there:

  • 12% of people are transgender
  • 24% are gay or lesbian, and 28% bi
  • 25% are vegetarian or vegan 

But as the authors explain, it's probably best explained by the way people deal with estimating small and large subgroups, since the mis-estimation holds up for neutral minorities as well as politically charged ones. If you look at both ends of the graph, the pattern is clear: the answers are most accurate when they are near 50% and more inaccurate the further the correct answer gets from 50%. 

It's still disturbingly duh, though. 

It makes me wonder how the survey was given: did respondents have a chance to realize they had said 67% of Americans live in either Texas, California, or New York City? Or that 73% are either Jewish, Muslim, or atheist — but that 57% are also Christian? Or that 119% (yes, 119%, which is weird even accounting for the overlap of Hispanic and other categories) are BIPOC, while 59% are also somehow white?

Someone responding in the Twitter thread about this linked to a Washington Post story about research that showed people became more accurate when they were incentivized to be more accurate in their answers. I wonder if it's honesty was the reason, or were they just more careful in their answers? If you thought you might financially benefit from accuracy on some random survey, you'd likely be more careful and think a little harder. Honest or careful, it doesn't matter which: either way, the survey would likely get better answers.


2 comments:

Michael Leddy said...

Really interesting.

I'd like to see YouGov scramble the categories so the reader can guess the percentages. I covered the numbers with another window to do so but realized right away that guessing was pointless, as I knew I would be guessing from low to high.

I’m surprised how relatively few Catholics there are. You can guess how I was raised (at least sort of).

Daughter Number Three said...

I wish I had given myself the questions before I read about it. On some I know the exact percentage (especially some of the religion and race numbers) from previous reading. But there are others where I just don't have a good idea so I wonder if I would have had the same tendency toward the middle.

I tried it out by giving the survey to my other half, and he did a lot better than the YouGov results overall, with a few interesting exceptions. He thought a lot fewer people had guns than do (which was the opposite of the survey results) and also many fewer Republicans and Democrats than in reality.