The Washington Post Ran a Correction to Its Disability Story. Here’s Why It’s Still Wrong.

cherries growing on tree — Unlike the data used by the Washington Post, these cherries haven’t been picked yet. Image: Nick Grabowski via Flickr Creative Commons, https://flic.kr/p/2iiFZE

Last week, TalkPoverty pointed out several serious problems with The Washington Post’s recent analysis of Social Security disability benefits in rural America. Yesterday, The Post issued a correction alongside new calculations. Unfortunately, there are still major problems with their data—and their central thesis.

For starters, The Post continues to over-count “working-age” beneficiaries by including more than half a million people over 65—even adding in some people who are more than 80 years old. Moreover, instead of using the Census Bureau’s American Community Survey (ACS)—what the Census calls “the premier source for detailed information about the American people”—The Post uses a far less common data set from the Centers for Disease Control and Prevention (CDC). Compared to ACS data, these data undercount the number of working-age people in rural counties, which in turn jacks up The Post’s findings on the percentages of working-age people who are receiving disability benefits in these counties.

But let’s not lose the forest for the trees here. Even using The Post’s flawed methods, they were only able to find one county—out of more than 3,100 counties nationwide—where the story’s central claim that “as many as one-third of working-age adults are receiving monthly disability checks” holds up. Not a single other county even comes close. In fact, The Post’s own analysis—which it has now made available in a public data file next to the story, yields an average rate of about 9.1 percent of working-age adults receiving benefits across rural counties—just three percentage points higher than the national average.*

And yet the article is framed as follows: “Across large swaths of the country,” the article still reads, “disability has become a force that has reshaped scores of mostly white, almost exclusively rural communities, where as many as one-third of working-age adults are receiving monthly disability checks.”

If by “large swaths” and “scores of… rural communities” The Post means McDowell County, West Virginia, population less than 21,000 residents—and nowhere else in America—then sure.

But the fact is there’s a word for using data this way: cherry-picking.