Please consume responsibly: Earnings data and the gender wage gap
April 14 of this year marked what is known as Equal Pay Day, the representing “how far into the year women must work to earn what men earned in the previous year.” The choice of date is based on research demonstrating that women earn approximately 77 cents for every dollar earned by their male counterparts. Whether it is 77, 84, or 93 cents on the dollar, there is reason to believe that women earn less than men in this country—but it takes a lot of careful work to meaningfully demonstrate it.
Recently, I have been asked for data on men’s versus women’s wages in Virginia, overall, as well as within smaller regions, such as particular localities or zip codes. Each time, the asker—a journalist—has explained that his goal is to find quantitative data addressing the topic of gender wage equity in the workplace. I’m thrilled that journalists are not only interested in asking these questions, but also looking for high-quality data to use as they write about this complex topic. But while comparing wages might appear to be the most obvious way to get at the answer, careful consideration of the data reveals why this task is not as straightforward as it may appear.
First, it is not enough to look at wage statistics alone. This is particularly true for small geographies; this is what today’s post will demonstrate. In addition, it is important to include information other than strict wages when investigating wage parity. I’ll take a look at what additional data is necessary to examine in my next post on this topic. Finally (since it might, after two posts, appear that this topic is too complex to even approach without making a doctoral dissertation of it) I’ll follow up with some examples of (1) Existing, high-quality research on the topic of gender-based wage disparity in the workplace, and (2) Other questions that are equally as important to ask, and answer, when discussing whether men and women have different experiences in the labor force.
Part 1: The trouble with wage data and small geographies, or, Why accuracy and precision are not the same thing.
The best source for wage data by sex at any level of US geography comes from the Census Bureau. In fact, for small geographies, we have more than one option for examining labor force data using Census products. Each has strengths and limitations, and it’s critical to acknowledge these.
The Quarterly Workforce Indicator Explorer is an excellent resource for learning about employment and wages, and allows the user to compare men’s and women’s wages at the state and locality levels, which is great. This resource is especially valuable because it reflects Unemployment Insurance earnings data provided by employers. This is reliable information, not only because it is collected for each employee for official records, but also because it is beneficial to the employers—the ones who collect it—to be careful in their reporting. This source has a high degree of accuracy; that is, we can be fairly sure that it very closely reflects reality, because it does a good job giving data on all the individuals of interest. Take a look at what it reveals about wages for male and female workers in selected Virginia localities in the first quarter of 2014:
This tool, however, comes with a notable limitation: It does not distinguish between part-time and full-time workers. If we compare the average quarterly wages of workers, but can’t guarantee that the two groups worked a comparable number hours, on average, then our analysis will lose its meaning. More women than men—both proportionally, and by raw number*—are employed in part-time positions, and these part-time positions are paid less than full-time positions, on average. I want to compare workers that are as similar as possible on all measures except for sex, so I want data that keeps hours as constant as possible.
The U.S. Census is another great option, and allows the user to compare full-time, year-round male and female workers’ wages. Also, this survey reports median earnings, allowing us to escape the possibility of what’s sometimes called the “Bill Gates effect,” or an inflation of the average due to just a handful of extremely high values. Let’s take a look at median annual earnings between 2009 and 2013 among year-round, full-time workers in the same localities we examined earlier:
The results look fairly clear, right? Not so fast.
When using this data set, we have to pay attention to the fact that we are only able to learn about wages though a survey, which brings some fuzziness to the table. First, the wages are self-reported, which means that they’re not as reliable as if we actually had individuals’ tax returns—or Unemployment Insurance records—in hand. But, perhaps more importantly, we are only able to get this information from a subgroup of the population, which makes this an estimate, not a direct calculation. And when we look at small subgroups of Virginians—like those living in particular localities or even zip codes—this makes for some uncertainty.
Why? When we look at sample data, the most reliable way to compare values of interest between groups—such as median wages between men and women—is actually to compare ranges, or margins of error, around these values. Offering this range, as opposed to a single value, serves as an acknowledgement that we need to be generous in our calculation, since don’t have data from every single person in the population of interest. In order to determine how wide the range must be, we can use some calculations based on number of people surveyed and the values of responses we did get. Minimum and maximum values give, in this case, boundaries on where we believe the true median wage would lie for each group, if we could actually poll the full population of each locality. The narrower the range of possible values, the more precise we say our estimate is.
As a direct result of the math involved, smaller sample size yields a wider ranges of possible values for statistics of interest. Because of the way the American Community Survey is conducted, small localities are likely to have fewer people included than large localities. While the number of people surveyed across the entire commonwealth allows us to make pretty precise claims about Virginia, our estimates for small areas—such as zip codes or even certain localities—are less precise .
So, how good are these estimates? In this case, we can be 90 percent certain the real value lies in these ranges. In other words, if I tell you, “Between 2009 and 2013, women in Caroline county earned $38,218 annually, with a margin of error of $3800,” then you can interpret that statement as “If I collected 100 similarly-sized samples of earnings data from women in Caroline county, the median income would be at least $34,418 and at most $42,018 in at least 90 of these samples.” That’s, you know, reasonably good. An A-, on a lot of scales.
So now, let’s take another look at the graph from before—this time, including bars to signify margins of error around the estimated values.
Once we add in that range—the margin of error—we see that the likely wages for men and women at the locality level frequently overlap. In fact, in Caroline, Lancaster, Northampton, and Bristol, we can’t even say that men and women’s earnings are meaningfully different. This complicates even the easy question: Do men earn more than women in these areas? For these areas, and other small localities, we don’t have enough data to unequivocally say, “Yes.”
But even for the localities in which the margins of error don’t overlap—that is, those in which it seems pretty evident that men out-earn women—we have not yet demonstrated why this is true. In other words, even in areas where we can make the claim that men earn more than women, it is too early in the analysis to show that any wage disparity we find is due explicitly and exclusively to gender. In order to investigate that, I want to compare workers that are as similar as possible on all measures except for sex. Holding constant the full-time, year-round status of these workers is a step in the right direction, but only gets us part of the way to a meaningful comparison.
What, other than wages, do we want to know in order to examine gender parity in compensation? If you said you want to know more about these workers’ education status, experience, age, or industry, then stay tuned—I’ll be taking up this particular topic in my next post.
(*Yes, given the context of the post, I understand that it is a little ironic that I use survey data to demonstrate this point, but the numbers are so dramatically different, and the anecdotal evidence so strong, that I feel confident citing it.)