Race and Ethnicity in Survey Data: Challenges in Asking and Reporting

• Colleen Porter, M.A.

In an increasingly diverse society, how should surveys collect and report data about people’s race and ethnicity? In the last year, serious questions have been asked about such practices, as the nation’s population becomes ever more varied and intermingled in ethnic heritage. Some of these concerns have been explored in a book and prominent New York Times opinion article by Kenneth Prewitt, former director of the U.S. Census Bureau and professor of public affairs at Columbia University.

Prewitt argues that official definitions have failed to capture the reality of diverse human skin color and identity. “Statistical race” is important to public policy, often cited as legislators and business leaders discuss rates of home ownership, incarceration, high school graduation, and health insurance coverage. Prewitt observes that in a society where “how many of which races are doing what” drives decision-making, then defining those racial categories is crucial.

During Prewitt’s tenure at the U.S. Census Bureau, for the first time Americans could report more than one race on the 2000 Census form. This was a radical change, because only a single race had been allowed ever since the first census in 1790. Those racial designations were not just statistics in a ledger; they had been used for political purposes, notably in the early days of the country when slavery was legal and in the Jim Crow South.

In 1977, in an attempt to standardize race data collection on forms across all federal government agencies, the Office of Management and Budget (OMB) issued Directive 15, setting standards for federal data on race and ethnicity.

The OMB guidelines have become the driving force behind survey data collection regarding race nowadays. They were revised in 1997 to facilitate collection of multiple races, and updated since then, such as the 2003 revision that divided the “Asian or Pacific Islander” category into two separate categories: "Asian" and "Native Hawaiian or Other Pacific Islander.” Further refinements have been discussed and may be implemented in the future, such as breaking out a category for those of Mideast or Arabic heritage who are currently classified as “White.”

Prewitt argues for further changes that move away from mere skin color, including changing the basic question to one of “race or origin?” that allows Hispanicity as one of the choices from which one or more can be selected.

Not everyone supports the idea of making Hispanic a race. There are a significant number of Americans who identify as both Black and Hispanic. In Florida, these include immigrants from the Dominic Republic. Also, there are a growing number of Brazilians in South Florida; Brazilians are considered Latino, being from South America, but not Hispanic, since they are not from a Spanish-speaking culture.

At the University of Florida Survey Research Center (UFSRC), we ask these item according to the OMB guidelines, querying about Hispanicity first and always allowing survey participants to select one or more races.

However, investigators who report findings from our studies do not always have a statistically valid sample of each racial group. For example, even though the Florida Health Insurance Study of 2004 had one of the largest sample sizes of any statewide study with data from 17,435 Florida households, the number of Asians in the sample was still insufficient to generate direct estimates of health insurance among Asians Floridians; they were lumped into the “Other” category.

An insight into how race is reported in various settings came from a 2003 patient satisfaction survey we conducted of enrollees in a South Florida Medicaid pilot project using the standardized Consumer Assessment of Health Plans questionnaire. Only a small number (n=5) chose multiple races on the survey and were dropped from this analysis.

Overall, a third (34%) of survey respondents gave a different race/ethnicity on the survey than was listed in the Medicaid administrative database (Fig. 1). The largest disconnect was in the “Other” category, which was the race that had been listed in the program files for about a quarter (24%) of the survey respondents. The percentage of “Other” dropped to less than 1% in the survey findings.

How did those original 24% “Other” describe themselves when asked in the survey? About 83% said Hispanic, 11% Non-Hispanic Black, 4% Non-Hispanic White, and only 2% remained in the “Other” category, including Asians and Native Americans (Fig 2).

Part of the disconnect may have been people who were enrolled in the Medicaid program by a social worker or other helper who observed the enrollee’s appearance and checked “Other” on the form, rather than asking the enrollee for a self-designation. When they were actually asked, they gave a different answer. We presented the findings at a national conference, and heard from other researchers who observed similar patterns with third-party reports of race, including military recruiters and educational counselors.

Discrepancies between health agency administrative data and self-report have also been documented in other states, including Minnesota, California, Massachusetts. More recent improvements in Florida Medicaid’s enrollment processes may eliminate such confusion, but the observed discrepancies highlight the challenges in collecting race data.

In the wake of these ongoing changes in data collection, at the UFSRC we will continue to look to the Census Bureau and OMB for guidance on the best practices regarding collection of survey data on race and ethnicity.

ACKNOWLEDGEMENT:  Thanks to Stephanie Fuenmayor for assistance with preparation of this article. 

POSTED:  July 11, 2014