Download data and study materials from OSF
Principal investigators:
Jennifer A. Heerwig
Stony Brook University
Email: jenheerwig@gmail.com
Homepage: http://www.jenheerwig.com/
Brian J. McCabe
Georgetown University
Email: mccabeb@georgetown.edu
Homepage: http://brianjamesmccabe.com/
Sample size: 1560
Field period: 6/7/2007-6/13/2007
Survey research consistently reports a positive association between educational attainment and socially tolerant attitudes, but critics hold that respondents with high levels of education may simply purport to hold attitudes seen as socially desirable. In this research, we seek to adjudicate between the claim that the association between education and social tolerance is simply an artifact of sophisticated social desirability reporting on the part of well-educated respondents and the competing theory that education has a real impact on increasing forms of social tolerance. Using support for a black Presidential candidate as our measure of social tolerance, we utilize an innovative online list experiment approach to test whether high levels of support are inflated because of social desirability reporting amongst the educational elite. We find no evidence of systematic over-reporting of support for a black Presidential candidate amongst respondents with high levels of education, and note that social desirability bias declines as educational attainment increases. This research bolsters arguments about the liberalizing effect of education on socially tolerant attitudes, and challenges evidence that attributes this relationship to high levels of social desirability bias.
The first hypothesis is that education is positively associated with socially tolerant attitudes. Taken at face value, this hypothesis purports that higher levels of education correspond with higher levels of professed support for socially tolerant behaviors.
The second hypothesis is that education is positively associated with socially desirable reporting. When asked about socially sensitive topics, we hypothesize that highly educated respondents are the most likely to misrepresent their true beliefs, attitudes or practices in order to align themselves with existing social norms.
We conduct an on-line list experiment with three groups. In the first group, respondents are asked to indicate whether or not they agree with the following statement: “I am willing to support a black Presidential candidate.” In the second group, respondents were presented with a series of three statements and asked how many statements they supported. Respondents were specifically instructed not to indicate which statements they supported, but simply to select the total number. The three statements were: I think Presidential campaigns are too costly; I am willing to support stronger immigration laws; and I think the war in Iraq will ultimately make the US safer. In the third group, respondents were presented with a series of four statements and asked how many statements they supported. Again, respondents were specifically instructed not to indicate which statements they supported, but simply to select the total number. The statements included the same three statements as the second group, as well as the research statement about a black Presidential candidate.
The outcome variable is respondent's attitudes towards supporting a black Presidential candidate. "Overt" estimates (from Group 1) were compared to "True" estimates (from subtracting Group 2 from Group 3) to measure social desirability bias.
Our findings suggest that education is associated with an increase in socially tolerant attitudes, rather than an increase in social desirability reporting. Respondents with high levels of education were more likely to report support for a black Presidential candidate and less likely to offer the socially desirable response than respondents with low levels of education. Given our unobtrusive estimation strategy, we believe that the large measured gap in true support between college-educated respondents and those with less than a high school education points to a serious, authentic difference in racial tolerance. This gap lends strong support to the hypothesis that education may have a liberalizing impact on measures of social tolerance, and that this observed relationship is not a spurious effect of social desirability bias.
Heerwig, Jennifer A. and Brian J. McCabe. 2009. Education and Social Desirability Bias: the Case of a Black Presidential Candidate. Social Science Quarterly 90(3): 674-688.
Versions of this research were also presented at the 2008 Midwest Political Science Association conference and the 2008 American Association of Public Opinion Researchers conference.