Sex, lies, and the survey industry: U-M study looks at phone interviews
ANN ARBOR—People are more likely to lie about sex and money to a real person in a telephone survey than a computerized voice, University of Michigan survey researchers found.
And people are no more likely to hang up on a computerized voice than they are on the recorded voice of a real human interviewer, the study found. The study was funded by the National Institute for Child Health and Development. The findings could translate into large cost savings in the multibillion-dollar polling, survey and market research industry, with text-to-speech computerized systems much cheaper to use in large-scale telephone surveys than either live interviewers or labor-intensive, digitized recordings of human voices.
In the survey, completed by 1,396 men and women, Mick Couper, a senior associate research scientist at the U-M Institute for Social Research (ISR), and colleagues Eleanor Singer and Roger Tourangeau found no significant differences in the survey drop-out rates of people interviewed by a recorded human voice compared to those interviewed by a text-to-speech (TTS), computer-generated voice. The people surveyed were recruited by a live interviewer, then switched to either the recorded voice or the computerized voice. About 23 percent of those interviewed by the recorded voice hung up before the interview was finished, compared to 22.5 percent and 25.8 percent of those interviewed by a human-sounding and a machine-like TTS system, respectively.
The researchers also examined the effect of the interviewer’s gender on men and women’s willingness to answer sensitive survey questions about drug use and sexual behavior. “We were surprised to find that even in questions about sexual behavior, the gender of interviewers did not seem to affect either men’s or women’s answers,” said Couper, a faculty member in the new Michigan Program in Survey Methodology at ISR, the latest in a growing number of graduate programs at American universities offering certificates, master’s and doctorate degrees in the increasingly complex field of scientific survey research.
To gauge the impact of the interviewer’s voice quality on the nature of the people’s response, Couper and colleagues compared responses to sensitive questions like these: “In the past 12 months, did you buy any sexually explicit videos? If yes, press 1. If no, press zero.” “In the past 12 months, did you buy any sexually explicit magazines or books? If yes, press 1. If no, press zero.” When asked those questions by a computerized voice that sounded like a machine, nearly 15 percent of the random sample of adults surveyed pressed “1.” Only 7 percent of those asked by a live telephone interviewer admitted to buying pornographic material.
The study also provided insight on the truthfulness of survey respondents by including in its sample a list of residents who had declared personal bankruptcy in the last year. One of the questions asked was “Have you ever declared bankruptcy?” When asked by a live phone interviewer, nearly 19 percent of those who had declared bankruptcy answered “No.” When asked by a human-sounding computerized voice, only 7 percent lied. For more information about the Michigan Program on Survey Methodology, see http://www.isr.umich.edu/gradprogram/
Established in 1948, the Institute for Social Research (ISR) is among the world’s oldest survey research organizations, and a world leader in the development and application of social science methodology. ISR conducts some of the most widely-cited studies in the nation, including the Survey of Consumer Attitudes, the National Election Studies, the Monitoring the Future Study, the Panel Study of Income Dynamics, the Health and Retirement Study, the Columbia County Longitudinal Study and the National Survey of Black Americans. ISR researchers also collaborate with social scientists in more than 60 nations on the World Values Surveys and other projects, and the Institute has established formal ties with universities in Poland, China, and South Africa. Visit the ISR Web site at www.isr.umich.edu for more information. ISR is also home to the Inter-University Consortium for Political and Social Research (ICPSR), the world’s largest computerized social science data archive.