Return
to Research Methods Home Page
[Slightly abridged version for class use]
By William C. Adams and Dennis J. Smith
Public Opinion Quarterly 44 (Fall 1980): 389-395
In deciding how to allocate campaign resources, strategists have remarkably little scholarly research available on the effects of particular kinds of campaign activities. The large volume of election related research includes few attempts to isolate and examine the impact of specific tactics.
As grass-roots party organizations atrophy, efforts to get out the vote are increasingly conducted over the telephone, rather than house-to-house, and are often run by paid workers rather than by volunteers. Despite the large sums of money invested in these telephone operations, clear tests of their effectiveness have not been published. Instead, most research in this area concerns the effects of party activity, broadly construed, on elections in the 1950s (Eldersveld, 1956; Wolfinger, 1963; Crotty, 1971). Aside from Eldersveld's finding that personal contact was somewhat more effective than letters in increasing turnout, the effects of particular tactics were not examined.
One study that did specifically address the impact of door-to-door canvassing was that of Gerald Kramer (1970-71). Using 1952 through 1964 SRC election surveys and controlling for a variety of sociodemographic characteristics, Kramer used maximum-likelihood methods to estimate the consequences of such canvassing. Aside from the 1964 campaign, when Republican door-to-door electioneering did seem to influence preferences, Kramer found that "the primary effect of door-to-door canvassing during a presidential campaign is to increase turnout; there is little effect on voter preferences for national or local offices" (p. 572).
Kramer's findings are appealing because they affirm the conventional wisdom of campaign strategists and of political scientists: conversion effects are not easily come by, so the stress should be on "getting out the vote" of committed partisans. However, it is not at all clear that current attempts to "get out the vote" are working, as they frequently use brief telephone messages instead of a personal conversation at the door. With a design intended to isolate the independent effects of preelection telephone calls, we investigated this question.
Research Design
The District of Columbia scheduled a special election for May 1, 1979, to fill an at-large seat on the city council made vacant by Marion Barry's recent election as mayor. John Ray, a young lawyer backed by Barry, was a strong contender for the seat. Ray's chief opponent was an outspoken, controversial former council member, Rev. Douglas Moore.
Ten precincts were chosen at random from wards one, two, and three in northwest Washington -- an area thought to be generally favorable to Ray. These 10 precincts included middle-class black neighborhoods and some poorer black areas as well as one Hispanic neighborhood, several middle-class white areas, and affluent sections of Georgetown and far Northwest.
From computer-generated, alphabetical lists of registered voters in each precinct, systematic random sampling was used to select 2,650 names. The newly published May 1979 telephone directory was used to obtain current telephone numbers. If no number was available for the name selected, the next name following that sampling point was chosen. The process continued until a list of 2,650 registered voters with phone numbers was completed. Individuals were then alternately assigned to either the control or experimental group.
On Sunday and Monday prior to the Tuesday special election, calls were made to the 1,325 individuals in the experimental group. A few calls were also made on election day. Calls were made by professional telephone interviewers working from the phone bank of Smith, Bloomingdale & Hees, a Washington opinion research firm. Of the 1,325 registered voters assigned to the experimental group, 950, or 72 percent, were successfully contacted.
People who were telephoned heard a relatively brief message that identified the caller with the John Ray campaign, urged voting in the special election, noted the virtues of Ray, and suggested voting for him. The text, designed with the sanction of the Ray campaign, was read only to the selected prospective voters in the experimental group, not to spouses or others in the household.
In the District of Columbia, whether or not a person votes is a matter of public record. With the cooperation of the D.C. Board of Elections, lists of those who voted in the special election were obtained Tuesday evening immediately as the polls closed. Thus it was possible to determine precisely who voted in both the experimental and the control groups. Those in both groups who actually voted were then telephoned for an interview within three days after the special election.
The postelection interview calls from Smith, Bloomingdale & Hees were introduced as part of an academic research project being conducted by a professor at a local university. Respondents were asked if they had voted and who they had supported in the recent special election, other local political questions, and their age, education, and race. [Telephone interviews were conducted with 63 percent of those who voted both in the control group and in the experimental group.]
This study design allows an especially strong test of the impact of a professional, telephone get-out-the-vote effort. In addition, it tests the impact of brief, persuasive communication about a candidate over the telephone. Random assignment avoided the usual nonequivalency problems associated with comparing groups in nonexperimental field research. Official voting records offered an ideal measure of the impact on turnout. The quick postelection interview with voters obtained information regarding candidate choices.
Turnout Effects
The magnitude of the turnout effect was unexpected, even in light of Kramer's earlier finding that a measurable impact on turnout was associated with preelection door-to-door contacts. Of the 1,325 registered voters in the control group, 315, or 23.7 percent, actually voted. However, 310 of the 950 people contacted in the experimental group, or 32.6 percent, turned out to vote on election day. The difference in proportions is significant at the .0001 level.
One potentially serious threat to the internal validity of these findings should be noted. Out of the 1,325 individuals initially assigned to the experimental treatment, 72 percent (950) were contacted and urged to vote before the polls closed. But the differences that were found between the control group and this somewhat reduced experimental group might derive from the unavoidable loss of one-fourth of the full experimental group, rather than from the telephone calls. Factors that made the contacted group more accessible may also account for their propensity to vote.
For this alternative explanation to be rejected the behavior of the uncontacted people originally assigned to the experimental group must not vary significantly from that of the control group. In fact, there was no statistically significant difference between the proportion of those in the control group who voted (23.7 percent of 1,325) and the proportion of those not contacted and intended as members of the experimental group who voted (21.8 percent of 375). Only because the behavior of these two groups was so very similar can differences between the control group and the contacted members of the experimental group be attributed to the effects of the contact itself.
The turnout of those called from the Smith, Bloomingdale & Hees phone bank was 37.6 percent greater than the turnout of those in the control group. The absolute difference was 8.9 percent – 32.6 percent versus 23.7 percent. The follow-up survey suggests that while this difference is sizable, these figures may underestimate the impact of a preelection call.
In most of these precincts there was some modest campaign activity by supporters of John Ray. About 26 percent of those in the control group who voted said they had been called by a campaign volunteer. If these telephone messages also increased the probability of voting to the rate of the contacted experimental group (.33), then the turnout, rate for members of the control group not contacted by anyone would be lowered to closer to 20 percent. Turnout citywide, by way of comparison, was 20.3 percent of 250,750 registered voters.
Perhaps one reason why turnout was so stimulated by the calls was that they were generally well received. In the postelection survey, voters were asked their reaction to the preelection telephone call if they said they had been called. Over half (53 percent) had positive reactions. In particular, many said that they appreciated the reminder to vote in the special election. Others said they would have voted anyway, but that they still liked getting the call. Most people thought the call was a helpful, appropriate, and friendly reminder to vote.
While 53 percent positive may not be a resounding endorsement, only 12 percent offered negative remarks to the open-ended question about reactions to the preelection call. A few complained that the caller had sounded like an impersonal tape recording or that they usually did not like any kind of outside intrusion, including the postelection interview. Some Moore supporters said they liked the reminder to vote in the special election, but did not enjoy hearing the Ray propaganda.
Neutral or indifferent reactions were voiced by 34 percent of the voters who were called. Their comments indicated that they did not object to having been called, but they did not volunteer any favorable response to it.
Among those who did not vote in the election or who were not surveyed afterwards, overall reactions to the calls may have been more or less positive. However, given the sizable difference in turnout rates between those who were called and those who were not, the net effect of the telephoned reminder to vote must have been strongly positive.
Preference Effects
Ray won the special election. Throughout the District Ray received 53 percent of the votes and Moore 37 percent, with the remaining 10 percent divided among 12 minor candidates. The special election campaign had been given little coverage by Washington newspapers and television. While Moore had acquired a reputation among politically attentive citizens, newcomer John Ray and other candidates in the race were much less well known. None of the candidates had a large advertising campaign, although Ray did run commercials on selected radio stations.
The telephone message used in this study echoed the themes used in Ray's campaign: "John Ray works," and Ray is concerned about jobs, housing, and high taxes. In this election context, it seemed plausible to think that the personal persuasive messages might influence candidate preferences, especially for people leaning to candidates other than Moore. But the calls appear to have had no persuasive effect on the candidate choice of the people who voted.
In the control group, 70.2 percent voted for Ray compared to 68.4 percent among those voters who had been telephoned in the experimental group; the difference is both statistically insignificant and in the wrong direction. Separating out those in the control group who said they received a Ray call still makes no difference. Control group voters who received no campaign calls voted for Ray 70.3 percent (N=158); 70.0 percent of the 40 control group voters who said they were called by someone on behalf of Ray also voted for him. From every perspective, there is no evidence here of any relationship between campaign telephone calls and candidate preferences of voters.
Discussion
This experiment suggests that telephone efforts need not waste much time talking about the glories of a particular candidate when the chief goal is to increase turnout in a favorable area. No hint of a preference effect was found. More extended telephone appeals directed to undecided voters might produce preference effects; the brief appeal used in this study did not.
This study does not address the relative efficacy of alternative approaches to increasing turnout. For a campaign with very limited resources, radio commercials or targeted mailings or some other technique may be more cost effective. Comparisons of the relative effectiveness of various approaches await further experimentation. In this field experiment, however, one widely used technique was found extremely productive for increasing voter turnout.
A turnout effect of the magnitude found here may be confined to special elections. The relative power of one telephone call to change the salience of an election and stimulate turnout may well be greater in special elections, primaries, and other lower-turnout elections than in higher-turnout general elections. However, it seems unlikely that such a strong turnout effect would disappear altogether even in a high-turnout election; in close, intensely fought elections, the differences introduced by such tactics would remain crucial.
Results of this study were consistent with those of Kramer regarding
door-to-door canvassing. A sizable turnout effect was found, but there
was no evidence to support preference effects. The findings confirm the
wisdom of get-out-the-vote strategies aimed at targeted strongly partisan
areas or aimed at individual voters previously identified as sympathetic.
They further confirm the effectiveness of using the telephone to stimulate
turnout.
References