Introduction to the Special Issue on Cost-Effectiveness
Analysis
JANE T. BERTRAND
Communication is at the heart of behavior change programs
in developing countries. It is hard to imagine a program designed to
bring about some type of change in behavior or health status that does
not utilize at least some type of directed communication. How effective
are these interventions, however, in bringing about the desired change?
And at what cost? This special issue of the Journal of Health Communication
examines the state of the art for addressing the familiar question,
‘‘What is the bang for the buck?’’
We expect this issue to be of interest to two main audiences:
(1) health communication specialists interested in designing the most
effective programs on limited budgets, and (2) researchers/evaluators
interested in applying state-of-the-art methods for conducting cost-effectiveness
analysis of communication programs for behavior change.
Health communication programs aim to bring about change
in individual behaviors and social norms using some combination of (1)
mass media, (2) community-level interventions (e.g., community mobilization),
and (3) interpersonal communication/counseling (Piotrow, Kincaid, Rimon,
& Rinehart, 1997). Programs today tend to incorporate the highly
effective elements of entertainment education (Singhal & Rogers,
2004). Earlier evaluations often measured the effects of a program consisting
of a few or even a single communication channel in a limited geographical
area. Today’s communication programs, however, often use a more
comprehensive approach consisting of multiple, mutually reinforcing
media. For example, the national HIV/AIDS prevention programs in many
developing countries today use all three types of communication listed
above, linking messages from the mass media to community-level activities,
often through visual branding and a memorable tag line. These ‘‘full-coverage
programs’’ potentially reach all members of the population,
which is highly appealing to communication specialists wishing to achieve
the greatest possible impact. Yet the potential reach of full coverage
programs creates major methodological challenges to evaluators charged
with measuring their effectiveness. While it is of some interest to
measure the effectiveness of communication interventions in a limited
geographical setting, the real challenge is to evaluate the impact of
large-scale (national level) communication programs in which governments
and international donor agencies have invested thousands if not millions
of dollars.
In this issue, we examine the state of the art for costing
communication programs as well as measuring their effectiveness, as
necessary elements in establishing cost-effectiveness. Our review of
the literature suggests that the costing of communication programs —
where done at all — lacks standardized procedures that would allow
for comparisons across programs. The articles in this issue briefly
describe the techniques used to date in costing programs and most acknowledge
limitations in this respect, resulting in a call for greater attention
to costing in future studies.
Yet measuring effectiveness proves to be the greater challenge.
In evaluations of national-level or full-coverage communication programs,
the randomized trial — widely used in public health to measure
effectiveness — is almost never a viable option. The evaluator
cannot randomly allocate subjects to experimental (exposed) and control
(nonexposed) groups if broadcast media are involved. Even quasiexperimental
designs using comparison (nonequivalent control) groups with similar
sociodemographic characteristics to the population exposed are rarely
feasible. Other quasiexperimental designs (e.g., time series, separate
sample pretest post-test design) may be useful in tracking change but
fail to control for confounding factors, especially history (Fisher,
Foreit, Laing, Stoeckel, Townsend, et al., 2002). Thus, evaluators faced
with the challenge of full-coverage problems rely increasingly on more
sophisticated analytic techniques, including econometric methods such
as simultaneous equations models, propensity score matching, or longitudinal/panel
data methods as reflected in the articles in this issue. Some health
communication specialists will be able to follow the advanced statistical
techniques described by the authors; many will not. To this latter group,
we strongly recommend identifying collaborators who can apply these
techniques to your programs.
As the authors in this special issue indicate, the challenge
in evaluating the cost-effectiveness of health communication programs
is (1) to obtain appropriate estimates of costs, and (2) to measure
the effectiveness of the program in bringing about the desired change.
Depending on the design of the analysis, the data allow one to answer
a series of questions:
1. What is the cost per person reached by different channels
or by any channel?
2. Which channel produced the greatest change for the cost?
3. What would be the incremental change in outcome expected from a specific
increase in cost?
4. How do communication programs and their subcomponents—individually
and in combination—rank in cost-effectiveness relative to other
programs and interventions that also seek to effect behavior change?
This special issue begins with a review of the existing
literature on the cost-effectiveness of communication programs to produce
health outcomes, both in developed and developing countries. Hutchinson
and Wheeler (in the first article in this issue) identify 45 studies
that meet their criteria for rigor and relevant health outcomes. They
also outline the methodological approaches used to date in arriving
at these results. They conclude with an appeal for more systematic attention
to the measurement of costs in future studies on this subject.
Next, Guilkey, Lance, and Hutchinson provide an insightful
overview of the methodological challenges inherent in calculating the
effectiveness and cost-effectiveness of communication programs. They
identify problems inherent in this type of evaluation—such as controlling
for nonrandom exposure, national and subnational program designs, and
targeting of interventions—and they provide guidance in addressing
them. This article serves as an excellent reference for researchers
and evaluators interested in conducting cost-effectiveness analysis
(CEA) on communication programs.
The following four articles present the results from cost-effectiveness
analyses
conducted in developing countries. Kincaid and Do address the question
of cost
per new acceptor of family planning and increase in contraceptive prevalence
in
the Philippines, attributable to a nationally televised program consisting
of four
spots. In addition to giving an in-depth discussion of methods, the authors
also
emphasize the role of theory in the design and evaluation of programs. Hutchinson,
Lance, Guilkey, Shahjahan, and Haque follow with a cost-effectiveness analysis
of
the Smiling Sun campaign in Bangladesh, designed to bring about changes
in the
use of health services for improved maternal and child health:antenatal
visits and
childhood immunizations. This article is unique in evaluating both the
national-level
campaign and local promotional activities.
The final two empirical papers involve HIV/AIDS. Sweat
and coauthors Kerrigan, Moreno, Rosario, Gomez, Jerez, Weiss, and Barrington
determine the cost-effectiveness of two approaches to averting HIV infections
among commercial sex workers in the Dominican Republic:one employing
an environmental approach, the second combining the environmental approach
with a structural approach. This article is unique among this set of
articles in using an outcome measure that can be compared readily across
different types of interventions:disability adjusted life years (DALYs).
Sood and Nambiar evaluate a mass media campaign on HIV prevention in
India, focusing on the relative impact and cost-effectiveness of three
different components—atelevision drama, a reality show for youth,
and television spots. Their analysis also examines factors that mediate
behavior change in different components of the campaign.
Finally, Frick provides a commentary that describes this
set of articles in the larger context of cost-effectiveness analysis
in other areas of public health and health care utilization. He specifically
discusses the role of cost-outcome analysis in the decision-making process
for resource allocation.
This special issue provides researchers and evaluators
with an understanding of the analytic techniques needed to measure the
cost-effectiveness of comprehensive programs—with multiple, mutually
reinforcing channels designed to reach all segments of the population.
Yet it does not provide a generalizable answer to the question, what
are the most cost-effective channels for delivering a health communication
program, for several reasons. First, the number of CEA studies on health
communication programs in developing countries is still very limited.
It would be hazardous to draw conclusions from such a small sample of
studies. Second, the outcomes measured are not comparable across studies;
indeed, only the Sweat and colleagues ’article uses DALYs, a measure
that allows for comparisons across programs that is still very uncommon
in the evaluation of health communication programs. Third, the intended
audiences vary greatly, even across the four studies presented herein:
men and women of reproductive age, caretakers of children, and commercial
sex workers. The effectiveness of communication using different channels
and media formats differs depending on the characteristics of the population
(e.g. , levels of literacy, access to media, urban/rural residency,
age, sex).
Our purpose in publishing this special issue is to promote
the greater use of cost-effectiveness analysis in relation to behavior
change programs. With the eventual accumulation of studies, one can
envision the future possibility of meta-analyses that will yield valuable
data on the relative cost-effectiveness of different communication channels
or different approaches to behavior change (e.g., structural versus
environmental). Moreover, we aspire to learning more about the effectiveness
of different media channels for specific audiences. Finally, as more
data become available on the cost-effectiveness of communication programs,
we will look to such meta-analyses to provide a basis for determining
the ‘‘reasonable cost’’ for a specific health communication
intervention designed to produce a certain level of change. Because
of marked differences in the costs of media production and diffusion
in different countries and in the characteristics of intended audiences,
such data are more likely to suggest trends than provide definitive
conclusions. Yet cost-effectiveness data will contribute to more rational
decision making in the allocation of resources to communication programs.
References
Fisher, A. , Foreit, J. , Laing, J. , Stoeckel, J. , &
Townsend, J. (2002). Designing HIV/AIDS intervention studies: An
operations research handbook . Washington, DC:Population Council.
pp. 1 –44, 73.
Piotrow, P. T. , Kincaid, D. L. , Rimon, J. G. , & Rinehart, W.
E. (1997). Health communication: Lessons from family planning and
reproductive health . Praeger, CT:
Praeger. Singhal, A. & Rogers, E. (2004). The status of entertainment-education
worldwide. In A. Singhal, M. Cody, E. Rogers, & M. Sabido (Eds.
), Entertainment-education and social change: History, research,
and practice (pp. 3 –21). NJ: Lawrance Erlbaum Associates.
_____
This issue of the Journal of Health Communication was developed
with support from the Health Communication Partnership (HCP), a project
funded by the United States Agency for International Development under
Cooperative Agreement Number GPH-A-02-00008-00. Based at the Center
for Communication Programs, Johns Hopkins Bloomberg School of Public
Health, HCP works to strengthen public health in the developing world
through strategic communication programs. The opinions expressed in
this issue are those of the authors and do not necessarily reflect the
views of USAID.
Address correspondence to Jane T. Bertrand, Director of
the Center for Communication Programs, Johns Hopkins Bloomberg School
of Public Health;and Professor, Department of Health, Behavior &
Society.
_____
Scott C. Ratzan MD, MPA, MA is Editor-in-Chief of the Journal of
Health Communication: International Perspectives. He also is Vice
President, Government Affairs, Europe for Johnson & Johnson with
academic appointments at George Washington University School of Public
Health, Tufts University School of Medicine, Yale University School
of Medicine, The College of Europe, and University of Cambridge.
