Strategies to increase the demand for childhood vaccination in low- and middle-income countries: a systematic review and meta-analysis
Mira Johri a, Myriam Cielo Pérez b, Catherine Arsenault c, Jitendar K Sharma d, Nitika Pant Pai e, Smriti Pahwa f & Marie-Pierre Sylvestre a
a. Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM), Tour Saint-Antoine, Porte S03-458, 850 rue St-Denis, Montréal, Québec, H2X 0A9, Canada.
b. Département d’administration de la santé, Université de Montréal, Montréal, Canada.
c. Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Canada.
d. National Health Systems Resource Centre (NHSRC), Ministry of Health and Family Welfare, New Delhi, India.
e. Department of Medicine, McGill University, Montréal, Canada.
f. Pratham Education Foundation (ASER Centre), New Delhi, India.
Correspondence to Mira Johri (email: firstname.lastname@example.org).
(Submitted: 08 September 2014 – Revised version received: 05 December 2014 – Accepted: 23 January 2015 – Published online: 23 March 2015.)
Bulletin of the World Health Organization 2015;93:339-346C. doi: http://dx.doi.org/10.2471/BLT.14.146951
Almost 40 years after the launch of the World Health Organization’s (WHO’s) Expanded Programme on Immunization, one in five children worldwide still does not have access to basic vaccines.1,2 In May 2012, the World Health Assembly approved the Global Vaccine Action Plan to ensure that the full benefits of immunization are extended to people in every region, country and community.1 Midway through the Global Vaccine Action Plan’s decade of vaccines (2011–2020)1, disparities in vaccine coverage within and between countries persist,3 and the search for effective strategies to reach underserved populations has gained urgency.
Interventions to improve vaccination outcomes are commonly grouped into those targeting health services delivery or supply (e.g. improving human resources training, logistics, cold chain maintenance and vaccine storage, effective financing, monitoring and evaluation and supportive supervision) and those that stimulate demand for vaccines (e.g. monetary or food incentives, knowledge transfer or communication campaigns). Certain demand-side interventions have been associated with improved vaccine coverage of children in low- and middle-income countries in previous systematic reviews.4–9 However, limitations in study quality and design precluded quantitative synthesis in these reviews. The most recent review considered articles published up to 2009.9 In the interim, several new studies of potentially higher quality have been published.
We carried out a systematic review and meta-analysis to evaluate whether demand-side interventions increase uptake of routine childhood vaccination in low- and middle-income countries. Our objectives were to ascertain the effect of demand-side interventions on vaccine coverage and to identify which strategies are effective. We focused on demand-side interventions, since these strategies can more easily reach underserved populations and reduce inequities in immunization coverage.
Protocol and registration
The review protocol was registered in the PROSPERO database (CRD42013005783; available at: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42013005783). The full report, available from the authors, includes both a narrative synthesis and a meta-analysis as outlined in the protocol.
We defined six criteria for inclusion of studies in the review and meta-analysis: (i) based on guidelines developed by the Cochrane Effective Practice and Organisation of Care (EPOC) group, randomized controlled trial, non-randomized controlled trial, controlled before-after or interrupted time series study designs were eligible;10 (ii) the study location was a low- and middle-income country as defined by the World Bank;11 (iii) the study was published in a peer-reviewed scientific journal, because in previous reviews that considered the grey literature, the quality of these studies was found to be low;4–7 (iv) the participants were generally parents and caregivers of children younger than two years, living in communities where interventions to increase demand for routine childhood vaccination had been carried out; (v) eligible interventions were defined broadly as any intervention that might increase demand for routine childhood vaccination, such as incentives of money or food, knowledge transfer initiatives, or communication campaigns (studies that included strategies targeting vaccine supply in addition to demand were eligible); and (vi) the study outcomes included quantitative estimates of routine childhood vaccination coverage.
We searched the MEDLINE (via Pubmed), EMBASE, Cochrane library, POPLINE, ECONLIT, CINAHL, LILACS (Latin America and the Caribbean Center on Health Sciences Information), BDSP (French public health database), Web of Science and Scopus databases using appropriate terms and descriptors. Searches were limited to articles concerning human data that were written in English, French, German, Hindi, Portuguese or Spanish. The search began on 1 September 2008 and was last updated on 25 March 2014. All published studies included in previous systematic reviews4–9 were also considered. We consulted experts and reviewed article reference lists for additional articles.
Search and selection
One of the authors and an academic librarian defined the MEDLINE search strategy (available from corresponding author). Search terms were combinations of “interventions”, “programs”, “approaches”, “subsidies”, “knowledge translation”, “vouchers”, “vaccination”, “immunization”, “vaccines”, “child”, “infant”, “newborn”, “kid”, and “low- and middle-income countries”. The author translated the strategy and terms for the databases in other languages. Identified records were uploaded into EndNote (Thomson Reuters, Philadelphia, United States of America) and duplicates eliminated. Two authors screened titles and abstracts independently and eliminated studies that failed to meet eligibility criteria. Full texts of remaining studies were retrieved. Two authors independently screened full text articles against study inclusion criteria and compared results; another author validated all decisions.
From each study, two authors independently extracted data on study design, aims, location, population, intervention, follow-up period and outcomes, using a pre-defined template (available from corresponding author). We adopted the original study definitions of comparator or control groups. We pilot tested the template on a subset of studies. In addition to vaccine outcomes, information on equity and economic outcomes were extracted. Together, three authors crosschecked and verified these data. Study authors were contacted for clarification if data were missing or unclear.
Risk of bias
We used Cochrane collaboration criteria to assess risk of bias.12 As these criteria were developed to assess randomized controlled trials, we supplemented them to accommodate additional study designs. To address issues specific to cluster randomized controlled trials, we systematically considered recruitment bias, unit of analysis bias and sample size as part of the “other” category.12 To accommodate non-randomized designs, we used the Cochrane EPOC group’s additional criteria and scoring for non-randomized controlled trials, controlled before-and-after and interrupted time-series studies.10 For each included study, two authors independently assessed risk of bias and compared results, and another author validated all decisions.
The principal measure was the relative risk (RR) of vaccination among children in intervention versus control groups. We performed a meta-analysis by estimating random effect models with inverse variance weighting. This method gives greater weight to studies with more precise estimates. Study-specific estimates of variance were obtained by deriving standard errors from the confidence intervals reported by the studies. For cluster randomized controlled trials, we used the standard errors with adjustment for clustering, which allowed us to use both clustered and non-clustered randomized controls trials in the same meta-analyses. In addition to conducting meta-analysis on all included studies, we performed meta-analysis on five pre-specified subgroups: (i) studies including the third dose of diphtheria-tetanus-pertussis (DTP3); (ii) studies using knowledge transfer interventions; (iii) studies using incentives, (iv) studies whose risk of bias was assessed to be moderate-to-high; and (v) studies whose risk of bias was assessed to be low. We also estimated three meta-regression models adjusting separately for: (i) baseline vaccination coverage (the proportion of children aged 12–23 months receiving DTP3 in the study area or country); (ii) intervention type; and (iii) study quality. Heterogeneity was assessed using Cochrane’s Q-test and I2 statistics.12 Potential publication bias was assessed using funnel plots. Analyses were performed using the metafor package13 for meta-analyses in R (R Foundation for Statistical Computing, Vienna, Austria) and the heterogi package14 for Stata (version 13.1, StataCorp LP, College Station, USA).
Search of the databases yielded 1705 citations. We also identified 59 records through previous systematic reviews, article bibliographies and subject-matter experts. After removal of 643 duplicate records, there were 1121 records for title and abstract screening. Of these, 1073 did not meet eligibility criteria and were excluded. The full text of the remaining 48 articles was retrieved for detailed review (available from corresponding author). It lists the 37 articles excluded after full-text evaluation and the principal reasons for their exclusion. A total of 11 studies15–25 comprising four randomized controlled trials, six cluster randomized controlled trials and one controlled before-and-after study were included in the review (Fig. 1 and Table 1).
Fig. 1. Interventions to increase the demand for childhood vaccination: selection of studies
Table 1. Studies included in systematic review and meta-analysis of strategies to increase the demand for childhood vaccination in low- and middle-income countries
The 11 studies were published in English between 1996 and 2013; eight were from lower-middle income countries15–22 and three were from low-income countries.23–25 Study locations included south Asia (n = 7),15–20,23 sub-Saharan Africa (n = 3),22,24,25 and central America (n = 1).21
Data were collected from 11 512 participants yielding outcomes for 11 512 children. As many interventions were directed to communities or populations rather than to individuals, the number of individuals reached by the interventions was considerably larger. Participants were mothers, caregivers, households of children who were within the target age-group for immunization (n = 9),15–18,21–25 or the general populations of target communities (n = 2).19,20
Of the 11 studies, seven described education or knowledge translation interventions,15–17,19,20,22,23 while four described interventions using incentives.18,21,24,25 Of the latter, two considered monetary incentives,21,24 and two non-monetary incentives.18,25 One study compared two types of monetary incentives;24 thus, the 11 studies yielded data on 12 interventions. Four studies considered both demand and supply interventions to improve vaccine coverage.16–18,21 For these studies, meta-analyses were based on the estimated demand-side effect.16–18,21
Nine studies undertook coverage surveys to assess outcomes,15,18–25 while two used administrative data.16,17 For six studies, the main aim of the intervention was to increase immunization coverage,15–19,22 while for five studies, improving immunization coverage was a secondary aim and data on immunization outcomes were included.20,21,23–25
Four studies described full immunization as defined by the country’s immunization schedule, all of which contain DTP3 as a subset.18,22–24 Six studies presented information on DTP3 vaccination.15–17,22–24 Three studies presented information on receipt of one or more vaccine doses18,20,21 and one considered age-appropriate vaccination.25 The timing of outcome measures was variable. While some studies addressed on-time delivery, a majority focused on the simpler assessment of presence or absence of vaccinations within a specified period. This period was based on the age of the child at the time outcomes were assessed and varied between studies: less than one year,15–17,20,25 less than two years,19,22 less than three years,18,21 or less than five years.23,24
One study recorded changes in immunization knowledge, attitudes and beliefs19 and two estimated intervention costs and cost–effectiveness,18,19 as additional vaccination-related outcomes. All studies considered equity in the choice of target populations by directing interventions to areas of greater need, but only two provided stratified analyses related to subgroups of interest.17,20
Risk of bias
We assessed risk of bias for nine criteria. Our assessments ranged from low risk of bias on all criteria in one study18 to high risk of bias on five criteria in one study.25 For the purpose of subgroup analyses, we classified five studies with high risk of bias on one or more criteria as moderate-to-high risk of bias.20,21,23–25 The remaining six studies were categorised as low risk of bias.15–19,22 Fig. 2 presents a summary of our assessment of the risk of bias (a detailed assessment for each study and criterion, and figure summarizing risk of bias assessments by criterion is available from corresponding author) High risk of bias occurred most frequently for the category “other bias” (five studies), while unclear risk of bias was most frequent for the category “selective reporting” (nine studies). Risk of bias was related to intervention type: three of the four studies that used incentives had moderate-to-high risk of bias.21,24,25
Fig. 2. Interventions to increase the demand for childhood vaccination: summary of the risk of study bias
Receipt of vaccine
RR estimates, 95% confidence intervals (CI) and weights for each individual study are shown in Fig. 3. For each study, the crude proportion of participants with and without vaccination is shown separately for intervention and control groups. For one study, Morris et al.,21 data necessary for meta-analysis were not available in the text or from the study author and were taken from a published article.26
Fig. 3. Interventions to increase the demand for childhood vaccination: meta-analysis of 11 studies
In the pooled analysis considering all 12 interventions assessed in the 11 studies (Fig. 3), interventions were associated with significantly higher receipt of vaccines (RR: 1.30; 95% CI: 1.17–1.44), but there was considerable heterogeneity (I2 = 88.96; 95% CI: 82.94– 92.16). Unweighted results were similar to weighted results (RR: 1.34; 95% CI: 1.21−1.50). To reduce diversity associated with outcomes measured, we considered only the nine studies reporting data for DTP3 or full vaccination (information available from corresponding authors). These results were similar to those for all studies (RR: 1.32; 95% CI: 1.15–1.51; I2 = 86.31; 95% CI: 75.31–91.11).
Bias across studies
To explore potential sources of heterogeneity, we plotted the observed outcome against the standard error for all meta-analyses and meta-regressions. Visual inspection of funnel plots revealed no evidence of asymmetry (available from corresponding author); however, due to the small number of studies the test has low power and we cannot exclude the possibility of publication bias.
The overall effect of educational or knowledge translation interventions was greater than for incentives RR: 1.40 (95% CI: 1.20–1.63) versus RR: 1.28 (95% CI: 1.12–1.45), respectively (available from corresponding author). The pooled analysis considering studies with a low risk of bias yielded a higher estimate of overall effect than that for studies with moderate-to-high risk of bias, RR: 1.53 (95% CI: 1.28−1.82) and RR: 1.15 (95% CI: 1.02–1.30), respectively (available from corresponding author). Heterogeneity was lower for educational or knowledge translation interventions: I2 = 80.48, (95% CI: 53.90–88.88) than for incentives I2 = 89.40, (95% CI: 76.83–93.73). Heterogeneity was also lower for studies with low risk of bias: I2 = 79.16, (95% CI: 42.92−88.82) versus studies with moderate-to-high risk of bias: I2 = 90.26, (95% CI: 81.40–93.86). Meta-regression revealed no important differences in results by baseline coverage level. Results for meta-regressions by intervention type and study quality were consistent with the corresponding meta-analyses.
This meta-analysis demonstrates that demand-side interventions lead to an increase in child vaccination coverage in diverse low- and middle-income settings and among communities with lagging health and social indicators. Immunization programmes have often focused on strengthening vaccine supply. Our results show that policymakers who seek to increase access to vaccines through the routine immunization system should also consider demand-side strategies.
Educational or knowledge translation interventions were more effective on average than interventions based on incentives; however, both strategies were effective. The highest estimates of effect and lowest heterogeneity were found among studies with low risk of bias. Risk of bias was related to intervention type, with three of the four incentives studies having moderate-to-high risk of bias.
Our analysis has six important limitations. First, vaccination coverage is shaped by interrelated supply and demand factors. Many studies, particularly those tackling vaccine and non-vaccine outcomes,20,21,23–25 did not give due consideration to the role of the health system in delivering vaccines during the study design phase leading to increased risk of bias. Three supply-side challenges confronted the studies reviewed: (i) increases in demand cannot be effective if supply-side constraints limit provision of vaccines;21,25 (ii) if coverage rates are already very high, there is limited scope to demonstrate improvement;24,25 and (iii) vaccination rates can be affected by changes in service delivery occurring independently of the study. Among the studies reviewed, only one study did not show a gain associated with the intervention: in this study, all experimental groups experienced substantial but similar gains in vaccination coverage.23 Another study noted a likely background trend with a differential impact on experimental groups.25 Unfortunately, neither of these studies collected data on health system trends that might affect immunization delivery. Studies addressing vaccine and non-vaccine outcomes, including three of the four incentive based studies21,24,25 may have faced challenges related to broad study scope or a lack of vaccine-specific expertise in study planning. In addition, two studies of financial incentives21,24 were large-scale interventions and faced substantial implementation challenges.
Second, the systematic review and meta-analysis included only 11 studies. This limited our ability to explore potential sources of heterogeneity quantitatively and to exclude possible biases related to publication and study size.
Third, studies measured different vaccines over different time periods. Receipt of DTP3 is the measure preferred by international agencies to assess immunization coverage and was included in nine of the 11 papers reviewed.3 All analyses consistently showed that demand-side interventions are beneficial in improving coverage. However, due to diversity in outcomes and the small number of studies, we cannot evaluate the effect of interventions for specific vaccines.
Fourth, health gains depend not only on increased vaccine coverage but also on appropriate timing of vaccination. With one exception,25 studies offered little information on when doses were delivered.
Sixth, our analysis excluded interventions that focused exclusively on improving vaccine supply. However, from a policy point of view, how demand-side interventions interact with supply side constraints is also important. With one exception,18 immunization system performance was not explicitly assessed by these studies.
Our results indicate that future research on demand-side interventions to increase vaccine coverage should (i) standardize measurement of outcomes; (ii) include vaccination experts during the study design phase; and (iii) collect data on health system characteristics that may affect vaccine delivery.
Demand-side interventions are effective in improving the uptake of childhood vaccines delivered through routine immunization services in low- and middle-income countries. Our results are more definitive than those of previous systematic reviews which employed narrative synthesis techniques.4–9
Demand-side strategies to improve vaccination coverage are important because they are inherently equity-oriented and address specific barriers to coverage related to financial constraints, opportunity costs, knowledge and prioritization. Future research should seek to refine our understanding of which approaches are most effective in specific contexts. Studies investigating the value of knowledge translation and incentives-based interventions offered in combination are also required. Studies that simultaneously consider supply- and demand-side interventions – and enable us to evaluate their relative effectiveness – are of particular interest. Finally, studies should consider whether interventions can be delivered effectively at scale and in the long term.
We thank Diego Bassani, Hospital for Sick Kids, Toronto, Canada.
The Canadian Institutes for Health Research (299960) and the Bill & Melinda Gates Foundation (OPP1067851) funded this study.
- Decade of Vaccines Collaboration 2012. Global Vaccine Action Plan 2011-2020. Geneva: World Health Organization; 2013. Available from: http://www.who.int/iris/bitstream/10665/78141/1/9789241504980_eng.pdf?ua=1 [cited 2015 March 3].
- Narrowing the gaps to meet the goals. New York: United Nations Children’s Fund; 2010. Available from: http://www.unicef.org/publications/files/Narrowing_the_Gaps_to_Meet_the_Goals_090310_2a.pdf [cited 2015 Mar 23].
- Global immunization data. Geneva: World Health Organization; 2014. Available from: http://www.who.int/immunization/monitoring_surveillance/global_immunization_data.pdf [cited 2015 Mar 23].
- Batt K, Fox-Rushby JA, Castillo-Riquelme M. The costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle-income countries: systematic review of the grey literature. Bull World Health Organ. 2004 Sep;82(9):689–96. pmid: 15628207
- Pegurri E, Fox-Rushby JA, Damian W. The effects and costs of expanding the coverage of immunisation services in low- and middle-income countries: a systematic literature review. Vaccine. 2005 Feb 18;23(13):1624–35. http://dx.doi.org/10.1016/j.vaccine.2004.02.029 pmid: 15694515
- Haines A, Sanders D, Lehmann U, Rowe AK, Lawn JE, Jan S, et al. Achieving child survival goals: potential contribution of community health workers. Lancet. 2007 Jun 23;369(9579):2121–31. http://dx.doi.org/10.1016/S0140-6736(07)60325-0 pmid: 17586307
- Ryman TK, Dietz V, Cairns KL. Too little but not too late: results of a literature review to improve routine immunization programs in low- and middle-income countries. BMC Health Serv Res. 2008;8(1):134. http://dx.doi.org/10.1186/1472-6963-8-134 pmid: 18570677
- Shea B, Andersson N, Henry D. Increasing the demand for childhood vaccination in low- and middle-income countries: a systematic review. BMC Int Health Hum Rights. 2009;9 Suppl 1:S5. http://dx.doi.org/10.1186/1472-698X-9-S1-S5 pmid: 19828063
- Oyo-Ita A, Nwachukwu CE, Oringanje C, Meremikwu MM. Interventions for improving coverage of child immunization in low- and middle-income countries. Cochrane Database Syst Rev. 2011; (7):CD008145. pmid: 21735423
- Effective Practice and Organisation of Care (EPOC). What study designs should be included in an EPOC review and what should they be called? EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2015. Available from: http://epoc.cochrane.org/epoc-specific-resources-review-authors [cited 2015 Mar 23].
- Country and lending groups. Data & statistics: country classification. Washington: The World Bank; 2013.
- Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Oxford: The Cochrane Collaboration; 2011. Available from: http://handbook.cochrane.org/ [cited 2015 April 10].
- Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36:1–48.
- Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002 Jun 15;21(11):1539–58. http://dx.doi.org/10.1002/sim.1186 pmid: 12111919
- Owais A, Hanif B, Siddiqui AR, Agha A, Zaidi AK. Does improving maternal knowledge of vaccines impact infant immunization rates? A community-based randomized-controlled trial in Karachi, Pakistan. BMC Public Health. 2011;11(1):239. http://dx.doi.org/10.1186/1471-2458-11-239 pmid: 21496343
- Usman HR, Akhtar S, Habib F, Jehan I. Redesigned immunization card and center-based education to reduce childhood immunization dropouts in urban Pakistan: a randomized controlled trial. Vaccine. 2009 Jan 14;27(3):467–72. http://dx.doi.org/10.1016/j.vaccine.2008.10.048 pmid: 18996423
- Usman HR, Rahbar MH, Kristensen S, Vermund SH, Kirby RS, Habib F, et al. Randomized controlled trial to improve childhood immunization adherence in rural Pakistan: redesigned immunization card and maternal education. Trop Med Int Health. 2011 Mar;16(3):334–42. http://dx.doi.org/10.1111/j.1365-3156.2010.02698.x pmid: 21159080
- Banerjee AV, Duflo E, Glennerster R, Kothari D. Improving immunisation coverage in rural India: clustered randomised controlled evaluation of immunisation campaigns with and without incentives. BMJ. 2010 May 17;340(1):c2220. http://dx.doi.org/10.1136/bmj.c2220 pmid: 20478960
- Andersson N, Cockcroft A, Ansari NM, Omer K, Baloch M, Ho Foster A, et al. Evidence-based discussion increases childhood vaccination uptake: a randomised cluster controlled trial of knowledge translation in Pakistan. BMC Int Health Hum Rights. 2009;9 Suppl 1:S8. http://dx.doi.org/10.1186/1472-698X-9-S1-S8 pmid: 19828066
- Pandey P, Sehgal AR, Riboud M, Levine D, Goyal M. Informing resource-poor populations and the delivery of entitled health and social services in rural India: a cluster randomized controlled trial. JAMA. 2007 Oct 24;298(16):1867–75. http://dx.doi.org/10.1001/jama.298.16.1867 pmid: 17954538
- Morris SS, Flores R, Olinto P, Medina JM. Monetary incentives in primary health care and effects on use and coverage of preventive health care interventions in rural Honduras: cluster randomised trial. Lancet. 2004 Dec 4-10;364(9450):2030–7. http://dx.doi.org/10.1016/S0140-6736(04)17515-6 pmid: 15582060
- Brugha RF, Kevany JP. Maximizing immunization coverage through home visits: a controlled trial in an urban area of Ghana. Bull World Health Organ. 1996;74(5):517–24. pmid: 9002332
- Roy SK, Bilkes F, Islam K, Ara G, Tanner P, Wosk I, et al. Impact of pilot project of Rural Maintenance Programme (RMP) on destitute women: CARE, Bangladesh. Food Nutr Bull. 2008 Mar;29(1):67–75. pmid: 18510207
- Robertson L, Mushati P, Eaton JW, Dumba L, Mavise G, Makoni J, et al. Effects of unconditional and conditional cash transfers on child health and development in Zimbabwe: a cluster-randomised trial. Lancet. 2013 Apr 13;381(9874):1283–92. http://dx.doi.org/10.1016/S0140-6736(12)62168-0 pmid: 23453283
- Briere EC, Ryman TK, Cartwright E, Russo ET, Wannemuehler KA, Nygren BL, et al. Impact of integration of hygiene kit distribution with routine immunizations on infant vaccine coverage and water treatment and handwashing practices of Kenyan mothers. J Infect Dis. 2012 Mar;205 Suppl 1:S56–64. http://dx.doi.org/10.1093/infdis/jir779 pmid: 22315387
- Bassani DG, Arora P, Wazny K, Gaffey MF, Lenters L, Bhutta ZA. Financial incentives and coverage of child health interventions: a systematic review and meta-analysis. BMC Public Health. 2013;13 Suppl 3:S30. http://dx.doi.org/10.1186/1471-2458-13-S3-S30 pmid: 24564520