Bulletin of the World Health Organization

Strategies to increase the demand for childhood vaccination in low- and middle-income countries: a systematic review and meta-analysis

Mira Johri a, Myriam Cielo Pérez b, Catherine Arsenault c, Jitendar K Sharma d, Nitika Pant Pai e, Smriti Pahwa f & Marie-Pierre Sylvestre a

a. Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM), Tour Saint-Antoine, Porte S03-458, 850 rue St-Denis, Montréal, Québec, H2X 0A9, Canada.
b. Département d’administration de la santé, Université de Montréal, Montréal, Canada.
c. Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Canada.
d. National Health Systems Resource Centre (NHSRC), Ministry of Health and Family Welfare, New Delhi, India.
e. Department of Medicine, McGill University, Montréal, Canada.
f. Pratham Education Foundation (ASER Centre), New Delhi, India.

Correspondence to Mira Johri (email: mira.johri@umontreal.ca).

(Submitted: 08 September 2014 – Revised version received: 05 December 2014 – Accepted: 23 January 2015 – Published online: 23 March 2015.)

Bulletin of the World Health Organization 2015;93:339-346C. doi: http://dx.doi.org/10.2471/BLT.14.146951

Introduction

Almost 40 years after the launch of the World Health Organization’s (WHO’s) Expanded Programme on Immunization, one in five children worldwide still does not have access to basic vaccines.1,2 In May 2012, the World Health Assembly approved the Global Vaccine Action Plan to ensure that the full benefits of immunization are extended to people in every region, country and community.1 Midway through the Global Vaccine Action Plan’s decade of vaccines (2011–2020)1, disparities in vaccine coverage within and between countries persist,3 and the search for effective strategies to reach underserved populations has gained urgency.

Interventions to improve vaccination outcomes are commonly grouped into those targeting health services delivery or supply (e.g. improving human resources training, logistics, cold chain maintenance and vaccine storage, effective financing, monitoring and evaluation and supportive supervision) and those that stimulate demand for vaccines (e.g. monetary or food incentives, knowledge transfer or communication campaigns). Certain demand-side interventions have been associated with improved vaccine coverage of children in low- and middle-income countries in previous systematic reviews.49 However, limitations in study quality and design precluded quantitative synthesis in these reviews. The most recent review considered articles published up to 2009.9 In the interim, several new studies of potentially higher quality have been published.

We carried out a systematic review and meta-analysis to evaluate whether demand-side interventions increase uptake of routine childhood vaccination in low- and middle-income countries. Our objectives were to ascertain the effect of demand-side interventions on vaccine coverage and to identify which strategies are effective. We focused on demand-side interventions, since these strategies can more easily reach underserved populations and reduce inequities in immunization coverage.

Methods

Protocol and registration

The review protocol was registered in the PROSPERO database (CRD42013005783; available at: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42013005783). The full report, available from the authors, includes both a narrative synthesis and a meta-analysis as outlined in the protocol.

Eligibility criteria

We defined six criteria for inclusion of studies in the review and meta-analysis: (i) based on guidelines developed by the Cochrane Effective Practice and Organisation of Care (EPOC) group, randomized controlled trial, non-randomized controlled trial, controlled before-after or interrupted time series study designs were eligible;10 (ii) the study location was a low- and middle-income country as defined by the World Bank;11 (iii) the study was published in a peer-reviewed scientific journal, because in previous reviews that considered the grey literature, the quality of these studies was found to be low;47 (iv) the participants were generally parents and caregivers of children younger than two years, living in communities where interventions to increase demand for routine childhood vaccination had been carried out; (v) eligible interventions were defined broadly as any intervention that might increase demand for routine childhood vaccination, such as incentives of money or food, knowledge transfer initiatives, or communication campaigns (studies that included strategies targeting vaccine supply in addition to demand were eligible); and (vi) the study outcomes included quantitative estimates of routine childhood vaccination coverage.

Information sources

We searched the MEDLINE (via Pubmed), EMBASE, Cochrane library, POPLINE, ECONLIT, CINAHL, LILACS (Latin America and the Caribbean Center on Health Sciences Information), BDSP (French public health database), Web of Science and Scopus databases using appropriate terms and descriptors. Searches were limited to articles concerning human data that were written in English, French, German, Hindi, Portuguese or Spanish. The search began on 1 September 2008 and was last updated on 25 March 2014. All published studies included in previous systematic reviews49 were also considered. We consulted experts and reviewed article reference lists for additional articles.

Search and selection

One of the authors and an academic librarian defined the MEDLINE search strategy (available from corresponding author). Search terms were combinations of “interventions”, “programs”, “approaches”, “subsidies”, “knowledge translation”, “vouchers”, “vaccination”, “immunization”, “vaccines”, “child”, “infant”, “newborn”, “kid”, and “low- and middle-income countries”. The author translated the strategy and terms for the databases in other languages. Identified records were uploaded into EndNote (Thomson Reuters, Philadelphia, United States of America) and duplicates eliminated. Two authors screened titles and abstracts independently and eliminated studies that failed to meet eligibility criteria. Full texts of remaining studies were retrieved. Two authors independently screened full text articles against study inclusion criteria and compared results; another author validated all decisions.

Data extraction

From each study, two authors independently extracted data on study design, aims, location, population, intervention, follow-up period and outcomes, using a pre-defined template (available from corresponding author). We adopted the original study definitions of comparator or control groups. We pilot tested the template on a subset of studies. In addition to vaccine outcomes, information on equity and economic outcomes were extracted. Together, three authors crosschecked and verified these data. Study authors were contacted for clarification if data were missing or unclear.

Risk of bias

We used Cochrane collaboration criteria to assess risk of bias.12 As these criteria were developed to assess randomized controlled trials, we supplemented them to accommodate additional study designs. To address issues specific to cluster randomized controlled trials, we systematically considered recruitment bias, unit of analysis bias and sample size as part of the “other” category.12 To accommodate non-randomized designs, we used the Cochrane EPOC group’s additional criteria and scoring for non-randomized controlled trials, controlled before-and-after and interrupted time-series studies.10 For each included study, two authors independently assessed risk of bias and compared results, and another author validated all decisions.

Statistical analysis

The principal measure was the relative risk (RR) of vaccination among children in intervention versus control groups. We performed a meta-analysis by estimating random effect models with inverse variance weighting. This method gives greater weight to studies with more precise estimates. Study-specific estimates of variance were obtained by deriving standard errors from the confidence intervals reported by the studies. For cluster randomized controlled trials, we used the standard errors with adjustment for clustering, which allowed us to use both clustered and non-clustered randomized controls trials in the same meta-analyses. In addition to conducting meta-analysis on all included studies, we performed meta-analysis on five pre-specified subgroups: (i) studies including the third dose of diphtheria-tetanus-pertussis (DTP3); (ii) studies using knowledge transfer interventions; (iii) studies using incentives, (iv) studies whose risk of bias was assessed to be moderate-to-high; and (v) studies whose risk of bias was assessed to be low. We also estimated three meta-regression models adjusting separately for: (i) baseline vaccination coverage (the proportion of children aged 12–23 months receiving DTP3 in the study area or country); (ii) intervention type; and (iii) study quality. Heterogeneity was assessed using Cochrane’s Q-test and I2 statistics.12 Potential publication bias was assessed using funnel plots. Analyses were performed using the metafor package13 for meta-analyses in R (R Foundation for Statistical Computing, Vienna, Austria) and the heterogi package14 for Stata (version 13.1, StataCorp LP, College Station, USA).

Results

Study selection

Search of the databases yielded 1705 citations. We also identified 59 records through previous systematic reviews, article bibliographies and subject-matter experts. After removal of 643 duplicate records, there were 1121 records for title and abstract screening. Of these, 1073 did not meet eligibility criteria and were excluded. The full text of the remaining 48 articles was retrieved for detailed review (available from corresponding author). It lists the 37 articles excluded after full-text evaluation and the principal reasons for their exclusion. A total of 11 studies1525 comprising four randomized controlled trials, six cluster randomized controlled trials and one controlled before-and-after study were included in the review (Fig. 1 and Table 1).

Fig. 1. Interventions to increase the demand for childhood vaccination: selection of studies
Fig. 1. <b>Interventions to increase the demand for childhood vaccination: selection of studies</b>
EPOC: Effective Practice and Organisation of Care.
CINAHL: Cumulative Index to Nursing and Allied Health Literature
PubMed: Public MEDLINE
LILACS: Latin-American and Caribbean Center on Health Sciences Information
ECONLIT: The American Economic Association’s electronic bibliography
POPLINE: Free resource from the John Hopkins Bloomberg School of Public Health
EMBASE: Elsevier’s biomedical database
BDSP : Banque de Données en Santé Publique

Study characteristics

Study inclusion

The 11 studies were published in English between 1996 and 2013; eight were from lower-middle income countries1522 and three were from low-income countries.2325 Study locations included south Asia (n = 7),1520,23 sub-Saharan Africa (n = 3),22,24,25 and central America (n = 1).21

Participants

Data were collected from 11 512 participants yielding outcomes for 11 512 children. As many interventions were directed to communities or populations rather than to individuals, the number of individuals reached by the interventions was considerably larger. Participants were mothers, caregivers, households of children who were within the target age-group for immunization (n = 9),1518,2125 or the general populations of target communities (n = 2).19,20

Interventions

Of the 11 studies, seven described education or knowledge translation interventions,1517,19,20,22,23 while four described interventions using incentives.18,21,24,25 Of the latter, two considered monetary incentives,21,24 and two non-monetary incentives.18,25 One study compared two types of monetary incentives;24 thus, the 11 studies yielded data on 12 interventions. Four studies considered both demand and supply interventions to improve vaccine coverage.1618,21 For these studies, meta-analyses were based on the estimated demand-side effect.1618,21

Outcomes

Nine studies undertook coverage surveys to assess outcomes,15,1825 while two used administrative data.16,17 For six studies, the main aim of the intervention was to increase immunization coverage,1519,22 while for five studies, improving immunization coverage was a secondary aim and data on immunization outcomes were included.20,21,2325

Vaccination outcomes

Four studies described full immunization as defined by the country’s immunization schedule, all of which contain DTP3 as a subset.18,2224 Six studies presented information on DTP3 vaccination.1517,2224 Three studies presented information on receipt of one or more vaccine doses18,20,21 and one considered age-appropriate vaccination.25 The timing of outcome measures was variable. While some studies addressed on-time delivery, a majority focused on the simpler assessment of presence or absence of vaccinations within a specified period. This period was based on the age of the child at the time outcomes were assessed and varied between studies: less than one year,1517,20,25 less than two years,19,22 less than three years,18,21 or less than five years.23,24

One study recorded changes in immunization knowledge, attitudes and beliefs19 and two estimated intervention costs and cost–effectiveness,18,19 as additional vaccination-related outcomes. All studies considered equity in the choice of target populations by directing interventions to areas of greater need, but only two provided stratified analyses related to subgroups of interest.17,20

Risk of bias

We assessed risk of bias for nine criteria. Our assessments ranged from low risk of bias on all criteria in one study18 to high risk of bias on five criteria in one study.25 For the purpose of subgroup analyses, we classified five studies with high risk of bias on one or more criteria as moderate-to-high risk of bias.20,21,2325 The remaining six studies were categorised as low risk of bias.1519,22 Fig. 2 presents a summary of our assessment of the risk of bias (a detailed assessment for each study and criterion, and figure summarizing risk of bias assessments by criterion is available from corresponding author) High risk of bias occurred most frequently for the category “other bias” (five studies), while unclear risk of bias was most frequent for the category “selective reporting” (nine studies). Risk of bias was related to intervention type: three of the four studies that used incentives had moderate-to-high risk of bias.21,24,25

Fig. 2. Interventions to increase the demand for childhood vaccination: summary of the risk of study bias
Fig. 2. <b>Interventions to increase the demand for childhood vaccination: summary of the risk of study bias</b>

Receipt of vaccine

RR estimates, 95% confidence intervals (CI) and weights for each individual study are shown in Fig. 3. For each study, the crude proportion of participants with and without vaccination is shown separately for intervention and control groups. For one study, Morris et al.,21 data necessary for meta-analysis were not available in the text or from the study author and were taken from a published article.26

Fig. 3. Interventions to increase the demand for childhood vaccination: meta-analysis of 11 studies
Fig. 3. <b>Interventions to increase the demand for childhood vaccination: meta-analysis of 11 studies</b>
CI: confidence interval; RR: relative risk.
Note: Weights are from random-effects analysis.

In the pooled analysis considering all 12 interventions assessed in the 11 studies (Fig. 3), interventions were associated with significantly higher receipt of vaccines (RR: 1.30; 95% CI: 1.17–1.44), but there was considerable heterogeneity (I2 = 88.96; 95% CI: 82.94– 92.16). Unweighted results were similar to weighted results (RR: 1.34; 95% CI: 1.21−1.50). To reduce diversity associated with outcomes measured, we considered only the nine studies reporting data for DTP3 or full vaccination (information available from corresponding authors). These results were similar to those for all studies (RR: 1.32; 95% CI: 1.15–1.51; I2 = 86.31; 95% CI: 75.31–91.11).

Bias across studies

To explore potential sources of heterogeneity, we plotted the observed outcome against the standard error for all meta-analyses and meta-regressions. Visual inspection of funnel plots revealed no evidence of asymmetry (available from corresponding author); however, due to the small number of studies the test has low power and we cannot exclude the possibility of publication bias.

Subgroup analyses

The overall effect of educational or knowledge translation interventions was greater than for incentives RR: 1.40 (95% CI: 1.20–1.63) versus RR: 1.28 (95% CI: 1.12–1.45), respectively (available from corresponding author). The pooled analysis considering studies with a low risk of bias yielded a higher estimate of overall effect than that for studies with moderate-to-high risk of bias, RR: 1.53 (95% CI: 1.28−1.82) and RR: 1.15 (95% CI: 1.02–1.30), respectively (available from corresponding author). Heterogeneity was lower for educational or knowledge translation interventions: I2 = 80.48, (95% CI: 53.9088.88) than for incentives I2 = 89.40, (95% CI: 76.83–93.73). Heterogeneity was also lower for studies with low risk of bias: I2 = 79.16, (95% CI: 42.92−88.82) versus studies with moderate-to-high risk of bias: I2 = 90.26, (95% CI: 81.40–93.86). Meta-regression revealed no important differences in results by baseline coverage level. Results for meta-regressions by intervention type and study quality were consistent with the corresponding meta-analyses.

Discussion

This meta-analysis demonstrates that demand-side interventions lead to an increase in child vaccination coverage in diverse low- and middle-income settings and among communities with lagging health and social indicators. Immunization programmes have often focused on strengthening vaccine supply. Our results show that policymakers who seek to increase access to vaccines through the routine immunization system should also consider demand-side strategies.

Educational or knowledge translation interventions were more effective on average than interventions based on incentives; however, both strategies were effective. The highest estimates of effect and lowest heterogeneity were found among studies with low risk of bias. Risk of bias was related to intervention type, with three of the four incentives studies having moderate-to-high risk of bias.

Our analysis has six important limitations. First, vaccination coverage is shaped by interrelated supply and demand factors. Many studies, particularly those tackling vaccine and non-vaccine outcomes,20,21,2325 did not give due consideration to the role of the health system in delivering vaccines during the study design phase leading to increased risk of bias. Three supply-side challenges confronted the studies reviewed: (i) increases in demand cannot be effective if supply-side constraints limit provision of vaccines;21,25 (ii) if coverage rates are already very high, there is limited scope to demonstrate improvement;24,25 and (iii) vaccination rates can be affected by changes in service delivery occurring independently of the study. Among the studies reviewed, only one study did not show a gain associated with the intervention: in this study, all experimental groups experienced substantial but similar gains in vaccination coverage.23 Another study noted a likely background trend with a differential impact on experimental groups.25 Unfortunately, neither of these studies collected data on health system trends that might affect immunization delivery. Studies addressing vaccine and non-vaccine outcomes, including three of the four incentive based studies21,24,25 may have faced challenges related to broad study scope or a lack of vaccine-specific expertise in study planning. In addition, two studies of financial incentives21,24 were large-scale interventions and faced substantial implementation challenges.

Second, the systematic review and meta-analysis included only 11 studies. This limited our ability to explore potential sources of heterogeneity quantitatively and to exclude possible biases related to publication and study size.

Third, studies measured different vaccines over different time periods. Receipt of DTP3 is the measure preferred by international agencies to assess immunization coverage and was included in nine of the 11 papers reviewed.3 All analyses consistently showed that demand-side interventions are beneficial in improving coverage. However, due to diversity in outcomes and the small number of studies, we cannot evaluate the effect of interventions for specific vaccines.

Fourth, health gains depend not only on increased vaccine coverage but also on appropriate timing of vaccination. With one exception,25 studies offered little information on when doses were delivered.

Fifth, despite contacting the authors, we were not able to retrieve data for all studies and had to use approximate values for one study21 based on another publication.26

Sixth, our analysis excluded interventions that focused exclusively on improving vaccine supply. However, from a policy point of view, how demand-side interventions interact with supply side constraints is also important. With one exception,18 immunization system performance was not explicitly assessed by these studies.

Our results indicate that future research on demand-side interventions to increase vaccine coverage should (i) standardize measurement of outcomes; (ii) include vaccination experts during the study design phase; and (iii) collect data on health system characteristics that may affect vaccine delivery.

Conclusion

Demand-side interventions are effective in improving the uptake of childhood vaccines delivered through routine immunization services in low- and middle-income countries. Our results are more definitive than those of previous systematic reviews which employed narrative synthesis techniques.49

Demand-side strategies to improve vaccination coverage are important because they are inherently equity-oriented and address specific barriers to coverage related to financial constraints, opportunity costs, knowledge and prioritization. Future research should seek to refine our understanding of which approaches are most effective in specific contexts. Studies investigating the value of knowledge translation and incentives-based interventions offered in combination are also required. Studies that simultaneously consider supply- and demand-side interventions – and enable us to evaluate their relative effectiveness – are of particular interest. Finally, studies should consider whether interventions can be delivered effectively at scale and in the long term.


Acknowledgements

We thank Diego Bassani, Hospital for Sick Kids, Toronto, Canada.

Funding:

The Canadian Institutes for Health Research (299960) and the Bill & Melinda Gates Foundation (OPP1067851) funded this study.

Competing interests:

None declared.

References