Bulletin of the World Health Organization

Estimation of maternal and child mortality one year after user-fee elimination: an impact evaluation and modelling study in Burkina Faso

Mira Johri a, Valéry Ridde b, Rolf Heinmüller a & Slim Haddad a

a. Centre de Recherche du Centre Hospitalier de l’Université de Montréal, Tour Saint-Antoine (Porte S03-458), 850 rue St-Denis, Montréal, Québec, H2X 0A9, Canada.
b. Institut de Recherche en Sciences de la Santé, Centre national de la recherche scientifique et technologique du Burkina Faso, Ouagadougou, Burkina Faso.

Correspondence to Mira Johri (email: mira.johri@umontreal.ca).

(Submitted: 21 September 2013 – Revised version received: 04 March 2014 – Accepted: 17 March 2014 – Published online: 03 September 2014.)

Bulletin of the World Health Organization 2014;92:706-715. doi: http://dx.doi.org/10.2471/BLT.13.130609

Introduction

Direct charges for health care at the point of use (user fees), can limit access to health services and push households into poverty.1 Many governments nonetheless continue to rely heavily on user charges to finance health systems. This is especially so in low-income countries, where the health sector is usually under funded.1 A recent survey of 50 high-mortality countries in Africa and Asia found that 44 of these countries continue to levy user fees for health care.2

User fees may also hinder the achievement of the health-related Millennium Development Goals (MDGs). Of the 75 countries where more than 95% of all maternal and child deaths occur, only 23 are on track to achieve MDG 4 (reduce the mortality rate by two thirds among children less than five years of age) and only nine are on track to achieve MDG 5 (reduce the maternal mortality ratio by three quarters).3

Eliminating user fees for pregnant women and children has been suggested as a strategy to increase coverage of high-impact interventions to achieve the MDGs and to move towards universal health coverage.46 In 2010, the African Union called for the elimination of user fees for children less than five years of age,7 and several African countries have now adopted this approach.2

Burkina Faso is a low-income country, with 44% of the population living in poverty.8 In its Sahel region in 2008, user fees were eliminated for pregnant women and for children less than five years of age, in two out of the region’s four health districts.9 This was achieved by a complex strategy building on existing programmes to cover health-financing gaps.

To evaluate the estimated effect of eliminating user fees on the target populations we asked two questions: For key maternal and child health interventions, did the user-fee elimination strategy increase receipt of needed health services (coverage)? Consequently, did it reduce mortality in the target populations? We used analytical techniques for ex-post impact evaluation and mathematical modelling to test whether the user-fee elimination strategy increased health-care coverage and reduced mortality. We also explored the anticipated mortality impact of scaling up a similar intervention to the entire Sahel region and to the national level.

Methods

This study was approved by the research ethics committee of the Centre de Recherche du Centre Hospitalier de l’Université de Montréal (project 09.122), Burkina Faso’s ministry of health and local ethics committees.

To assess whether eliminating user fees would affect child and maternal mortality, we triangulated evidence using multiple data sources and analytical techniques to assess the effect of the user-fee elimination programme on coverage of specific health interventions. We then used the Lives Saved Tool (LiST) to link evidence on coverage improvements to anticipated mortality declines. The user-fee elimination programme provided a subsidy to render medical services free-of-charge at point-of-care for all pregnant women and for children less than five years of age in two health districts: Dori and Sebba.9 Since 2007, the government organized an 80% emergency obstetrical and neonatal care (Soins obstétricaux et néonataux d’urgence) subsidy associated with facility-based delivery. For pregnant women, the user-fee elimination programme hence reimbursed the complementary 20%. For children, the government provides basic preventive interventions free-of-charge, but requires payment for curative services and drugs. The user-fee elimination programme reimbursed designated primary health centres to render all consultations and prescriptions free at point-of-care for children less than five years of age.9

The Humanitarian Aid Office of the European Commission provided funding. Regional health authorities and the German nongovernmental organization Hilfe zur Selbsthilfe were the implementation partners.

Mathematical modelling

We modelled the effect of eliminating user fees using the freely available mathematical model LiST, version 4.53 (Johns Hopkins Bloomberg School of Public Health, Baltimore, United States of America; available from: http://www.jhsph.edu/departments/international-health/centers-and-institutes/institute-for-international-programs/list/index.html). The model structure has been described elsewhere.10

LiST was chosen because its target populations are identical to those of the study, and the study’s health interventions (i.e. given birth at health facility and children treated for diarrhoea, malaria or pneumonia) are represented in LiST. In addition, the model has been shown to provide accurate predictions of neonatal and child mortality associated with intervention scale-up in diverse geographical settings,1113 including western Africa.12

Model parameters

We used LiST default demography, proportional mortality and intervention effectiveness estimates. All other variables were classified into one of two categories.

Category 1 consisted of variables unaffected by user-fee elimination. We updated LiST parameters using data from household surveys,8,1419 United Nations agencies’ estimates,20,21 national administrative data22,23 and research reports.24 To ensure accurate estimation of mortality, category 1 indicators reflect changes in health-care coverage over the study period. (Appendix A, available from: http://www.equitesante.org/helpburkina/articles-scientifiques/user-fee-elimination-reduces-maternal-child-mortality). Variables potentially affected by user-fee elimination, but for which data on coverage impact were lacking, were also grouped in this category.

Category 2 consisted of four health care variables potentially affected by user-fee elimination: one related to pregnant women (giving birth at a health facility) and three related to children (treatment for diarrhoea, pneumonia or malaria, with oral rehydration salts, antibiotics or artemesinin, respectively). For these four variables, we developed statistical models – adjusted for potential confounding variables – to estimate the impact of user-fee elimination on health-care coverage. Coverage data for 2008 were taken from external sources (Table 1).15,25,26

Statistical modelling

For each of the variables in category 2, the intervention effect from 2008 to 2009 reflects the difference in the proportion of individuals seeking medical care due to elimination of user fees. For the three variables related to children´s health, this was due directly to user-fee elimination. For pregnant women giving birth in a facility, the difference is due to the combined effect of the user-fee elimination programme and the governmental emergency obstetrical and neonatal care subsidy.

For pregnant women giving birth in a facility, we ran interrupted time-series analyses covering a 96-month period (January 2004 to December 2011), including three years before the introduction of the emergency obstetrical and neonatal care subsidy. Population-level information was extracted from administrative records contained in Burkina Faso’s national health information database for all 71 primary health centres in all four Sahel health districts, two in which user fees were eliminated (Dori and Sebba) and two which maintained user fees (Gorom-Gorom and Djibo) but had similar health and living standards, geography, culture and climate.

We developed multilevel Poisson models (district, primary health centre, monthly deliveries) to study the effect of the emergency obstetrical and neonatal care subsidy and user-fee elimination programme both individually and in combination on the likelihood of delivery in a primary health centre. We used sitewise random intercepts and slopes and estimated intervention effects as rate ratios.28 In total, 6302 deliveries were recorded. Rate ratios (RR) and 95% confidence intervals (CI) were estimated by multilevel Poisson regression, adjusting for district, primary health care centres, secular trend, seasonal variation, statistical overdispersion and (via an offset) population growth.

To evaluate the effect of eliminating user fees for childhood consultations for diarrhoea, pneumonia and malaria, we used data from a household survey in the two health districts where the user fees had been eliminated.9,29 The sampling approach was based on probability proportional to size selection of census tracts and random selection of households.30 The same households were surveyed in July and August 2008 before the intervention, and again one year after implementation (July and August 2009). Survey data included households’ social and economic characteristics, health symptoms and needs within the last 30 days for children less than five years of age and health-care utilization. We developed multilevel (district, primary health centre, household) logistic regression models to estimate the effect of eliminating user fees on consultations for children reporting symptoms of diarrhoea, cough and fever.

The final regression model predicting childhood consultations included a propensity score term to adjust for overt selection biases while reducing the covariates to a one-dimensional score.31,32 The dependent variable for the propensity score model was the probability of an observation belonging to the intervention year 2009. The final model also contained interaction terms distinguishing the clinical subgroups – diarrhoea, cough and fever. The 12-month intervention effects for Dori were used in the LiST analysis to represent the intervention effect on diarrhoea, pneumonia and malaria coverage at one year. Odds ratios (ORs) and their CIs were applied to different scenarios of baseline risks before conversion to approximate relative risks for use in LiST modelling, because the assumption of homogeneity over varying baseline risks holds much better for ORs than for risk ratios.33 Supplementary analyses with the variables any childhood consultation or severe diarrhoea were used to validate the main analysis. Severe diarrhoea was defined as an episode reported as life-threatening or incapacitating.

We used Stata 12 (StataCorp LP, College Station, USA) and SAS 9.3 (SAS Institute, Cary, USA) for the analyses.

LiST analyses

For each variable, to calculate health care coverage for 2009, we multiplied 2008 coverage values in Table 1 by estimated intervention effects from the main analyses in Table 2. We then used LiST to model the mortality impact of the difference in coverage between 2008 and 2009 due to the user-fee elimination. We projected results for the two Sahel health districts constituting the user-fee elimination target population and for the entire Sahel. We also projected the impact of scale-up to the national level in Burkina Faso. National projections apply estimated intervention effects from the main analyses in Table 2 to models parameterized to reflect national demography, epidemiology and intervention coverage. Proportional mortality and intervention effectiveness remained unchanged from analyses for the Sahel.

Sensitivity analyses were used to explore the consequences of a range of reasonable alternatives. As LiST does not permit probabilistic sensitivity analyses, we focused on univariate and scenario-based approaches. Scenarios considered uncertainty in values for key model parameters as well as statistical uncertainty when estimating intervention effects (Appendix B; available from: http://www.equitesante.org/helpburkina/articles-scientifiques/user-fee-elimination-reduces-maternal-child-mortality).

For the Sahel, we had no reliable data on existing levels of antibiotic coverage for pneumonia in 2008. We explored three scenarios representing average, low and high estimates of antibiotic coverage for children. Within each scenario, we estimated lives saved based on the midpoint, and the upper and lower 95% CIs for user-fee elimination intervention effects. The Sahel mortality scenario is based on Burkina Faso’s 2010 demographic and health survey and multiple indicator cluster survey, which estimated 235 deaths per 1000 live births for under-five mortality.15 Estimated Sahel maternal mortality was 840 per 100 000 births.34

National-level analyses considered high- and low-mortality scenarios due to uncertainty concerning the background of mortality rates. High-mortality scenario for children less than five years of age was based on United Nations Inter-agency Group for Child Mortality Estimation, which was 178 deaths per 1000 live births for Burkina Faso in 2010,35 similar to the 2008 estimates of the Institute for Health Metrics and Evaluation (164.7 deaths; 95% CI: 140.4–190.9).36 For maternal mortality, we based the high mortality scenario on the values given in LiST, 700 deaths per 100 000 births.25

The low-mortality scenario is based on Burkina Faso’s 2010 demographic and health survey and multiple indicator cluster survey, which recorded 129 deaths per 1000 live births for under-five mortality and 307 deaths per 100 000 birth for maternal mortality.15 For both scenarios, we calculated lives-saved estimates based on the midpoint, and the upper and lower 95% CIs for user-fee elimination intervention effects. Appendix B provides further details.

Results

The interrupted time series analysis showed that the likelihood that pregnant women would give birth at a health facility significantly increased at all time-points analysed after user fees had been eliminated. A year after elimination of user charges, the relative risk of delivery in a health facility was 1.80 (95% CI: 1.46–2.20) in Dori and 1.96 (95% CI: 1.50–2.00) in Sebba when compared with the pre-intervention period (Table 3).

Table 4 summarizes the household survey sample used to study the changes in medical consultations for children less than five years of age. Between 2008 and 2009, there was a 1.5-fold and 2.5-fold increase in all-cause consultations in Dori and Sebba, respectively.

When analysing each clinical subgroup, results of the propensity score analysis on childhood consultations show that medical consultations for diarrhoea and probable malaria increased in each of the two user-fee elimination districts, whereas consultations for pneumonia only increased significantly in Sebba (Table 5).

Table 2 provides intervention effect estimates used for the mortality projections. The estimates showed that the likelihood of using health services by those in need increased in the user-fee elimination groups. In Dori, the midpoint relative risk estimates on coverage ranged from 1.6 to 2.2 across all six indicators (Table 2). For all indicators, results for the two intervention districts were similar but the effect was slightly greater for Sebba (Table 3 and Table 5). As the results from Dori provided a more conservative estimate of the intervention effect, those values were used in LiST modelling (Table 2).

We estimated that eliminating user fees saved an average of 593 lives of children less than five years of age (estimate range: 168–1060 lives) in the study population during the first year of the user-fee elimination programme. Eliminating user fees throughout the Sahel region would save an average of 1350 lives of children less than five years of age (estimate range 383–2414 lives) and reduce child mortality by an estimated 11% (estimate range 3–20%); from 235 per 1000 live births in 200815 to 210 per 1000 live births in 2009 (estimate range 189–228) (Table 6). Delivery in a health facility had a modest estimated impact on maternal mortality. Model projections show that 40 (95% CI: 29–51) women’s lives were saved in the study population, and that 91 (95% CI: 67–117) would be saved if a similar intervention were introduced throughout the Sahel region.

When simulating a nationwide user-fee elimination, the projected under-five mortality was reduced by 16% (95% CI: 4–26) in the high-mortality scenario and 17% (95% CI: 4–26) in the low-mortality scenario (Table 7).

The main sources of the modelled mortality in children were malaria, diarrhoea, pneumonia and neonatal causes. The main health interventions for projected under-five lives saved were anti-malarials, antibiotics for pneumonia, oral rehydration salts for diarrhoea, labour and delivery management, antenatal corticosteroids for preterm labour, and neonatal resuscitation. The latter three interventions, associated with delivery in a health facility, reduced neonatal mortality. Other interventions contained in the LiST model reduced mortality by 5% or less (Table 8).

Discussion

Previous studies have shown that when user charges are abolished, the use of health services tends to increase. However, it has been unclear whether this increase reflects appropriate use of health services and evidence for improved health is lacking.38 Our findings show that a complex intervention based on eliminating user fees for pregnant women and children increased intervention coverage. Multiple independent statistical analyses exploiting distinct analytical methods and data sources confirmed that service coverage was approximately one and a half to two times higher in the districts where user fees had been eliminated, as compared with the comparison districts. Model-based projections for a single year show that coverage increases are likely to have brought about substantial reductions in neonatal and child mortality, and a modest reduction in maternal mortality. As the intervention was carried out in a region with poor health indicators, the estimated reductions in mortality are therefore particularly important.

Strengths of the study include the following: the use of a validated model to project mortality impact; availability of high-quality data from a study designed specifically to measure the user-fee elimination effect; employment of appropriate study designs and statistical methods to quantify the intervention impact while controlling for potential confounding variables and selection biases; exploitation of multiple data sources and statistical techniques to enable cross-checking of results and convergence on reasonable values for intervention effects; and a conservative approach to estimating mortality impact.

The modelling approach was conservative in several ways. First, numerous health interventions affected by the user-fee elimination programme could not be modelled due to lack of data – for example, antenatal care visits, childhood vaccinations, breastfeeding promotion, antibiotics for dysentery, zinc for diarrhoea and vitamin A for measles. Second, the analyses used estimates from the health district with the lower intervention effect. Third, time-series analyses for facility deliveries show increasing intervention effects over time not captured in this one-year analysis. Fourth, other analyses show a stronger intervention effect in those at higher mortality risk.

There were several study limitations. First, mortality projections were based on mathematical modelling. We lessened the risk of error by using a validated model to estimate mortality with conservative assumptions and extensive sensitivity analyses. Second, for some key parameters, there is uncertainty regarding baseline levels. We developed scenarios reflecting a range of reasonable values from the strongest available data sources. Third, we were unable to consider statistical uncertainty associated with parameter estimates from the demographic and health survey data,15 as only average values were available. This limitation is common to all LiST analyses and likely to have a non-systematic impact on mortality projections. Fourth, we focused on overall effects. A separate analysis demonstrated that the intervention benefitted the poor.29 Fifth, increasing consultations does not guarantee appropriate treatment and eliminating user fees can have potential negative effects on service quality.38,39 Fifth, during the user-fee elimination it was shown that the quality of prescriptions was maintained.40 Insofar as the quality of care was similar before and after the intervention, effectiveness remains constant and therefore the conclusion that lives would be saved holds valid. However, inappropriate treatment practices would reduce the estimates of absolute numbers of lives saved. Sixth, while convergence of findings from multiple data sources and methods promises good internal validity, generalization of results to the national level requires evidence of external validity which is currently lacking. We therefore recommend that national projection estimates should be seen as exploratory.

Research on interventions to improve the delivery, practice and organization of health-care services poses challenges. Mortality averted by such interventions cannot be directly observed and an individual or cluster randomized trial of proven interventions with a mortality endpoint is not likely to be performed for reasons of ethics and feasibility. Under the circumstances, judicious synthesis of observational evidence using rigorous statistical techniques, multiple data sources and mathematical modelling is a suitable approach to answer an important policy question. Conclusions should be interpreted with appropriate caution.

Three published studies have examined the estimated impact of eliminating user fees on mortality. A mathematical modelling study explored the impact of a hypothetical user-fee elimination programme in 20 African countries and concluded that eliminating user fees for pregnant women and children could contribute to saving lives.5 A case study used LiST to illustrate how a range of government policies in the Niger, including provision of free health care for women and children, decreased child mortality.41 An econometric analysis of Thailand’s “30-baht” health-care reform, which sharply increased funding for hospitals while reducing co-payments, found that the programme increased access to health care among the poor, reduced infant mortality and equalized disparities in infant mortality.42 These three studies are in line with our conclusions that the abolition of user fees has lifesaving potential. However, the data used in the modelling study5 to estimate intervention effects are dated, and rely on studies from several countries and programmes with differing degrees of controls for confounding factors. The studies in the Niger41 and Thailand42 traced observed results of an ensemble of policies and did not specifically address the question of user-fee elimination.

A single randomized controlled trial in Ghana provides the only direct evidence of the impact of removing out-of-pocket payments on health outcomes in developing countries.43 It shows that in this study setting, eliminating user fees had an effect on care-seeking behaviour but not on the measured health outcomes. Possible explanations for this lack of effect include limited statistical power, failure to remove other barriers to service use, the possibility that the increase in using the intervention was too small to produce a clear effect on health, and residual confounding factors.43,44 Results nevertheless demonstrate the value of measuring health endpoints in addition to those relating to service use.

Most low- and middle-income countries are unlikely to achieve MDGs 4 and 5 by the 2015 target date.3,35,45 Alongside other strategies targeting non-financial barriers to access,46 our findings demonstrate that well designed policies to eliminate user fees can contribute to attainment of MDGs 4 and 5 and advance universal health coverage.

The most important question now concerns whether these results can be replicated at scale. Although embedded in the existing health system, this instance of user-fee elimination was implemented in a specific locale with the help of a nongovernmental organization. To date, most user-fee elimination policies led by national governments in western Africa have faced implementation difficulties. In the light of the existing evidence and study design challenges, as the next step, we recommend that user-fee elimination should be introduced on a wider scale, accompanied by rigorous evaluation. Effective scale-up will require careful policy design to ensure that solid financing mechanisms are in place,39 and reduce potential problems related to weak health systems, including deficiencies in human resources management, quality of care, supply chain logistics, and informal payments.19


Acknowledgements

We thank the health authorities of Burkina Faso the health centre personnel, community leaders and beneficiaries; the managers and personnel associated with the user-fee elimination intervention and the Hilfe zur Selbsthilfe eV, Issiaka Sombié and Moussa Bougma (AFRICSanté) and Fortuné Sossa.

Funding:

The study was funded by the Fonds de Recherche en Santé du Québec and by Hilfe zur Selbsthilfe eV, which received a grant from the Humanitarian Aid Office of the European Commission.VR is a New Investigator of the Canadian Institutes of Health Research.

Competing interests:

MJ, RH, and SH have declared no competing interests. VR has served as a consultant on the issue of user-fee abolition to nongovernmental organizations such as Médecins du Monde Hilfe zur Selbsthilfe eV, and Médecins Sans Frontières implementing user-fee abolition in Burkina Faso, Guinea, Mali and Niger.

References