Bulletin of the World Health Organization

How valuable are environmental health interventions? Evaluation of water and sanitation programmes in India

Subhrendu K Pattanayak a, Christine Poulos b, Jui-Chen Yang b & Sumeet Patil c

a. Duke University, 126 Rubinstein Hall, 302 Towerview Drive, Durham, NC, 27708-0312, United States of America.
b. RTI International, Research Triangle Park, USA.
c. NEERMAN, Mumbai, India.

Correspondence to Subhrendu K Pattanayak (e-mail: subhrendu.pattanayak@duke.edu).

(Submitted: 22 March 2009 – Revised version received: 07 November 2009 – Accepted: 16 November 2009 – Published online: 26 January 2010.)

Bulletin of the World Health Organization 2010;88:535-542. doi: 10.2471/BLT.09.066050

Introduction

Over one billion people in the world lack access to clean water and more than twice that many lack access to basic sanitation.1 Inadequate water and sanitation services, the second most common cause underlying medical conditions that lead to child mortality, impose considerable illness and coping costs on households in developing countries. These costs fall disproportionately on the poor, women and children. One of the Millennium Development Goals (MDGs) is to halve the number of people in the world who live without access to adequate water and sanitation services.2,3 Reaching this goal would reduce coping costs, illness costs (including those generated by malnutrition), and therefore poverty as well. However, without clear empirical economic inputs into the planning and evaluation of interventions for improving water and sanitation services, the MDGs will be difficult to attain.

While epidemiological studies of the benefits of water and sanitation interventions have shown that diarrhoea can be reduced by 30–50%,4 these studies have had two limitations. First, they have not estimated the economic benefits of improving water and sanitation, despite the fact that such estimates are needed to properly allocate investments among different health interventions, different programmes that promote health or non-health water uses, and different sectors of the economy. Second, a recent meta-analysis and update of previous reviews found that half of them did not meet quality standards.4 For example, they did not: (i) account for baseline diarrhoea rates and pre-intervention hygiene behaviour, (ii) establish control groups, or (iii) consider whether non-programme events or conditions might have caused the observed effects. There are very few studies on the benefits of water and sanitation policies in India.

The limited literature on economic benefits is dominated by “stated preference” studies, such as contingent valuation and conjoint methods studies, which directly measure households’ “willingness to pay” for contingent (hypothetical) improvements in water and sanitation infrastructure (Table 1).2325 “Revealed preference” studies, on the other hand, measure economic benefits by examining actual preventive behaviours, such as treating the water.1319,26,27 Particular types of revealed preference studies15,16 known as avoided or coping cost studies – thus named because they examine the prevention costs incurred to cope with poor water and sanitation – measure the savings in prevention costs resulting from improvements in water and sanitation.

While stated preference benefit estimates are comprehensive, they are vulnerable to validity threats and may overestimate true economic benefits. Savings in coping costs, on the other hand, may underestimate the true economic benefits of a given intervention because they do not capture the economic value of a lowered risk of death or of reduced pain and suffering.28,29 We view avoided illness costs as coping costs because medical spending can lead to the prevention of severe diarrhoea and mortality.28,30 Although an extensive literature exists on the appropriate theoretical model for evaluating the impact of environmental interventions on economic welfare,28,29,31 no causal empirical studies have measured the economic benefits generated by water and sanitation policies.32

To address the knowledge gaps described above, we embedded a valuation study within a rigorous impact evaluation study and estimated the economic value of the average “treatment effect” of a community demand-driven water and sanitation programme. We employed a unique combination of propensity-score “pre-matching” and large panel data to estimate the economic impacts of a multi-dimensional environmental health programme. The results of the study are timely and relevant for new intervention planning and design efforts, including cost–benefit analyses of global and national investments in water and sanitation programmes aimed at improving child health (MDG 4) and improving sustainable access to safe drinking water and basic sanitation (MDG 7).

Methods

We estimated the economic impact of a community demand-driven programme launched by the government of the state of Maharashtra, India, with support from The World Bank to improve water, sanitation and hygiene in rural areas of the state. The main objectives of the programme were to increase access to drinking water and sanitation services in rural areas, institutionalize the decentralized delivery of such services by local governments, and improve rural living conditions. Village residents organized to improve their water supply and sanitation and hygiene conditions by choosing interventions that best matched their needs and capabilities, and they applied to the state government to participate in the programme. The latter lasted five years (2004–2008) and was implemented in approximately 2800 villages in 26 of the state’s 33 districts. Programme sustainability was promoted by requiring communities to pay 10% of capital costs and 100% of the operational and maintenance costs of the programme.

For our study of the programme’s benefits we sampled 242 villages from four geographically different districts. A published study protocol describes how restrictions, stratification and matching were combined to reduce sampling bias.32 Using propensity score matching, we matched each of 80 programme (treatment) villages in these districts to two similar non-programme (control) villages to ensure comparability between villages based on community indicators (e.g. demographics, housing, socioeconomic conditions, village infrastructure) and household variables (e.g. health outcomes, water and sanitation, personal hygiene). Table 2 illustrates the resulting similarity in key indicators. In control communities other government programmes, such as the Total Sanitation Campaign, were conducted as usual.32 Statistical power analysis indicated that a sample size of approximately 50 households with children 5 years of age or younger from each of the 242 villages would generate 80% power to detect an intervention impact of ≥ 30% in the population of children under 5 with a baseline diarrhoea prevalence of 22%. Household and community survey questionnaires were designed to collect impact indicator data on child health (as measured primarily by diarrhoea among children under 5 in the two weeks before the survey) and on child growth, personal benefits, and variables for computing cost savings (e.g. time costs), plus a range of individual variables (e.g. sex, age, social class, caste, religion) and household variables (e.g. family size and composition, education, housing conditions, assets, water and sanitation practices). Pre-intervention data were collected from a sample of approximately 9500 households before (May–June) and after (August–September) the 2005 monsoon. Post-intervention data were collected from the same households in the same months in 2007 to account for etiologic, structural and behavioural differences across seasons. To minimize confounding, we stratified our analysis by season and compared outcomes within seasons, but not across them. Thus, we can comment on seasonal differences but cannot fully explain them.

Each programme community was expected to improve in all intervention components (water supply, sanitation and hygiene). However, most village programme plans focused on strengthening water sources and/or on developing new ones, treating sources and distributing piped water, and constructing private toilets and drains. Unfortunately, hygiene education was not demanded by the community.

We followed the methods described in the literature to compute coping costs due to inadequate water and sanitation services15 and the private costs of water-related illnesses.34 For both sets of computations, time was valued at one-half of the village-level gender-specific average hourly wage.13 All costs are reported in terms of monthly household costs in purchasing power parity (PPP) adjusted to 2007 United States dollars (US$).

Monthly coping costs were calculated as the sum of expenses attributable to: (i) the time spent collecting water, (ii) the time spent on going to defecate, (iii) water treatment (i.e. filtering, boiling and use of chemicals), and (iv) water storage. To monetize the time spent collecting water, we multiplied the number of hours spent walking to and from water sources and waiting at the sources by the number of trips taken and the applicable wage estimate. For sanitation related costs, we assumed each household member made only one trip per day to the main sanitation site, aggregated the total sanitation-related travel time to the household level, and multiplied it by one-half of the hourly wage.

Turning to water treatment costs, for households that used filters, we converted the one-time cost of filters into monthly costs by amortizing the reported value over the lifespan of the filters at an estimated annual discount rate of 15%, typical for India. For households that boiled their water, we relied on previously reported estimates of the cost of boiling water27,35 (i.e. the estimated cost of boiling 5 litres was 27 Indian rupees (Rs), or US$ 0.68) and assumed 5 litres of boiled water per day provided enough drinking water in these households. Finally, to estimate storage costs, we first obtained the costs and the useful life of storage containers from households and shop keepers. For each type of storage container, we first amortized the storage costs over the respective lifespan at a discount rate of 15% and then multiplied the implied monthly costs by the number of each type of container the household owned.

The private cost of illness due to the most recent episode of diarrhoea was represented by the sum of out-of-pocket expenditures (i.e. costs borne by private households rather than the public health system) and lost income. We monetized the time costs incurred by patients and caregivers as above using community average wages. Following an efficiency wage logic, we valued children’s time in proportion to their age. For example, a 4-year-old’s time costs were pre-multiplied by 0.25, and those of a 16-year-old by 1.

The resulting measures of coping costs (or defensive expenditures) and cost of illness measures were used to estimate the economic impacts caused by the programme by applying a difference-in-difference strategy on this pre-matched sample. The difference-in-difference estimate is the mean of the difference between the difference in outcomes (over time) in programme villages and the difference in outcomes in control villages.33,36 We inflated the standard errors to account for correlated information from multiple households in the same village using robust variance estimation techniques. The pre–post data collection allowed us to use a difference-in-difference estimator to measure “treatment effect” by comparing coping cost and illness cost outcomes in the treatment and control villages during both periods.

Results

Baseline findings

At baseline only 21% of households were using private taps and only 12% were using individual household latrines. Although 50% of respondents reported that they filtered water, less than 5% reported using chemicals to purify it and only about 2% reported boiling water. On average, household members washed their hands during critical times – i.e. before preparing food or cooking, before eating, before feeding children, after changing babies or handling children's faeces, and after defecating – only half as often as required.

As shown in Fig. 1, the average monthly time costs of collecting water were: in the dry season, US$ 31.94 in 2005 and US$ 30.24 in 2007; in the rainy season, US$ 38.76 in 2005 and US$ 20.10 in 2007. The household-level mean monthly time costs related to sanitation were: in the dry season, US$ 9.31 in 2005 and US$ 8.05 in 2007; in the rainy season, US$ 9.32 in 2005 and US$ 7.73 in 2007.

Fig. 1. Average household coping costs, by season, year and intervention, in community demand-directed water, sanitation and hygiene programme, Maharasthra, India, 2005 and 2007
Fig. 1. Average household coping costs, by season, year and intervention, in community demand-directed water, sanitation and hygiene programme, Maharasthra, India, 2005 and 2007

The mean monthly cost of boiling water was: in the dry season, US$ 1.96 in 2005 and US$ 0.56 in 2007; in the rainy season, US$ 1.64 in 2005 and US$ 0.83 in 2007. On average, the monthly cost of filtration was about US$ 0.13. Finally, households that treated their drinking water with chemicals spent no more than US$ 0.07 per month.

For storage, household monthly expenses during the dry season were an average of US$ 10.37 in 2005 and US$ 8.36 in 2007, and during the rainy season, they averaged US$ 6.75 in both 2005 and 2007.

Cost of illness

One diarrhoea case had occurred in the two weeks before the interviews in at least 30% of the households in 2005 and about 20% in 2007. Out-of-pocket medical expenses included clinic/hospital fees, medicines, transportation, lodging and meals. In 2005, households spent an average of US$ 16.63 in the dry season and US$ 8.58 in the rainy season, whereas in 2007, they spent an average of US$ 12.52 in the dry season and US$ 5.50 in the rainy season (Fig. 2).

Fig. 2. Average household illness costs, by season, year and intervention, in community demand-directed water, sanitation and hygiene programme, Maharasthra, India, 2005 and 2007
Fig. 2. Average household illness costs, by season, year and intervention, in community demand-directed water, sanitation and hygiene programme, Maharasthra, India, 2005 and 2007

As shown in Fig. 2, lost implicit income per household due to diarrhoea episodes among adult patients amounted, on average, to US$ 4.24 in 2005 and US$ 3.18 in 2007 during the dry season, and to US$ 3.86 in 2005 and US$ 2.19 in 2007 during the rainy season. During the dry season, caregiving led to an average loss in implicit income of US$ 4.11 in 2005 and US$ 3.38 in 2007; in the rainy season, the loss amounted to US$ 4.39 in 2005 and US$ 2.25 in 2007.

Programme impact

Difference-in-difference estimates showed that the community demand-driven water, sanitation and hygiene programme had a moderate but significant impact on the reported adoption of taps and toilets: we found 13% more private tap use and 7% more private toilets in programme villages compared to control villages. However, we did not find a statistically significant impact on handwashing or home water treatment.

The changes in tap and toilet use reduced coping costs among households in treatment villages by an average of US$ 6.98 compared with households in control villages (Table 3). Most of these savings varied by season and were due to reductions in the time household members spent travelling to and waiting at water sources (a saving of about US$ 5) and going to defecate in the open (a saving of about US$ 2). Difference-in-difference analyses of cost of illness components and overall cost of illness showed no programme impact in either the dry or the rainy season, since both programme and control villages incurred lower illness-related costs in 2007 than in 2005. This is not surprising because the programme did not result in any differential change in diarrhoea or in handwashing or other key hygiene practices associated with diarrhoea.

Coping cost savings were larger for households below the poverty line, whereas scheduled castes and tribes saved more in both absolute and relative terms (Table 4). The programme targeted communities with a high proportion of poor and socially marginalized groups and attempted to involve them in programme planning, implementation and monitoring. Therefore, the better outcomes attained by these groups are in line with the programme’s objectives.

Although some sanitation-related time savings occurred in the rainy season, programme benefits were noted primarily in the dry season. While the study was not designed to explain seasonal differences, we found, for example, that in the rainy season household members used more water sources and walked and waited less. These behavioural adjustments could be induced by structural factors such as the availability of water for longer periods or higher wages in the rainy season. Also, a secular trend suggests that more households can access taps and toilets during the rainy season (which follows the dry season) at baseline, so that there is less potential for improvements in the follow-up period. Our study was not designed and powered to determine which factor dominated.

Discussion

This paper reports on the average monthly household-level cost savings attributable to a community demand-driven water, sanitation and hygiene programme in rural India. Our rigorous quasi-experimental impact evaluation relied on: (i) data on household cohorts collected before and after the intervention (2005 and 2007, respectively) from programme and control villages; (ii) propensity score estimation to pre-match programme villages with observationally equivalent control villages; (iii) sufficient sample size (four repeat measures of approximately 9500 households in 242 communities); and (iv) a panel-based difference-in-difference estimation strategy to control for pre-existing differences between programme and control villages. These costly data collection and analysis strategies will reduce bias, but may not have eliminated them. However, given the political and practical considerations of studying a real world programme, we have shown that this quasi-experimental design is the best way to maintain rigor, without interfering with programme implementation.32

This paper makes several contributions to what is known about the economic benefits of programmes that improve water and sanitation. First, we present methods that make it possible to attribute such benefits to environmental programmes and policies in the sector using a real world case study. Second, we found that the average household in programme communities could save an average of US$ 6.98 per month, or roughly 5% of monthly cash expenditures. Moreover, the benefits were higher in poorer and socially marginalized households, in line with the social targeting objectives of the programme. These coping cost reductions were largely due to improved access to better water and sanitation services, which shortened the time household members spent travelling to and waiting at the service source. Improved access leads to greater use of better services and to better public health outcomes as long as households also adopt complementary hygienic practices, such as handwashing, in response to effective hygiene promotion.

Diarrhoea incidence and the associated illness costs fell during the evaluation period in both intervention and control villages presumably because of overall socioeconomic development in rural Maharashtra and routine water and health programmes (e.g. activities in control villages). This general improvement in socioeconomic conditions, coupled with the lack of behavioural change in programme villages, may explain why, on average, we failed to find an impact on child diarrhoea and illness costs when we compared treatment villages with control villages.

Our intention-to-treat estimates of programme benefits are realistic because they are based on the actual uptake of improved services by households (e.g. use of taps) rather than on assumed or planned full coverage. Because the programme was driven by community demand, the communities and the households in them had to be willing and able to access and use the improved services. In practice this resulted in less than 100% community coverage. Consequently, in direct contrast to multivariate regressions of cross-sectional data on connections to piped sewage and water, our method avoids excessive attribution of economic benefits to the programme.

For comparison of our results with those of other studies (Table 1) on the economic benefits of obtaining a tap or a toilet per se, we used the difference-in-difference strategy to compare the cost change among adopters (those who used taps and toilets) to the cost change among non-adopters and found that adopting taps and toilets resulted in a cost reduction of US$ 15.59 in 2007 PPP terms. While these figures represent the economic benefit of adopting and using taps and toilets, they do not reflect the programme’s overall impact. Instead, the coarser intention-to-treat statistic (US$ 6.98) provides a conservative estimate of the programme’s benefits by averaging across adopters and non-adopters in programme communities.

In summary, renewed attention to the call for improving water and sanitation has spurred several global and regional assessments of the costs and benefits of delivering water and sanitation services.3739 These assessments rely heavily on the extrapolation of limited data or on assumptions because of a scanty and inconclusive literature on the causal economic benefits of water, sanitation and hygiene policies. Unless environmental health interventions apply the kind of rigorous causal impact evaluations described in this paper, designed to avoid excessive attribution, water and sanitation programmes will be pipe dreams.


Acknowledgements

For several helpful comments, we are grateful to Jack Colford, Maureen Cropper, Priti Kumar, Kseniya Lvovsky and seminar participants at the Institute of Economic Growth, RTI, University of California (Berkeley), The World Bank and the Indian Council for Medical Research.

Funding:

This study was funded by The World Bank.

Competing interests:

None declared.

References

Share