Trends in the quality of health care for children aged less than 5 years in Afghanistan, 2004–2006
Anbrasi Edward a, Vikas Dwivedi b, Lais Mustafa c, Peter M Hansen d, David H Peters a & Gilbert Burnham a
a. Johns Hopkins University, 615 North Wolfe Street, Baltimore, MD, 21205-2179, United States of America.
b. EPOS Health Consultants, Bad Homburg, Germany.
c. Ministry of Public Health, Kabul, Afghanistan.
d. GAVI Alliance, Geneva, Switzerland.
Correspondence to Anbrasi Edward (e-mail: email@example.com).
(Submitted: 13 May 2008 – Revised version received: 29 January 2009 – Accepted: 05 March 2009 – Published online: 25 August 2009.)
Bulletin of the World Health Organization 2009;87:940-949. doi: 10.2471/BLT.08.054858
Afghanistan has some of the poorest health indicators in the world. According to available estimates, in 2002 Afghanistan had an infant mortality rate of 165 per 1000 live births, an under-five mortality rate of 257 per 1000 live births and a maternal mortality ratio of 1600 per 100 000 live births.1 Many different factors impede access to health services and their effective delivery; they include lack of health infrastructure and human resource capital (especially female health providers), low literacy and other cultural barriers, and poor road access and security conditions. Since 2002, the Ministry of Public Health and its donor partners have made considerable investments in Afghanistan’s health system infrastructure and human resources.2–5 The Ministry of Public Health prioritized the implementation of a basic package of health services (BPHS) to improve health service coverage and quality. This implementation has been facilitated in part through contracts with nongovernmental organizations (NGOs) and the Ministry of Public Health providers. Under an innovative “contracting-in” mechanism, state providers are sometimes contracted by the government under the same conditions and targets set for NGOs and are eligible for performance bonuses.6 Services are implemented at three health facility levels: basic health centres, comprehensive health centres and district hospitals. According to a report, in 2006 approximately 82% of the Afghan population had access to basic health services.6
Since 2004, the BPHS has been assessed through Afghanistan’s National Health Service Performance Assessment. This assessment provides an overall measure of system performance and patient care by employing a balanced scorecard (used to manage performance in large and complex organizations) to measure six performance domains: patient and community perspectives, staff perspectives, capacity for service provision, service provision, financial systems and overall vision.7,8 The overall score for health system performance improved between 2004 and 2006.8–10 However, these gains need to be sustained, and additional efforts are needed to address those performance indicators that remain suboptimal.
More than 100 countries now use the Integrated Management of Childhood Illness (IMCI) strategy to address the major disease burden in children aged less than 5 years. The IMCI strategy is integrated into Afghanistan’s BPHS and is included in the country’s health and nutrition sector strategy.1,11 Despite controversies and debates on global investments in IMCI, the strategy remains a promising cost-effective way to improve quality of child health care and reduce child mortality.12–18 Afghanistan initiated IMCI training in 2003. The training is organized by NGOs that provide the BPHS, with IMCI facilitators from the Ministry of Public Health and donor organizations. More than 2300 health providers (mostly physicians) and 25 facilitators have been trained by means of the standard 11-day course.
This paper focuses on a subset of the data from the balanced scorecard to assess trends in the quality of health care for children less than 5 years in 2004–2006 and the factors associated with those changes.
This study, which was approved by the Institutional Review Board at Johns Hopkins University and the Ministry of Public Health Ethical Review Board in Afghanistan, presents data from the 29 provinces of Afghanistan that were included annually in the National Health Service Performance Assessment from 2004 to 2006. The methods used for designing the instruments and sampling health facilities are described elsewhere.8,9 Under the BPHS, each level of health-care facility has a specified level of human resource and infrastructure support. For each year of the study, up to 25 facilities were selected from each province through stratified random sampling of 3 district hospitals, 7 comprehensive health centres and 15 basic health centres.
Trained survey teams comprising clinicians, nurses or vaccinators were selected from each province to perform the observations and interviews. Observation of case management was based on a systematic sample of five children aged 2 months to 5 years. Children were selected using a random starting point and a sampling interval determined by the average number of new patients seen in a day. Characteristics of patients and providers were similar across the sample of cases and providers observed. Providers included clinicians, nurses and others responsible for clinical management of sick patients in the facility. To determine facility characteristics, four providers randomly selected from a list of providers were interviewed. To determine assessment and counselling quality, 10 indicators related to patient assessment were aggregated to create an assessment index and 5 indicators related to counselling were aggregated to create a counselling index. Each indicator was given equal weight and scored on a percentage scale.
In relation to IMCI, we considered the extent to which health-care providers adhered to case management procedures. To identify determinants of the quality of clinical and interpersonal care, we used bivariate and multivariate analysis of patient, provider and facility-related factors known to affect such quality.
More than 98% of the caretakers and providers sampled responded. The same facilities, providers and patients were not compared, since the sample was selected randomly each year.
We used standard procedures for quality control of data editing in the field, followed by double data entry and consistency checking. Initially, we used univariate analysis to examine the pattern of responses and extent of missing values to construct the indexes. The missing values (< 5%) were omitted from the analysis and confirmed to be missing at random. We used bivariate analysis to compare differences for indicators of assessment and counselling between 2004 and 2006 and to compare various groups of the selected independent variables. Multiple linear regression models were constructed using ordinary least squares to compare the quality index between the groups, with clustering at the facility level accounted for using Huber-White estimates of standard error. The model fit was examined using residual plots, and multicollinearity was assessed by estimating the variance inflation factor.19
Table 1 describes the study sample. Overall, about one-third of the children observed were less than 1 year of age, and diarrhoea was the most prevalent presenting symptom. About 80% of the cases were seen by doctors or assistant doctors in all 3 years of the study, and the rest by nurses or midwives. The proportion of female health-care providers interviewed was higher in 2006 than in 2004.
Table 1. Characteristics of children included in study of the quality of health care provided to children aged less than 5 years, by year, Afghanistan, 2004–2006
Quality of care for sick child consultations was determined by adherence to selected indicators on taking a history, physical examination and counselling. Most assessment and counselling indicators improved significantly between 2004 and 2006 (Table 2). For example, the proportion of the providers observed who checked for danger signs in the child’s ability to drink or breastfeed increased from about 40% in 2004 to about 60% in 2006; those who asked about diarrhoea, fever and cough or difficulty in breathing increased from less than 70% in 2004 to more than 80% in 2006; and those who explained the disease condition to the caretaker and provided instructions for home care of the sick child and administration of medications increased from about 55% in 2004 to about 68% in 2006.
Table 2. Provider performance with respect to patient assessment and counselling in study of the quality of health care provided to children aged less than 5 years, by year, Afghanistan, 2004–2006
In some areas, performance remained poor in 2006, despite significant improvements over the study period. These areas included checking the signs for anaemia (17% in 2006), oedema (19%) or the immunization card (29%); and giving instruction on signs and symptoms requiring an immediate return (< 45%). Factors that did not improve significantly over the study period were providers explaining adverse reactions to medications and measures to undertake when these reactions occurred.
The mean scores for both the assessment and counselling indexes improved significantly between 2004 and 2006 (Table 2).
The results of bivariate analysis of several factors (patient, provider and facility-related) that affect the quality of care are shown in Table 3 and Table 4.
Table 3. Results of bivariate analysis of quality of care for children aged less than 5 years based on provider and patient characteristics, by year, Afghanistan, 2004–2006
Table 4. Results of bivariate analysis of quality of care for children aged less than 5 years based on facility characteristics, by year, Afghanistan, 2004–2006
Provider cadre and sex: Performance of health-care providers across all cadres improved between 2004 and 2006. Doctors performed significantly better than assistant doctors and nurses in both assessment and counselling in 2005 and 2006. Sex differences were evident, with female providers showing significantly higher adherence to assessment standards than male providers in 2005 and 2006, and to counselling standards in 2006.
Child sex and age: There were no significant differences for assessment and counselling quality in relation to the sex of the child, but adherence to assessment standards was higher when providers were attending children less than 2 years of age.
Caretaker sex: Children accompanied by female caretakers received significantly better care at facilities in 2005 and 2006 than those who had male caretakers.
Consultation time: In all the years of the study, providers who spent 10 or more minutes in a consultation provided better care, as shown by significantly better assessment and counselling quality.
Type of health facility: Assessment quality was significantly better in comprehensive health centres in 2005 and in district hospitals in 2006. The quality of counselling was significantly better in comprehensive health centres than in district hospitals in 2004 and 2005, but the difference was not significant in 2006. Mean scores indicate that over the years of the study care improved in all types of facilities, without any observable trend between types of facilities.
Provider knowledge, satisfaction, refresher training and IMCI training: The proportion of providers who received refresher training was more than 13% greater in 2006 than in 2004. The increase in the proportion trained in IMCI was modest; more providers reported IMCI training in basic health centres and comprehensive health centres than in district hospitals. Knowledge scores were significantly higher in 2006 than in 2004. Provider knowledge, job satisfaction, refresher training and IMCI training were associated with better performance in assessment and counselling.
Supervision, availability of clinical guidelines and shura-e-sehie (village health committees): Facilities that received six or more supervisory visits in the previous 6 months and those with case management guidelines provided significantly better quality of care in 2006. The presence of active shura-e-sehie also significantly improved the quality of care in 2006.
Type of contracting: Although there was an overall improvement in performance in all contracting mechanisms between 2004 and 2006, providers in facilities with a contracting-in mechanism had higher mean scores for assessment and counselling across the 3 years of assessment than those in facilities with other contracting mechanisms.
Table 5 shows the results of multiple linear regression analysis using selected patient, provider and facility characteristics. The quality of care was higher in 2005 and 2006 than in 2004, upon holding other variables constant. Other factors that were significantly associated with quality of care were high provider knowledge, presence of IMCI-trained providers, support of facilities by the contracting-in mechanism, the provider being a doctor, a consultation time of at least 10 minutes, having a female caretaker and the child being aged less than 24 months. Other factors that were significant in the bivariate analysis were not significant predictors of quality in the multivariate analysis. The model explained 18% of the variance in assessment quality and 10% of the variance in counselling quality.
Table 5. Multiple linear regression coefficients for assessment and counselling quality in a study of the quality of health care provided to children aged less than 5 years, by year, Afghanistan, 2004–2006
Efforts to restructure the health system in Afghanistan have improved overall health-care provider training and satisfaction, the availability of drugs and equipment, service provision, the availability of clinical guidelines, infrastructure and the use of the health management information system. This is illustrated by the balanced scorecard results for 2004 to 2006.9
This study demonstrates that specific improvements are also being made in the quality of the clinical care provided to children aged less than 5 years, specifically in relation to IMCI. The results indicate significant improvements between 2004 and 2006, particularly in the assessment of danger signs and presenting complaints. Nonetheless, the mean assessment and counselling index in 2006 yielded an average score of only about 50 points out of a possible 100. These findings strengthen the case for continued investments to support provider adherence to standards of care. Failure to check immunization status and provide appropriate counselling about the administration of medications and the conditions that signify a need to return to the health facility will have a detrimental effect on overall patient outcome and continuity of care. Given that Afghanistan has exceptionally high rates of infant and child mortality, providers must be motivated to provide better counselling and health education to those caring for young children.
One question is “What type of health worker provides better care?” In a study from Morocco, adherence to IMCI tasks was significantly higher for nurses (compared to doctors), female providers, providers who did not identify supervision as a constraint, younger children, children accompanied by their mothers, and those presenting with multiple complaints.20 In other studies, nurses and female providers have also been found to deliver better care than doctors and male providers, respectively.21,22 In our study, female health-care providers gave better care to younger children and showed better performance than their male counterparts in 2006, but doctors performed better than other providers.
Supervision and training are clearly important factors; for example, providing six or more supervisory visits and IMCI training of assistant doctors and nurses were associated with better quality of care in 2004.23 Similarly, in a study from Uganda, the quality index was 44% higher in facilities with at least one provider trained in IMCI,24 a finding that further strengthens the case for greater investments in IMCI. We found that the proportion of providers receiving refresher training increased by 16% between 2004 and 2006, but the proportion of IMCI-trained providers increased by only 6% over that period. Our results also support findings from other studies in which provider performance and adherence to standards improved with more frequent and better supervision and with the presence of case management guidelines.25
This study used quality-of-care indicators based on clinical standards and structural indicators, but did not use indicators that focus on patient or community perceptions of quality. Effective health service delivery in Afghanistan faces formidable challenges due to scarce resources, cultural barriers and political and security risks. The challenge for improving and sustaining the gains achieved by the Afghan Ministry of Public Health and international donor community will require innovative mechanisms and the support of local communities. In view of the low numbers and potential turnover of providers and the costs of IMCI training, policy-makers need to consider other creative mechanisms to ensure equitable coverage of basic health services through the training and deployment of community health-care providers.
In high-mortality settings, IMCI training has to be accompanied by efforts to strengthen district health systems and community IMCI.14 According to recent estimates, more than 19 000 community health workers have been trained and deployed in health posts in Afghanistan, and a recent evaluation endorsed their value and demand by the community.26 Community providers, particularly women, could compliment efforts at the facility level to achieve equitable coverage of health services. Other strategies for improving health-care quality can be participatory assessments by users of the health system, team-based problem-solving, and creative engagement of health providers through national quality improvement initiatives.25,27–29
As in all assessments that observe provider performance, the presence of observers may have led the providers in our study to alter their usual manner of working. This is illustrated by a study in Benin.30 Other limitations of this study are the lack of re-examination of the sick child by an independent health‑care provider to establish a comparison of diagnosis and treatment, lack of risk adjustment for illness severity and the inability to link observed providers to those interviewed. Provider client load has also been associated with quality of care.22 However, we found that the observed quality continued to improve even though patient consultations more than doubled between 2004 and 2006. The finding that the type of contracting mechanism can affect the quality of the care given by providers raises opportunities for benchmarking best practices in health service delivery.6
Further empirical evidence is needed to assess how improvements in clinical care affect health outcomes or other aspects of the health system. The inclusion of data on community-based efforts for prevention, improvement of health-care seeking behaviours and reduced disease burden would provide more realistic measures of health system performance. The Afghan Ministry of Public Health has demonstrated its commitment to improving quality by instituting a National Quality Assurance Committee and by periodically reviewing evidence through the high-level Consultative Group on Health and Nutrition. Improving the quality and coverage of the BPHS must be a high priority if Afghanistan is to effectively address health needs and achieve those Millennium Development Goals that call for reduced child mortality and improved maternal health.31
Afghanistan needs to increase support for effective implementation of IMCI at the facility and community level if it is to sustain and build on gains in health-care quality. The concerted plan to improve basic health services across Afghanistan is clearly improving the quality of essential services for child health. However, to reduce child mortality, equitable coverage and access to services are needed. This will require continued investments in health infrastructure; a focus on improving health-worker performance through training, material support and supervision; greater accountability; and increased community capabilities, particularly in under-served rural areas. Notwithstanding the difficulties faced by children and health providers in Afghanistan, health care for children is moving in the right direction and deserves greater support. ■
The article represents the investments of Afghan and international staff who supported the design, implementation and analysis of the National Health Service Performance Assessment. We are also grateful to the supervisors, health-care providers, mothers and children who participated in the assessments. The authors would like to thank Dr Jennifer Bryce and Dr Cesar Victora for their insights on the results. We appreciate the comments and recommendations provided by the editorial committee and the anonymous reviewers.
Funding: Funding was provided through a Third Party Evaluation Contract between Afghanistan’s Ministry of Public Health and the Johns Hopkins Bloomberg School of Public Health, in collaboration with the Indian Institute of Health Management Research.
Competing interests: None declared.
- Afghanistan, Ministry of Health. A basic package of health services for Afghanistan. Kabul: Islamic Traditional Government of Afghanistan; 2003.
- Reilley B, Frank T, Prochnow T, Puertas G, Van Der Meer J. Provision of health care in rural Afghanistan: needs and challenges. Am J Public Health 2004; 94: 1686-8 doi: 10.2105/AJPH.94.10.1686 pmid: 15451732.
- Bartlett LA, Mawji S, Whitehead S, Crouse C, Dalil S, Ionete D, et al., et al. Where giving birth is a forecast of death: maternal mortality in four districts of Afghanistan, 1999-2002. Lancet 2005; 365: 864-70 doi: 10.1016/S0140-6736(05)71044-8 pmid: 15752530.
- United Nations Children’s Fund. Best estimates of social indicators for children in Afghanistan 1990-2005. New York, NY: UNICEF; 2006.
- Waldman R, Strong L, Wali A. Afghanistan’s health system since 2001: condition improved, prognosis cautiously optimistic (Unit Briefing Paper Series). Kabul: Afghanistan Research and Evaluation; 2006.
- Sabri B, Siddiqi S, Ahmed AM, Kakar FK, Perrot J. Towards sustainable delivery of health services in Afghanistan: options for the future. Bull World Health Organ 2007; 85: 712-8 doi: 10.2471/BLT.06.036939 pmid: 18026628.
- Kaplan RS, Norton DS. The balanced scorecard. Boston, MA: Harvard Business School Press; 1996.
- Peters DH, Noor AA, Singh LP, Kakar FK, Hansen P, Burnham G. A balanced scorecard for health services in Afghanistan. Bull World Health Organ 2007; 85: 146-51 doi: 10.2471/BLT.06.033746 pmid: 17308736.
- Hansen PM, Peters DH, Niayesh H, Singh LP, Dwivedi V, Burnham G. Measuring and managing progress in the establishment of basic health services: the Afghanistan health sector balanced scorecard. Int J Health Plann Manage 2008 23-.
- The Johns Hopkins University and Indian Institute of Health Management. Afghanistan health sector balanced scorecard national and provincial results round 3 (2006). Kabul: Ministry of Public Health and Planning Division; 2006.
- The world heath report 2005: making every mother and child count. Geneva: World Health Organization; 2005. Available from: http://www.who.int/whr/2005/en/index.html [accessed on 24 October 2008].
- Armstrong Schellenberg J, Bryce J, de Savigny D, Lambrechts T, Mbuya C, Mgalula L. The effect of Integrated Management of Childhood Illness on observed quality of care of under-fives in rural Tanzania. Health Policy Plan 2004; 19: 1-10 doi: 10.1093/heapol/czh001 pmid: 14679280.
- Chopra M, Patel S, Cloete K, Sanders D, Peterson S. Effect of an IMCI intervention on quality of care across four districts in Cape Town, South Africa. Arch Dis Child 2005; 90: 397-401 doi: 10.1136/adc.2004.059147 pmid: 15781932.
- Adam T, Amorim DG, Edwards SJ, Amaral J, Evans DB. Capacity constraints to the adoption of new interventions: consultation time and the Integrated Management of Childhood Illness in Brazil. Health Policy Plan 2005; 20: i49-57 doi: 10.1093/heapol/czi057 pmid: 16306069.
- Amaral J, Leite AJ, Cunha AJ, Victora CG. Impact of IMCI health worker training on routinely collected child health indicators in Northeast Brazil. Health Policy Plan 2005; 20: i42-8 doi: 10.1093/heapol/czi058 pmid: 16306068.
- Arifeen SE, Bryce J, Gouws E, Baqui AH, Black RE, Hoque DM. Quality of care for under-fives in first-level health facilities in one district of Bangladesh. Bull World Health Organ 2005; 83: 260-7 pmid: 15868016.
- Gouws E, Bryce J, Habicht JP, Amaral J, Pariyo G, Schellenberg JA, et al., et al. Improving antimicrobial use among health workers in first-level facilities: results from the multi-country evaluation of the Integrated Management of Childhood Illness strategy. Bull World Health Organ 2004; 82: 509-15 pmid: 15508195.
- Schellenberg JR, Adam T, Mshinda H, Masanja H, Kabadi G, Mukasa O. Effectiveness and cost of facility-based Integrated Management of Childhood Illness (IMCI) in Tanzania. Lancet 2004; 364: 1583-94 doi: 10.1016/S0140-6736(04)17311-X pmid: 15519628.
- Chatterjee S, Hadi AS, Price B. Regression analysis by example. 3rd ed. New York, NY: John Wiley and Sons; 2000.
- Naimoli JF, Rowe AK, Lyaghfouri A, Larbi R, Lamrani LA. Effect of the Integrated Management of Childhood Illness strategy on health care quality in Morocco. Int J Qual Health Care 2006; 18: 134-44 doi: 10.1093/intqhc/mzi097 pmid: 16423842.
- Amaral J, Gouws E, Bryce J, Leite AJ, Cunha AL, Victora CG. Effect of Integrated Management of Childhood Illness (IMCI) on health worker performance in Northeast-Brazil. Cad Saude Publica 2004; 20: S209-19 doi: 10.1590/S0102-311X2004000800016 pmid: 15608935.
- Adam T, Amorim DG, Edwards SJ, Amaral J, Evans DB. Capacity constraints to the adoption of new interventions: consultation time and the Integrated Management of Childhood Illness in Brazil. Health Policy Plan 2005; 20: i49-57 doi: 10.1093/heapol/czi057 pmid: 16306069.
- Hansen PM, Peters DP, Edward A, Gupta S, Arur A, Niayesh H, et al., et al. Determinants of primary care service quality in Afghanistan. Int J Qual Health Care 2008; 20: 375-83 doi: 10.1093/intqhc/mzn039 pmid: 18799469.
- Bishai D, Mirchandani G, Pariyo G, Burnham G, Black R. The cost of quality improvements due to integrated management of childhood illness (IMCI) in Uganda. Health Econ 2008; 17: 5-19 doi: 10.1002/hec.1231 pmid: 17387710.
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al., et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health technology assessment (Winchester, England) 2004; 8: 1-72.
- The Johns Hopkins University and Indian Institute of Health Management. Operations research on community health worker performance, 2006. Kabul: Ministry of Public Health and Planning Division; 2006.
- Bradley J, Igras S. Improving the quality of child health services: participatory action by providers. Int J Qual Health Care 2005; 17: 391-9 doi: 10.1093/intqhc/mzi057 pmid: 15951311.
- Marquez L, Madubuike C. Country experience in organizing for quality: Zambia. QA Brief 1999; 8: 16-9 pmid: 12322022.
- Ovretveit J. Formulating a health quality improvement strategy for a developing country. Int J Health Care Qual Assur Inc Leadersh Health Serv 2004; 17: 368-76 pmid: 15552392.
- Rowe AK, Lama M, Onikpo F, Deming MS. Health workers’ perception of how an observer influences their practices during consultations with ill children, Benin. Trop Doct 2002; 32: 166-7 pmid: 12139161.
- Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet 2004; 364: 900-6 doi: 10.1016/S0140-6736(04)16987-0 pmid: 15351199.