Bulletin of the World Health Organization

Health systems performance assessment in the Bulletin

Ties Boerma a, Mickey Chopra b & David Evans c

a. Department of Health Statistics and Informatics, World Health Organization, 20 avenue Appia, 1211 Geneva 27, Switzerland.
b. Medical Research Council of South Africa, Cape Town, South Africa.
c. Department of Health Systems Financing, World Health Organization, 20 avenue Appia, 1211 Geneva 27, Switzerland.

Correspondence to Ties Boerma (e-mail: boermat@who.int).

Bulletin of the World Health Organization 2009;87:2-2. doi: 10.2471/BLT.08.061945

In 2000, WHO published its first attempt to assess the performance of the world’s health systems in The world health report 2000.1 This report generated enormous interest but, in many ways, the scientific progress was overshadowed by the political debate related to the estimates of country-level performance and the associated league tables. Since then, the WHO European and Eastern Mediterranean Regional Offices have maintained health system observatories, with detailed descriptions of country systems. The considerable interest in measuring the performance of health systems worldwide is illustrated by the recent European Ministerial Conference on Health Systems, which culminated in the Tallinn Charter entitled Health systems for health and wealth.2 In developed countries, primary concerns include costs, quality of care, aging and chronic diseases. In developing countries, health system constraints have restricted progress towards the UN Millennium Development Goals.3

The methods and metrics to assess progress and performance of specific health programmes are often well established. This is less so when trying to assess the extent to which health systems achieve their goals. When strengthening health systems and establishing the most cost-effective ways of delivering care, there is a need for a basic set of indicators of health system functions and of scientifically sound, practical and user-friendly tools.

An increasing number of countries conduct self-assessments of their performance, and these are mostly performed by government-funded research institutes in partnership with academic institutions. Comparisons are made with other countries, efficiency is examined by comparing outputs and outcomes to inputs. Given the investment, what are the results?

In the United States of America, for example, 37 indicators were selected to assess performance in the domains of long, healthy and productive lives, quality, access, efficiency and equity.4,5 The indicators were benchmarked against best performers, typically those achieved by the top 10% of countries, states, health plans, hospitals and other providers. In England, considerable investments are being made in assessing performance of the National Health Service through monitoring a large set of indicators with targets.6 In Canada, the emphasis is on developing a monitoring system for primary health care performance that focuses on population-based data sources.7,8 The Netherlands publishes a bi-annual Dutch health care performance report, focusing on quality, access and costs using more than 100 indicators.9

Indonesia conducted a comprehensive within-country assessment using WHO’s health systems performance assessment framework, as part of its Healthy Indonesia 2010 policy.10 In South Africa, the Health Systems Trust has now published three editions of its district health barometer, which monitors about 20 indicators.11 In addition to comparisons across districts, metropolitan areas and parts of the country considered to be severely disadvantaged were included. In Afghanistan, the rapid expansion of health services was monitored using a balanced scorecard that focused on service delivery through a comprehensive facility survey.12 In Mexico, a report card for all states was used to assess the effects of the health system reforms during 2001–2006, including a summary measure based on 11 indicators derived from a variety of clinical- and population-based data sources.13

Methodological progress has been made in key components of health systems performance assessment, such as responsiveness, catastrophic expenditure, equity and effectiveness, but less progress has been made in terms of summary outcome measures. Ways to measure efficiency have received considerable attention and more sophisticated econometric methods were developed in response to the methods used in The world health report 2000.14 However, the debate is by no means resolved. Moreover, the question of deciding the appropriate outcome measure for use in efficiency analysis, or simply to summarize overall health system performance, is still under debate. A variety of summary measures have been proposed ranging from life expectancy and avoidable mortality to complex measures combining multiple dimensions of health system goals, including financial fairness, responsiveness, mortality and morbidity.

In addition to the challenges of developing indicators and analytical methods, the actual availability and quality of core data on basic health system building blocks, such as financing, human resources, medicines and service delivery, remains a challenge in low-income countries. WHO, The World Bank and partners have proposed a core set of indicators and measurement approaches and ways to address these information gaps (available at: http://www.who.int/healthinfo/statistics/toolkit_hss/en/index.html).

The field of health systems performance assessment is closely related to health systems research and evaluation. Increased demand for accountability and demonstration of results mean that evaluation research is also receiving much more attention. Evaluation of the different ways of scaling up health programmes, and of relaxing key health systems constraints through global health partnerships or other initiatives, is likely to yield information that is highly relevant to the assessment of the performance of health systems.

The Bulletin invites papers on health systems performance assessment. Policy papers may include country applications, results and implications of the assessment for policy, while research papers should address issues related to the measurement and analysis of health systems performance. ■