Methodological considerations in implementing the WHO Global Survey for Monitoring Maternal and Perinatal Health
Archana Shah a, Anibal Faundes b, M'Imunya Machoki c, Vicente Bataglia d, Faouzi Amokrane e, Allan Donner f, Kidza Mugerwa g, Guillermo Carroli h, Bukola Fawole i, Ana Langer j, Jean José Wolomby k, Alberto Naravaez l, Idi Nafiou m, Marius Kublickas n, Eliette Valladares o, Alejandro Velasco p, Nelly Zavaleta q, Isilda Neves r, José Villar s
The WHO Global Survey on Maternal and Perinatal Health aims to develop a network of health institutions worldwide that collects up-to-date information on services provided and on how evidence-based recommendations are implemented in maternal and perinatal health care. Information is collected through a technologically simple online data entry and management system for large data sets. It is expected that use of this information will help to identify gaps at the facility and sub-national levels and to assist in effective planning, implementation and monitoring.
The survey was first implemented in the WHO regions of Africa and the Americas between September 2004 and March 2005 to study the relationship between intra-partum care and maternal and perinatal health outcomes. Preliminary results on increasing rates of caesarean section in Latin America were published.1 This paper describes methodological issues related to the establishment and implementation of the survey, and sets the foundation for reports to be published from this project.
The survey eventually will be implemented in 54 countries, four from each of the 14 WHO defined subregions. WHO subregions, classified by the levels of under-five child and adult mortality rates2 were used as a proxy for the burden of maternal and perinatal mortality. A stratified multistage cluster sampling design was used to obtain a sample of countries and health institutions worldwide.
Selection of countries, provinces and health facilities
From each subregion, four countries were selected with probability proportional to population size (Table 1). When there were less than four countries in a subregion, all countries within that subregion were included. This process resulted in 12 subregions having four countries each, and two subregions having three countries each (Table 1). It was decided that no replacement would be made for a country that did not participate.
In each country, the capital city was always included in the sample. In addition, two provinces were randomly selected from the other administrative areas. The third-stage sampling unit was obtained by drawing a random sample of up to seven health institutions, each of which reported at least 1000 deliveries in the year before the implementation of the survey. If there were fewer than seven eligible health institutions in the capital city or other provinces, then all available health institutions were selected.
In each country, an up-to-date census of health institutions in the selected areas was obtained. In the absence of a recent census, a list of health institutions was prepared by the country coordinators, in collaboration with WHO country offices and ministries of health.
All women who were delivered in the participating sites during the specified period comprised the study population. Those delivered elsewhere were not included. Data were collected over a two- or three-month period depending on the annual number of deliveries in each health institution. For those health facilities with less than 6000 deliveries, data were collected for three months; for those with over 6000 deliveries, data were collected for a two-month period.
As a one-time event, an institutional level data collection form (available at: http://www.who.int/making_pregnancy_safer/health_systems/global_survey/en/index.html) was completed by institution’s medical director. Data were collected on services influencing maternal and perinatal care and outcomes such as laboratory tests, anaesthesiology resources, intrapartum care including emergency obstetric care, and human resources for maternal and perinatal health.
Individual level data were abstracted directly from medical records onto a two-page data collection form (available at: http://www.who.int/making_pregnancy_safer/health_systems/global_survey/en/index.html) by trained data collectors. These included: maternal risk indicators, mode of delivery, and maternal and newborn outcomes up to hospital discharge or up to a maximum stay of seven days. These forms were completed after delivery and before hospital discharge of the woman and newborn. Incomplete data in medical records were updated in consultation with attending staff before patients’ discharge. Data were entered online (via Internet) at the health institutions and/or country level using existing computing facilities.
Definitional criteria for data collection items
Criteria for medical record data abstraction and definitions were described in the operational manual, available to all participating health institutions. A cross-checking mechanism was also incorporated to identify missing data. A separate manual was available for data transfer from individual forms to the online data entry system; this described data entry, cross-checking of data and mechanisms for handling missing data.
Data abstraction instruments were pre-tested on a convenience sample of records and at the hospital level in 4 countries. A pilot test was performed after a two-week training period to check the skills acquired by data collectors and to identify further problems with individual forms. Revisions were made based on these pre-tests.
Country coordinators were trained during two coordinators’ meetings at WHO Headquarters. Hospital coordinators and data collectors were trained by country coordinators and the WHO coordinating unit, with the support of regional staff.
One person, usually a labour ward midwife, was responsible for daily data collection in each health institution, while the hospital coordinator (midwife or obstetrician) was responsible for supervision and data quality monitoring before forwarding to the provincial or country coordinator. Data were entered online at hospital, provincial and/or national level depending on available resources. The numbers of completed forms were checked against the number of deliveries recorded in the logbook in the health institution. Completed data forms were sent to the provincial or country coordinator. When data entry was not possible in the health institution, it was done by the national coordinating unit. Random checks were performed periodically by the country coordinator using the online data entry system to check for completeness and accuracy of data. Online data were also checked for quality by the overall project coordinator. Problems identified were addressed immediately by the country coordinator; technical questions were resolved in consultation with the project coordinator.
Online data management and entry system
Survey data were managed in collaboration with the WHO coordinating unit by an online systems provider (MedSciNet AB, Stockholm, Sweden), which developed and provided the application and stores the data on its server. The system enables data collection and storage in a user-friendly format that allows for reporting and downloading data for analysis. It also allows for use of different languages and for data to be entered online using Microsoft Explorer and a dial-up connection. The system was pilot tested in Africa and Latin America and modified wherever required.
Online screens corresponded to the sections of the individual data collection form. The system prompted for the next field to be filled in; nonapplicable fields were automatically skipped. During data entry, fields were validated on screen according to pre-specified validation rules. A cross-checking validation was performed to ensure that only forms without errors were saved. Data were transmitted after encryption using 128-bit key security.
The system provided the facility to search, sort and update patient information, and to generate descriptive analysis reports; system description, manuals, and data entry tutorials; the facility to share information by uploading and downloading other documents; and, at project coordinating unit level, the facility to create and modify user information.
The application permitted different types of access to the site and data at global, national, sub-national and health institutional levels. Each data entry operator could access only the data that they had entered. Administrators had access to information at their level and below, but not to information at higher level. The project coordinator had administrative rights to access all data.
Preparatory work commenced in mid-2003. This included discussions with WHO regional offices, the selection of countries and provinces, and the preparation of a sampling framework obtained from the participating countries. Following the first meeting, with investigators from Africa and the Americas, to explore the feasibility of the study, all health institutions randomly selected were informed about the nature of the project. Institutional consent was obtained from the responsible authorities. Plans for data collection were tested, from September to November 2003, in both regions in selected health facilities.
The second global preparatory meeting, in November 2003, concentrated on finalization of individual and institutional data forms, training plans for the health institution staff, as well as data monitoring and management. At the third global meeting, in June 2004, final decisions on the implementation of the project in both regions were made. The country coordinator was responsible for project supervision at the national level, while the overall project was coordinated by WHO Headquarters in Geneva, supported by the WHO regional offices and country coordinators in Africa and the Americas.
Each institution submitted the ethical clearance approval before commencing the project. Ethical clearance was provided by the institutional committees of the participating facilities, where available, or by the national review committees (available at: http://www.who.int/making_pregnancy_safer/health_systems/global_survey/en/index.html). In addition, ethical clearance was obtained from WHO’s Scientific and Ethical Review Group and Ethics Review Committee. Individual informed consent was not obtained as this was a cluster-level study, where data were extracted from medical records without any subject identification. However, key subject information (name, study number, birth date and delivery date) was recorded in the logbook at the institution level by the data collector to assist with follow-up if required.
The sampling scheme was expected to produce a total of 1134 health institutions having a minimum of 1000 deliveries per annum among the selected countries in both Africa and the Americas. For the African region, we calculated a sampling frame of 699 health institutions in 7 participating countries (Algeria, Angola, the Democratic Republic of the Congo, the Niger, Nigeria, Kenya and Uganda). Of the randomly selected 133 health institutions, 131 participated. Among the 410 health institutions in Latin American countries (Argentina, Brazil, Cuba, Ecuador, Mexico, Nicaragua, Paraguay and Peru), 122 were randomly selected and 119 of these participated. Failure of recruited health institutions to participate was mainly due to change of staffing and leadership, urgent repairs or unplanned closure of the health facility because of conflict. Ethiopia and Haiti, though selected, did not participate because the implementation process could not be initiated within the given time frame. The survey could not be implemented in Canada and the United States of America because of administrative problems.
Local teams were selected by regional and country coordinators in consultation with WHO regional and country offices. Training of field personnel was conducted between January and April 2004. Site visits were undertaken by the project coordinator along with the country coordinators.
Data collection started in most participating countries between 1 September and 1 October 2004, and ended between 15 December 2004 and 30 January 2005. However, data collection in Ecuador started on 1 October 2004 and ended on 28 March 2005. In the Democratic Republic of the Congo, it started on 15 March 2005 and ended on 15 August 2005.
The initial results from Latin America have been published.2 Analyses of African data are ongoing.
There were various challenges to implementation of this survey. Internet access was not consistently available in all settings due to lack of reliable electricity supply and slow connection speeds. In some countries, data were entered online from Internet cafés; in others, data entry took place at local WHO offices.
In some very large health facilities where deliveries took place in places other than the main labour ward (e.g. in the health facility corridors), ensuring completeness of data was problematic. This was identified and rectified through cross-checking with the health facility logbook.
Where courier services were unreliable, country coordinators deputed people to visit health institutions in other provinces once a week to collect completed data forms and return incomplete ones. Lastly, some of the above problems were worsened in countries affected by conflict.
The implementation of this project was an ambitious yet achievable effort. The project created a network of health institutions, professionals and researchers in the area of maternal and perinatal health and demonstrated capacity for data management in Africa and Latin America.
Definitions and systems for monitoring maternal and perinatal health tend to vary among and within countries and regions3–5 and to be used in an isolated and non-harmonized manner. Efforts to standardize definitions to make data interoperable are limited. The implementation of this large-scale project has demonstrated the successful use of standardized approaches for data collection within countries and regions. This system has been effectively used even in settings with poor and lack of reliable maternal and perinatal health information and has generated high-quality and comparable data. In addition, this system allows data analyses, thus providing access to real-time information on maternal and perinatal health, a feature that allows for quality improvement and planning.
Health institutions were randomly selected. Besides scientific merits, random selection had the advantages of avoiding political problems, conflicts of interest at different levels and other related issues, all of which could have resulted in major selection biases. Moreover, the use of cluster-level information helps to maximize data comparability at health institution and country level. However, inclusion of the capital city as one of the three geographical areas surveyed may bias results.
To provide sufficient data within limited time, the survey was conducted over a short period and focused on health facilities with at least 1000 deliveries per annum. The results therefore provide information on the health status of women and newborns who had access to these facilities. Seasonal variations are reported in maternal and perinatal health6–9 and this survey should be extended, if required, to capture seasonal variations. Issues related to generalizability of information should be considered, especially in countries with low institutional delivery rates.
Women and infants were not followed up after hospital discharge. For women and infants in intensive care units, follow-up was for a maximum period of seven days after delivery. If a woman or newborn remained in the hospital for more than seven days, data were recorded as “seven days or more” without specifying number of days.
We chose to minimize the data collection burden in this survey by measuring only short-term, in-hospital maternal and perinatal morbidity and mortality indicators. Therefore, the survey in the present form does not capture deaths that occur after hospital discharge. Also other relevant medium- and long-term maternal and perinatal outcomes with potentially serious consequences remained unmeasured. The reason for this strategy was pragmatic: limited availability of resources and complexity of organizing longer postpartum follow-up. However, most severe maternal and neonatal morbidity and mortality occur during the hospital stay, and since postpartum visits are infrequent in many countries, useful information on outcomes can be collected from health facility surveys.
This network can implement large, simple, short, yet comprehensive studies. However given the need for strong motivation of staff involved in the survey, data collection over a short period in several more participating institutions in each province may be an option for future surveys. It is too early to comment on the sustainability of the system; the purpose was to assess the feasibility of implementing an Internet-based maternal and perinatal data monitoring system. Existing staff involved in routine data collection in facilities were used for the survey, and it may be possible to sustain the information system with existing resources.
Allowing more flexibility in the system to include data of local importance may increase interest and further use of the system for planning purposes. If the system is used for routine monitoring, access to data should be restricted to authorized users within the institution to ensure confidentiality and better address ownership issues.
This study has successfully demonstrated how a multi-country, multi-centre system can be established and implemented for routine monitoring of maternal and perinatal health. A network of collaborating institutions and an online data collection and management system have been established. WHO is uniquely qualified to implement such a large-scale project, which can form the basis for creation of a network of centres for monitoring maternal and perinatal health services worldwide. Routine use of simple technology for monitoring progress will assist programmatic decision-making in maternal and perinatal heath at various levels. The next challenges are to maximize the use of collected information, disseminate results, encourage local investigators and health authorities to use the data, and for all institutions to continue to maintain this network to be able to respond to other priority questions. ■
- Villar J, Valladares E, Wojdyla D, Zavaleta N, Carroli G, Velazco A, et al., et al. Caesarean delivery rates and pregnancy outcomes: the 2005 WHO global survey on maternal and perinatal health in Latin America. Lancet 2006; 367: 1819-29.
- Murray CJL, Lopez AD, Mathers CD, Stein C. The Global Burden of Disease 2000 project: aims, methods and data sources. The Global Burden of Disease 2000 in Ageing Populations Research Paper No 01.1, 2001. Available from: www.hsph.harvard.edu/burdenofdisease/publications/papers/gbd2000.pdf
- Suellentrop K, Morrow B, Williams L, D’Angelo D. Monitoring progress toward achieving Maternal and Infant Healthy People 2010 objectives — 19 states, Pregnancy Risk Assessment Monitoring System (PRAMS), 2000-2003. MMWR Surveill Summ 2006; 55: 1-11.
- Pattinson RC. Why babies die — a perinatal care survey of South Africa, 2000-2002. S Afr Med J 2003; 93: 445-50.
- Health Canada. Canadian perinatal health report 2003. Ottawa: Minister of Public Works and Government Services Canada, 2003.
- Anya SE. Seasonal variation in the risk and causes of maternal death in the Gambia: malaria appears to be an important factor. Am J Trop Med Hyg 2004; 70: 510-3.
- Makhseed M, Musini VM, Ahmed MA, Monem RA. Influence of seasonal variation on pregnancy-induced hypertension and/or preeclampsia. Aust N Z J Obstet Gynaecol 1999; 39: 196-9.
- Phillips JK, Bernstein IM, Mongeon JA, Badger GS. Seasonal variation in preeclampsia based on timing of conception. Obstet Gynecol 2004; 104: 1015-20.
- Romagosa C, Ordi J, Saute F, Quinto L, Machungo F, Ismail MR, et al., et al. Seasonal variations in maternal mortality in Maputo, Mozambique: the role of malaria. Trop Med Int Health 2007; 12: 62-7.
- Department of Making Pregnancy Safer, World Health Organization, 20 avenue Appia, 1211 Geneva 27, Switzerland.
- Centro de Pesquisas em Saúde Reprodutiva de Campinas (Cemicamp), Campinas, Brazil.
- Department of Obstetrics and Gynaecology, University of Nairobi, Kenya.
- Department of Obstetrics and Gynaecology, Hospital Nacional de Itaugua, Itaugua, Paraguay.
- Ministère de la Santé, de la Population et de la Recherche, Hospitalière El-Madania, Alger, Algeria.
- Department of Epidemiology and Biostatistics, Faculty of Medicine and Dentistry, University of Western Ontario, London, Canada.
- Centre for Quality of Health Care, Institute of Public Health, Makerere University, Kampala, Uganda.
- Centro Rosarino de Estudios Perinatales (CREP), Rosario, Argentina.
- Department of Obstetrics and Gynaecology, College of Medicine, University of Ibadan, Nigeria.
- EngenderHealth, New York, NY, United States of America.
- Département de Gynécologie et Obstétrique, Cliniques Universitaires de Kinshasa, Kinshasa, Democratic Republic of the Congo.
- Fundacion Salud, Ambiente y Desarrollo, Quito, Ecuador.
- Faculte des Sciences de la Sante, Universite Abdou Moumouni Dioffo, Niamey, Niger.
- Karolinska Institutet, Stockholm, Sweden.
- Universidad Nacional Autonoma de Nicaragua, Leon, Nicaragua.
- Hospital Docente Ginecobstetrico America Arias, La Habana, Cuba.
- Instituto de Investigacion Nutricional, Lima, Peru.
- Delegação Provincial de Saúde de Luanda, Luanda, Angola.
- University of Oxford, Oxford, England.