Bulletin of the World Health Organization

Monitoring effectiveness of programmes to prevent mother-to-child HIV transmission in lower-income countries

Elizabeth M Stringer a, Benjamin H Chi a, Namwinga Chintu a, Tracy L Creek b, Didier K Ekouevi c, David Coetzee d, Pius Tih e, Andrew Boulle d, Francois Dabis f, Nathan Shaffer b, Catherine M Wilfert g, Jeffrey SA Stringer a

Introduction

Despite the availability of proven interventions for the prevention of mother-to-child HIV transmission (PMTCT) and substantial donor investments for implementing them in developing countries, paediatric AIDS remains a largely uncontrolled epidemic.1 The majority of cases occur in sub-Saharan Africa, where high HIV prevalence among pregnant women combines with an under-resourced health-care infrastructure to produce nearly 90% of the world’s 800 000 children who are believed to be infected each year. Several ambitious goals for paediatric AIDS control have been set by various international bodies, including a 50% reduction in paediatric infections by 2010 (United Nations General Assembly Special Session on HIV/AIDS),2 reaching at least one million women by 20073 and provision of PMTCT services to 80% of those in need by 2010.4

While these goals are clearly appropriate in their scope, the disparity in which outcomes they actually target reveals a lack of clarity and consensus around how to monitor the effectiveness of PMTCT programmes. Without this clarity, it is difficult for policy-makers in developing countries to mount a coordinated response. In this paper, we argue for a validated consensus model for PMTCT effectiveness monitoring. This approach is urgently needed to coordinate the global response to paediatric AIDS prevention and to compare progress across programmes with a range of intervention strategies. We develop the case for using population HIV-free child survival as a gold standard metric and examine potential strategies for its measurement across Africa, including its addition into regular country-wide Demographic and Health Surveys (DHS). The standardized modification of the DHS described in this report could provide a reliable and easily replicable method to assess the impact of services at a population level.

Experiences in developed and developing countries

The first clinical trials of antiretroviral (ARV) drug prophylaxis for women and infants were conducted in Europe and the United States of America, where most HIV-infected women have access to good prenatal and delivery care, a range of laboratory tests and replacement feeding.5 Effectiveness – defined as the prophylactic benefit of a PMTCT intervention when implemented in real practice – closely approximates clinical trial efficacy in these settings because of strong supporting health-care infrastructure, low HIV seroprevalence and near-universal service coverage. Multiple studies have shown that high maternal HIV plasma viral load, low maternal CD4+ lymphocyte cell count, vaginal birth and breastfeeding are the most important risk factors for perinatal HIV transmission.6,7 Paediatric AIDS has been all but eradicated in Europe and the USA by ensuring high service coverage and by systematically targeting each risk factor.8,9

The situation in most developing countries differs dramatically.10 Although the risk factors for transmission remain the same, there are far fewer options for most women: few have access to completely suppressive ARV regimens, elective Caesarean or safe alternatives to breastfeeding, and even basic antenatal service access is far from universal. Because of this, most studies in developing countries have focused on simpler and more cost-effective regimens that can be deployed widely. The myriad of PMTCT trials that have been conducted in developing-world settings have been reviewed in detail elsewhere.11 What is important to note is that, with the results of the first short-course zidovudine (ZDV) trials conducted in Côte d’Ivoire12,13 and Thailand,14 as well as the HIV Network for Prevention Trials (HIVNET 012) single-dose nevirapine trial in Uganda,15 it has been shown that simple, short-course ARV regimens work, and that more suppressive regimens for longer periods of time work better. In addition, breastfeeding remains an important route of transmission; however, benefits of replacement feeding in Africa are becoming less clear due to competing co-morbidities.16,17

Thus, while there is abundant information about the efficacy of interventions to reduce perinatal transmission (these data come from intensely monitored clinical trials), our current understanding of their effectiveness (i.e. field performance) is lacking. Currently few high-prevalence countries have ongoing universal, country-level monitoring of PMTCT. We believe that closing this public health knowledge gap is critical to global success in the fight against paediatric AIDS, and can only be achieved through the development of international consensus around PMTCT effectiveness monitoring. Although various bodies have issued guidelines, there is very little agreement upon the appropriate outcomes to measure, the best source populations in which to measure those outcomes, or the appropriate methodology for measurement. Once a consensus methodology is agreed upon, it is important to provide tools that can monitor programmes and be used to implement specific interventions.

Previous evaluations of programme impact

A commonly used surrogate marker for programme effectiveness is programme coverage, i.e. the proportion of HIV infected/exposed mother/infant pairs in a population that receive a PMTCT intervention.18 This measure assumes that the benefits of a PMTCT intervention (established from clinical efficacy trials) will accrue to a population of mothers and infants who access the intervention appropriately. Coverage is defined as the product of a critical pathway of events that must be in place for the prophylaxis to be delivered. This critical pathway – which we and others have called the PMTCT cascade1922 – can be constructed from process indicators that are collected routinely with varying success by PMTCT programmes and health-care facilities (Fig. 1).

Fig. 1. Series of events required for a pregnant woman to successfully receive full ARV prophylaxis<sup>a</sup>
Fig. 1. Series of events required for a pregnant woman to successfully receive full ARV prophylaxisa

Attrition along each step of the cascade can be significant. For example, the acceptability of HIV testing strategies varies greatly in many high-HIV-prevalence settings, with significant proportions declining HIV testing or failing to return to collect test results.2224 Even after a woman is diagnosed with HIV, there is no guarantee that she will agree to ARV drug prophylaxis. Studies in Burkina Faso, Côte d’Ivoire and Kenya have found that up to 40–60% of HIV-infected women decline short-course ZDV prophylaxis in pregnancy once diagnosed,21,25 although this experience is not universal.22 Reasons for non-acceptance of testing or interventions certainly vary among these settings, but may be tied to poor understanding, patient denial and fear of stigma.26

In Lusaka, Zambia, we estimated city-wide effectiveness of our PMTCT programme using an anonymous cord blood surveillance for HIV antibodies and detectable nevirapine (NVP) drug levels.18 This study, which recreated the PMTCT cascade for 10 194 women delivering over a period of three months across all of Lusaka’s 10 public-sector facilities, demonstrated three critical areas of programme effectiveness that are not measured by most process indicator-derived PMTCT cascades: (1) women refusing testing were more likely to be HIV-infected than those accepting testing, (2) laboratory result errors and seroconversions caused a proportion of women (6%) who should have received prophylaxis not to receive it, and (3) one-third of women who were given a NVP tablet for self-administration at labour onset did not actually swallow the pill. This surveillance study demonstrated that despite a seemingly robust PMTCT programme in Lusaka, only a minority of HIV-infected women and HIV-exposed infants (30%) were receiving even minimum prophylaxis19 and that field effectiveness was likely to be much lower than estimated from standard process indicators.

Other methods for estimating PMTCT programme effectiveness have been used. Coetzee et al.27 used a facility-based case finding approach to identify infants born to women accessing PMTCT services in Khayelitsha, South Africa. Infants were located and tested for HIV using polymerase chain reaction (PCR) for early diagnosis, with a documented transmission rate of 8.8% at six weeks of age. This evaluation directly measured short-term PMTCT effectiveness but was limited by incomplete tracing of cases. Even in this intensive monitoring exercise aimed at following-up recently delivered infants, nearly 20% (149 of 684 mother–infant pairs) could not be located.

Other groups have used a prospective cohort approach to estimate PMTCT effectiveness in Africa. In Cameroon, Ayouba et al.28 followed infants in a NVP-based PMTCT programme and tested exposed infants for HIV infection at 6–8 weeks and then again at 5–6 months. The transmission rate was 13/123 (10%) at six weeks and 16/123 (13%) at 5–6 months. In rural Kenya, Songok used two-year HIV-free survival as the effectiveness outcome of a ZDV-based PMTCT programme25 and found that HIV-exposed infants whose mothers took short-course ZDV had a significantly higher 24-month HIV-free survival (59%) than those whose mothers did not take the PMTCT regimen (30%; P < 0.001). Unfortunately, follow-up losses were significant in both of these programmatic settings, introducing biases that cannot be quantified. For example, in Cameroon, nearly one-quarter of HIV-infected antenatal attendees were lost between HIV testing and time of delivery; in Kenya, 30% of live infants in the cohort were not available for HIV testing during the two-year follow-up period. Prospective approaches, although ideal, are not practical for routine monitoring in most developing countries given their expense and complexity.

Outcome measures

One critical question that must be answered to develop a consensus approach for effectiveness monitoring is “what is the appropriate outcome indicator?” The reports outlined above have used a variety of indicators, including: (1) PMTCT intervention coverage (which serves as a surrogate for infant infections prevented), (2) infant infections prevented, (3) infant deaths prevented, and (4) HIV-free survival. We propose that this latter metric, HIV-free survival, is the ideal measure for most resource-poor settings, because it captures not only the essential purpose of the programme (i.e. HIV infections at birth and through breastfeeding, as well as deaths prevented) but also incorporates survival benefits that may accrue to HIV-exposed children who do not become infected themselves. Many PMTCT programmes aim to improve general antenatal and obstetrical care (e.g. improved syphilis screening, intermittent presumptive treatment of malaria). Thus, a metric that captures benefit to all children, irrespective of their exposure or infection status, is essential.

Source populations

Another critical consideration is the source population from which the effectiveness measure is derived. Facility-based source populations have the clear advantage of convenience. Since PMTCT services are typically situated in antenatal/maternity clinics (even those that provide services in the field through traditional birth attendants use clinics as their base of operation), facilities provide the best opportunity to access patients for evaluation and the best chance of linking particular mother–infant pairs to their medical records. In a facility-based model, it is relatively easy to recreate the PMTCT cascade for individual mother–infant pairs from programme indicators, although this is subject to the reliability caveats outlined previously. Furthermore, a health-care facility offers advantages for specimen acquisition, since clinic staff members are technically skilled at blood draws and patients are accustomed to providing specimens at the clinic. The major drawback to a purely facility-based approach is representativeness. No information can be gathered on those members of the population who do not receive antenatal, obstetric or paediatric care under the current service delivery method, and only limited information is available for those who drop out. Thus, any sample from a facility is likely to be biased towards the generally better outcomes of those who receive at least antenatal or delivery services and who attend infant follow-up visits.

An alternative source population could be community-based. The advantage of community-based populations would be that the overall effect of a PMTCT programme could be measured in a more representative sample. Outcomes of those dropping out of PMTCT services at various steps would be included, and indirect benefits of the programme on family members other than the index mother–infant pair could potentially be measured. As an example, if the presence of a PMTCT programme affected general population rates of HIV testing and care, improved partner testing rates, and/or access to care and treatment, this could be measured in a population sample. The primary disadvantages of a community-based sample are expense and the need for larger sample sizes to gain precision in outcome measures such as HIV-free survival.

Design approaches

Prospective approaches are commonly used in research settings to determine the efficacy of PMTCT interventions and are suited to directly measure transmission rates and HIV-free survival in a well-defined group of patients. In either a cohort or case-control design, a comparison between PMTCT-exposed and -unexposed infants can be made, yielding a risk or odds ratio for HIV-free survival between the groups. However, this approach has major drawbacks. First, in many settings it would be prohibitively complex to enrol and follow large groups of mother–infant pairs for periods long enough to gain meaningful HIV-free survival estimates. Second, when applied to real-world effectiveness evaluations, prospective approaches suffer from an intractable problem with the Hawthorne effect.29 Since investigators would be ethically bound to provide cohort participants with best possible care (e.g. co-trimoxazole prophylaxis, referral of mother and infants for HIV care and treatment, reinforcement of clinical follow up), members of the cohort would thus become poorly representative of the population as a whole. Although this bias could be mitigated by less frequent follow-up, our experience is that fewer follow-up visits ultimately lead to a greater number of follow-up losses. These issues, combined with the technical difficulties of cohort follow up and its expense, make cohort approaches less attractive for assessing programme effectiveness.

Survey approaches that gather cross-sectional or retrospective data, with or without specimen acquisition, can also be used to determine PMTCT effectiveness.19,30 Advantages include their relative simplicity, shorter study periods and suitability to collect population-based outcomes. Another advantage of a survey approach is that retrospective information gathering could potentially detect trends over time if the number and general cause of deaths of all children under, say, five years of age, could be ascertained. If there are changes in PMTCT coverage, uptake, drug regimens or overall programme performance over time, or if implemented in areas where PMTCT services have only recently been established, the survey could compare outcomes in age-stratified groups of children and thus provide information for specific age cohorts and on outcomes as PMTCT programmes are introduced and evolve. One potential disadvantage of a survey approach would be recall bias.

A proposed model

At present there is no consensus gold standard to assess population effectiveness of PMTCT programmes. We believe that the development and implementation of such a model is an urgent public health issue. Most countries conduct periodic national surveys to monitor key health indicators. In Zambia and in over 30 other African countries, DHS provide these important population-level data.31 Developed with wide national consensus and funded by a variety of donor partners, the DHS measures, among other things, infant and child survival, maternal mortality, HIV seroprevalence, community knowledge about HIV/AIDS and various health service utilization indicators. Data from DHS have proven reliable for longitudinal research, with enough precision to measure differences in mortality over time.32

We propose that two relatively minor modifications of the DHS would allow nearly Africa-wide estimation of HIV-free child survival, and thus effectiveness of current PMTCT programmes. First, more detailed questions regarding maternal HIV history, PMTCT programme enrolment and interventions received, infant feeding practices and household child mortality are needed. Second, in sampled households, we advocate the addition of a “heel stick” for dried blood spot collection among all children less than two years of age. This specimen can be used to approximate the child’s HIV exposure and HIV infection status. To determine the proportion of children with HIV exposure, the sample is tested for HIV antibodies. For children less than 18 months and/or who are still breastfeeding, the presence of HIV antibodies indicates HIV exposure, but not necessarily infection; for children more than 18 months, their presence indicates HIV infection. Specimens of those children who are 18 months or less or still breastfeeding with positive HIV antibody tests then undergo HIV DNA PCR testing to determine infection status. One limitation of this methodology is that maternal antibodies are gradually disappearing until 18 months, and an HIV-exposed child could potentially be misclassified as non-exposed. However, if the mother were present for the survey this would not be an issue.

Through these relatively minor additions to the DHS, we believe that a reasonable and reliable measure of two-year HIV-free survival can be calculated as a simple proportion. The denominator consists of the number of children born within the past two years, estimated through the survey component of DHS. The numerator is based on the same figure, minus the number of children found to be HIV-infected [determined via HIV antibody and deoxyribonucleic acid (DNA) PCR testing] and the number of children reported to have died (derived via survey methodology). In an ideal setting, this statistic would approximate 1.0 (or 100%). In reality, however, there are many competing HIV-related and non-related factors that make perfect survival impossible on a population basis.

Although we believe HIV-free survival to be the gold standard for assessing PMTCT effectiveness, additional modifications to the DHS could provide other important population-level indicators of programme success. The addition of a “verbal autopsy” interview for recent child deaths, for example, could help approximate HIV-attributable infant mortality. In numerous African settings, verbal autopsy has been used to evaluate infant death with reliability when regular autopsy is unavailable.33,34 In households with a recent delivery, the collection of anonymous maternal venous specimens could lead to the estimates of HIV prevalence (proportion with HIV antibodies), maternal disease burden (distribution of CD4 cell counts or HIV viral load), antiretroviral therapy coverage (proportion with detectable drug concentrations among women with CD4 cell counts below 200 cells/µL) or postpartum HIV seroconversion (proportion with incident HIV infections detected through detuned assays).5

Conclusion

Given the significant health investments that have been made to eradicate paediatric HIV worldwide, the development of a consensus model to evaluate PMTCT effectiveness is long overdue and urgently needed. Unlike many currently used metrics, the outcome of HIV-free survival considers the direct and indirect benefits of PMTCT-bolstered health-care services for both antenatal women and their newborn children. The modification of the DHS described here could provide a reliable and easily replicable method. This approach would also have the added advantage of including women in the community who fail to access institutional obstetric care and are thus typically excluded from most PMTCT effectiveness assessments.

In mid-2007, we launched the PEARL (PMTCT Effectiveness in Africa: Research and Linkages to Care and Treatment) study, a consortium project funded by the US Centers for Disease Control and Prevention and the Elizabeth Glaser Paediatric AIDS Foundation. This study will seek to determine the impact of PMTCT services in Cameroon, Côte d’Ivoire, South Africa and Zambia. PEARL will be the first to utilize this novel community-based methodology to evaluate the effectiveness of PMTCT services in a variety of settings. By performing a standardized evaluation across 24 communities, we hope to better understand how PMTCT services relate to long-term infant survival and hope to contribute to the continued evaluation and improvement of PMTCT services worldwide. ■


References

Affiliations

  • Centre for Infectious Disease Research in Zambia (CIDRZ), Plot 5977, Benakale Road, Northmead, Lusaka, Zambia.
  • Centers for Disease Control and Prevention, Global AIDS Program, Atlanta, GA, United States of America.
  • PAC-CI Programme, Abidjan, Côte d’Ivoire.
  • School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa.
  • Cameroon Baptist Convention Health Board, Nso, Northwest Province, Cameroon.
  • Institut de Sante Publique d’Epidemiologie et de Developpment (ISPED), Universite Victor Segalen, Boudeaux, France.
  • Elizabeth Glaser Pediatric AIDS Foundation, Chapel Hill, NC, USA.
Share