Bulletin of the World Health Organization

Child maltreatment prevention: a systematic review of reviews

Christopher Mikton a & Alexander Butchart a

a. Department of Violence and Injury Prevention and Disability, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland.

Correspondence to Christopher Mikton (e-mail: miktonc@who.int).

(Submitted: 01 October 2008 – Revised version received: 23 January 2009 – Accepted: 01 February 2009.)

Bulletin of the World Health Organization 2009;87:353-361. doi: 10.2471/BLT.08.057075

Introduction

Child maltreatment prevention is poised to become a global health priority due to four main factors. First, retrospective and prospective studies have established that child maltreatment has strong, long-lasting effects on brain architecture, psychological functioning, mental health, health risk behaviours, social functioning, life expectancy and health-care costs.1,2 Second, the full implications of these effects on human capital formation, the workforce, and, ultimately, social and economic development in low-, middle- and high-income countries are now better understood.3,4 Third, epidemiological studies have clearly established that child maltreatment is not peculiar to the West but a truly global phenomenon that occurs in some low- and middle-income countries at higher rates than in wealthier countries.5,6 Fourth, evidence strongly suggests that treating and later trying to remedy the effects of child maltreatment are both less effective and more costly than preventing it in the first place.7 Despite this, epidemiological data on and policies and programmes against child maltreatment are conspicuously lacking in most low- and middle-income countries, and in high-income countries, such as the United States of America (USA), investment in child protection systems continues to outweigh prevention budgets.8

This systematic review of reviews of the effectiveness of child maltreatment prevention interventions aims to add to existing reviews by:

  • providing an up-to-date synthesis of recent evidence;
  • evaluating the quality of the systematic reviews included;
  • assessing the methodological quality of the outcome evaluations included in the reviews;
  • mapping the geographical distribution of the studies included in the reviews.

Methods

The following English and non-English language electronic databases were searched by one reviewer, with no language restrictions: Medline, PsychINFO, Embase, CINAHL, Social Sciences Citation Index, Science Citation Index, LILACS, ERIC, NCJRS, the Campbell Library, the Cochrane Library, WorldWideScience, KoreaMed, IndMED, and Google. In addition, reference lists of review articles and the Journal of Child Abuse and Neglect were searched, and 10 international experts were consulted. For inclusion, reviews had to evaluate the effectiveness of “universal interventions” (those aimed at the general population without regard to risk) or “selective interventions” (those aimed at people at higher risk), but not “indicated interventions” (those carried out once child maltreatment has already occurred); be published between January 2000 and July 2008; be either systematic or comprehensive (i.e. covering a wide range of relevant studies); and include at least one of the following outcomes: physical abuse, sexual abuse, neglect, or emotional abuse perpetrated by a parent or caretaker against a child (bullying and witnessing intimate partner violence were excluded). Only easily accessible reviews were included (i.e. published in a peer-reviewed journal, a book, or online), since the aim was to focus on reviews with a wide influence on policy and practice. Hence, less easily accessible grey literature, such as theses and dissertations, conference proceedings and reviews that were neither published nor available online, was excluded. Full details of the search strategy and of inclusion and exclusion criteria, as well as a list of the studies excluded, are available from the authors.

A second reviewer independently screened 25% of the studies identified and previously screened by the first reviewer. Both reviewers assessed the full text of all reviews in light of the inclusion criteria. Uncertainties were discussed and consensus was reached in all cases.

Evidence for the effectiveness of each main type of intervention was graded independently by two reviewers (Table 1)934 with an adaptation of a pre-existing system,35 and there were no disagreements.

The methodological quality of the systematic reviews was evaluated with AMSTAR, a measurement tool for the assessment of multiple systematic reviews that has good reliability and validity.36,37

To assess the quality of relevant individual outcome evaluation studies included in the reviews and map their geographical distribution, single publications on individual outcome evaluation studies were used as the unit of analysis. The rationale for selecting individual publications over studies was that different publications based on the same outcome evaluation study could be reporting different outcomes of interest. However, when a publication was on two different outcome evaluation studies, for instance, it was included twice. The number of times individual publications were included in the reviews also served as a rough proxy for the comprehensiveness of reviews’ search strategies.

Two dimensions of the methodological quality of individual outcome evaluation studies were assessed: (i) internal validity, based on the research design (randomized controlled, non-randomized controlled and having no control group); and (ii) construct validity of the outcome measure, categorized into direct measures of child maltreatment (e.g. reports from child protective services), proxy measures (e.g. emergency department visits, hospitalization for injury), or risk factors (e.g. measures of child abuse potential, parental stress).

Results

Synthesis of evidence

Overall

Of the 3299 titles identified through the search strategy, 26 met the inclusion criteria (Table 1).934 The full-text versions of 53 further reviews were considered and excluded. The 26 reviews included summarized 298 publications on primarily single outcome evaluation studies and another 85 reviews and commentaries. The following seven main types of interventions were included in at least two of the 26 reviews and are the most widely implemented and evaluated types of interventions. This typology reflects that used by the reviews themselves.

Early childhood home visitation

Trained personnel visit parents and children in their homes and provide support, education and information to prevent child maltreatment. They also seek to improve child health and parental caregiving abilities. Of the 26 reviews that satisfied the inclusion criteria, 17 summarized evidence on early childhood home visitation programmes (Table 1) based on 149 publications on individual outcome studies and several further reviews. This made it the most extensively evaluated type of intervention.

Although Bilukha et al., MacMillan, and Daro & McCurdy recommend early child home visitation for preventing child maltreatment on the basis of “good” or “strong” evidence from direct outcome measures,17,22,25 others reach more tentative conclusions. Barlow et al. consider the evidence equivocal due mainly to surveillance bias (i.e. an increased likelihood that child maltreatment will be observed and reported due to the presence of a visitor in the home).9 Bull et al. and Elkan et al. both consider the evidence inconclusive due to methodological problems, including surveillance bias.10,18 Sweet & Appelbaum found that the effect size for this type of intervention was not significantly different from 0 in the case of actual abuse.15 Overall, these reviews suggest that early home visitation programmes are effective in reducing risk factors for child maltreatment, but whether they reduce direct measures is less clear-cut. Several reviews single out Olds et al.’s Nurse Family Partnership in the USA as the only home visiting programme whose effectiveness has been unambiguously demonstrated. A randomized controlled trial showed a 48% reduction in actual child abuse at 15-year follow-up.38

Parent education programmes

This type of intervention, usually centre-based and delivered in groups, aims to prevent child maltreatment by improving parents’ child-rearing skills, increasing parental knowledge of child development, and encouraging positive child management strategies. Seven of the 26 reviews summed up evidence relevant to this type of intervention from a total of 46 individual publications on outcome evaluation studies and from several other reviews.

Two of the meta-analyses reported small and medium effect sizes for parent education programmes on the basis of both risk factors and direct measures of child maltreatment.12,13 Other reviews concluded, however, that while the evidence shows improvements in risk factors for child maltreatment, evidence of an effect on actual child maltreatment remains insufficient.

Child sexual abuse (CSA) prevention programmes

Most of these programmes are universal programmes delivered in schools and teach children about body ownership, the difference between good and bad touch, and how to recognize abusive situations, say no, and disclose abuse to a trusted adult. Of the 26 reviews, 11 included evidence on CSA prevention programmes from a total of 74 publications and several other reviews.

These reviews are all but unanimous in the finding that, on the one hand, school-based interventions to prevent child sexual abuse are effective at strengthening protective factors against this type of abuse (e.g. knowledge of sexual abuse and protective behaviours) and, on the other, that evidence about whether such programmes reduce actual sexual abuse is lacking. Two studies that measured future sexual abuse as an outcome reported mixed results.39,40

Abusive head trauma

Only three reviews, which included a total of four publications on outcome evaluations, focused on interventions to prevent abusive head trauma, also referred to as shaken baby syndrome, shaken infant syndrome and inflicted traumatic brain injury.

The most important study to date in this field, included in two of the reviews,21,28 is an evaluation of a comprehensive hospital-based parent education programme in New York State.41 The programme was found to reduce the incidence of abusive head trauma by 47%, yet Klevens concludes that, because of methodological flaws in existing studies, it remains unclear whether interventions to reduce abusive head trauma are effective.21

Multi-component interventions

Three reviews, which included a total of seven publications, discussed multi-component interventions, which typically include services such as family support, preschool education, parenting skills and child care. Two reviews judged the evidence for their effectiveness in reducing risk factors for child maltreatment as mixed9 or insufficient,22 and another27 as promising. A meta-analysis found the effect size of multi-component interventions to be 0.58.14

Media-based interventions

Media campaigns to raise public awareness are often regarded as a critical part of any child maltreatment strategy. Three reviews focused on the effectiveness of such campaigns and surveyed a total of five publications. Two found the evidence was either mixed25 or insufficient.33 MacLeod & Nelson, based on two studies (only one of which was included in the two other reviews) found a large effect size (1.26) in the reduction of risk factors for child maltreatment for this type of intervention.14

Support and mutual aid groups

Two reviews focused on social support and mutual aid groups aimed to strengthen parents’ social network. MacLeod & Nelson found an effect size of 0.38 for interventions that used risk factors for child maltreatment as an outcome,14 whereas Barlow et al. conclude that such interventions are not effective.9

Quality of the systematic reviews

Three of the reviews were of low quality (i.e. AMSTAR scores between 0–4), 10 were of moderate quality (5–8), and two, including the Cochrane Review by Zwi et al., were of high quality (9–11)16 (Table 2). The overall mean AMSTAR score for the 15 systematic reviews included in this study was 6.3 (standard deviation, SD: 1.88).

The minimum standards for the research designs of the studies included were specified in 17 of the 24 reviews (excluding the reviews of reviews). In six of the 11 reviews that included studies with no control group, the latter comprised 20% or more of the total, and in one case, as much as 50%, In 11 of 24 reviews, the proportion of outcome evaluations in which risk factors for child abuse were the outcome measure was at least half. A high proportion of designs without control groups and with outcome variables based on risk factors was equally frequent among meta-analyses, systematic reviews and comprehensive reviews (Table 2).

Individual publications were included in the 26 reviews a mean of 1.68 times (SD: 1.51). Included most often – a total of 11 times – were two evaluations of the Nurse Family Partnership by Kitzman et al.42 and Olds et al.38

Quality of the studies included in the reviews

Internal validity

Of the 298 publications included, 140 (47%) were studies with randomized controlled designs; 82 (27.5%) had non-randomized controlled designs; and 45 (15.1%) had designs with no control group (Table 3). The remaining 3.1% of studies had other designs (e.g. time-series designs, surveys, or qualitative analyses). For early home-visiting programmes, around 59.1% had randomized designs. For parent education the proportion with randomized controlled designs was considerably lower, at 28.3%, than the 39.1% with non-randomized controlled designs.

Construct validity of the outcome variable

Direct measures of child maltreatment were used in less than one-third of the publications on outcome evaluations (Table 4), and in around 64.4% of them, risk factors were used as an indicator of child maltreatment. The proportion of direct measures was highest (44.3%) for early home visiting programmes, considerably lower (17.4%) for parent education programmes, and exceedingly low for child sexual abuse prevention (2.7%), for which only risk factors were included in around 97% of the studies.

Geographical distribution of the evidence

Of the 298 publications on outcome evaluation studies included in the reviews, 296 (99.4%) were on studies in high-income countries (around 83% in the USA), two (0.6%) in middle-income countries – a study on a sex abuse prevention programme in China and another on kangaroo mother care and the mother–child bond in Colombia – and none in low-income countries (Fig. 1). Of all publications, 290 (almost 97.3%) were on studies in English-speaking countries. Studies carried out in French-speaking Canada were not included among those carried out in English-speaking countries. In 10 (4%) of the 298 publications it was not possible to determine the country where the study was carried out, so the authors’ institutional affiliation was used as a proxy instead.

Fig. 1. Distribution by country of outcome evaluations in a systematic review of reviews of child maltreatment interventions
Fig. 1. Distribution by country of outcome evaluations in a systematic review of reviews of child maltreatment interventions

Discussion

There is evidence that four of the seven types of universal and selective interventions examined in the 26 reviews are promising for preventing actual child maltreatment: home visiting, parent education, abusive head trauma prevention and multi-component programmes (Table 1). The evidence, in relation to actual child mistreatment, on the three remaining types – child sexual abuse prevention, media-based interventions and social support and mutual aid groups – is either insufficient or mixed. It is important to emphasize that when a particular type of intervention is judged to be promising, it may mean that only a single programme has been unambiguously shown to be effective, as is the case for home-visiting programmes.

Due to methodological limitations of the reviews themselves and the outcome evaluations they are based on, conclusions about effectiveness must remain tentative. The mean AMSTAR score of 6.3 indicates that the quality of the systematic reviews is, on the whole, only moderate. A conspicuous weakness was the failure of seven of the 24 reviews to explicitly set a minimum threshold for the quality of the research designs of the outcome evaluations to be included. Furthermore, the mean number of times individual publications were included in the reviews was 1.68, which suggests that searches were less than comprehensive.

Two methodological weaknesses of the outcome evaluation studies were repeatedly highlighted in the reviews themselves: weak internal validity and inappropriate outcome measures. The analysis of internal validity showed that some 15% of the publications included in the reviews failed to use a control group, and that for child sexual abuse prevention and parenting education the proportion increased to 18.9% and 23.9%, respectively. Such designs offer a particularly poor basis for causal inference and often result in “uninterpretable” findings.43 Non-randomized controlled designs were used in about 27.5% of the publications overall and in 21.5%, 39.1% and 31.1% of the publications on home visiting, parent education and child sexual abuse prevention, respectively. Although the quality of the non-randomized controlled studies was not assessed here, the internal validity of the research designs of most of these studies is generally considered to be weak.43,44

The empirical examination of surveillance bias, a problem affecting outcome measures in home visiting evaluations, suggests that its importance is often exaggerated and that it rarely substantially alters findings.45 However, evidence that score changes on risk factors for child abuse do not always correspond to the likelihood of future abuse is further reason to treat some of the conclusions of this review with caution:46 of the outcome variables reported in the publications included in these reviews, 64.4% were risk factors rather than measures of actual abuse.

Of the three meta-analyses that examined the association between methodological quality and effect size, all found that studies with poorer methodological quality had larger effect sizes.11,13,15 The significant proportion of methodologically weak studies in this evidence base is hard to justify. Sound principles of evaluation and prevention research were formulated some time ago47,48 and have recently been developed into a clear set of standards.44 Cumulative knowledge on child maltreatment prevention is ill served by an ever increasing accumulation of methodologically questionable studies.

This study has revealed a woeful imbalance in the geographic distribution of child maltreatment prevention research: over 99% of the publications were on studies conducted in high-income countries, a parallel of the 10/90 gap in other areas of health research. It cannot be assumed that current evidence about the effectiveness of universal and selective child maltreatment programmes applies outside high-income countries. Given differences in culture and risk factors and reduced institutional capacity for evidence-based child maltreatment programme implementation and evaluation, it is likely that programmes would require extensive adaptation and re-evaluation in low- and middle-income countries to be effective.

This review has the following limitations. First, although the databases searched covered some non-English language sources, the inclusion of further non-English language databases might have identified additional reviews. Second, a recent review published in the Lancet was not included, since it appeared after this review was completed.49 However, its main conclusions – e.g. that more controlled trials using actual outcomes of maltreatment are needed – reinforce the main messages of this review. Third, in the case of four reviews it was not possible to separate the conclusions derived from the small number of “indicated” interventions included. Fourth, three other methodological quality dimensions of outcome evaluation studies – namely, treatment fidelity, statistical conclusion validity and descriptive validity – were not assessed.50 Lastly, only the most easily accessible grey literature was searched. The two main types of grey literature excluded were theses and dissertations and conference proceedings, neither of which is, in general, an important source of systematic and other reviews. Overall, these limitations are unlikely to undermine the main conclusions.

Conclusion

Methods and standards for developing sound and effective child maltreatment prevention interventions are available and have been successfully applied. There is evidence that four of the seven main types of universal and selective interventions to prevent actual child maltreatment are promising, but methodological weaknesses in both the reviews and the individual studies included in them render this conclusion tentative.

In low- and middle-income countries, child maltreatment represents a greater health burden and slows economic and social development to a greater extent than in high-income countries. Yet research on the effectiveness of universal and selective interventions appears to be almost exclusively the affair of English-speaking, high-income countries. ■


Competing interests: None declared.

References

Share