Bulletin of the World Health Organization

Setting research priorities for adolescent sexual and reproductive health in low- and middle-income countries

Michelle J Hindin a, Charlotte Sigurdson Christiansen b & B Jane Ferguson b

a. Johns Hopkins Bloomberg School of Public Health, Department of Population, Family and Reproductive Health, 615 N Wolfe Street, Baltimore, MD 21205, United States of America.
b. Department of Maternal, Newborn, Child and Adolescent Health, World Health Organization, Geneva, Switzerland.

Correspondence to Michelle J Hindin (e-mail: mhindin@jhsph.edu).

(Submitted: 28 May 2012 – Revised version received: 08 October 2012 – Accepted: 09 October 2012 – Published online: 02 November 2012.)

Bulletin of the World Health Organization 2013;91:10-18. doi: 10.2471/BLT.12.107565


Adolescent sexual and reproductive health is an area in need of research and evidence-based policies. Nearly one fifth (17.5%) of the world’s inhabitants are adolescents (i.e. people aged 10–19 years), and in the least developed nations, this group comprises an even higher proportion (23%) of the population.1 In 2004, 2.6 million deaths occurred among the world’s 1800 million youth between the ages of 10 and 24 years, and 97% of these deaths took place in low- and middle-income countries.2 Over the past 50 years, the health of adolescents has improved at a slower pace than the health of younger children.3 This is partly because early pregnancy carries a high risk of serious complications and also because approximately 40% of all new HIV infections occur in people between 15 and 24 years of age.4 Improving the sexual and reproductive health of adolescents is essential for achieving Millennium Development Goals 4, 5 and 6.3,5,6

Despite governments’ commitment to address the health problems commonly affecting adolescents,7,8 little evidence has been generated on whether or not such commitment has made a difference.9 Findings from research are important; they can provide vital information for the public, inform health policy and reinforce efforts to protect adolescents’ rights. The support given by the World Health Organization (WHO) to research on adolescent sexual and reproductive health since the mid-1980s has contributed to the development of programmes in this area in many countries,1013 yet in a recent survey that investigated perceived research priorities in reproductive health, most respondents still put adolescents at the top of the list.14 The exercise described in this paper is intended to help policy-makers and donors to identify those areas of adolescent sexual and reproductive health research that should be prioritized for research funding.


To help decision-makers, including donors, to effectively allocate limited resources to reduce morbidity and mortality, the Child Health and Nutrition Research Initiative (CHNRI) developed a method for ranking the relative importance of competing research options.14 The CHNRI approach was specifically modified to identify and rank those areas of adolescent sexual and reproductive health in which research is most urgently needed. Although this paper describes the first application of the CHNRI method to health problems affecting adolescents, more than 50 similar applications have been undertaken among various populations to prioritize research outcomes in other areas of health.1521

We implemented the CHNRI approach in three phases. In Phase 1, we asked research and programme experts to rank 10 health outcome areas in order of importance. In Phase 2, we asked these individuals to propose research questions for each outcome area. In Phase 3, we asked them to prioritize the research questions generated in Phase 2 using a scoring scheme based on five criteria.

Phase 1

We asked researchers and programme experts in adolescent health to rank 10 potential priority areas (Fig. 1) having to do with the sexual and reproductive health of adolescents, defined as people aged 10–19 years, in low- and middle- income countries. These areas were selected based on a review of the literature on the known leading causes of adolescent morbidity and mortality linked to sexual and reproductive practices in low- and middle-income countries.2,22 We developed a survey tool using SurveyMonkey (Palo Alto, United States of America) and sent e-mails to 94 researchers and programme experts working in the field of adolescent sexual and reproductive health, our aim being to get feedback from people with international expertise in the outcome areas of interest. With these criteria in mind, we used a snowball method to try to generate 100 names but succeeded in generating 94 (64 females and 30 males). Of the experts we identified, 50 had interregional experience: 16 of them primarily in Africa; 16 in Asia; 8 in Latin America and 3 in the eastern Mediterranean region. We also requested input from 27 WHO staff members (at headquarters in Geneva, Switzerland, and in regional and country offices); 11 representatives of donor organizations from the United States and Europe; representatives of United Nations organizations other than WHO; and 14 employees of the International Planned Parenthood Federation in field offices in Africa, Asia, Europe and the Americas. As this Phase of our study was anonymous, we cannot give more details on the final pool of respondents.

Fig. 1. Ranking of potential priority areas for research on adolescent sexual and reproductive health (Phase 1 of the research priority-setting process)
Fig. 1. Ranking of potential priority areas for research on adolescent sexual and reproductive health (Phase 1 of the research priority-setting process)
ANC, antenatal care; FP, family planning; GBV, gender-based violence; HIV, human immunodeficiency virus; HPV, human papillomavirus; MTCT, mother-to-child transmission; STI, sexually-transmitted infection. Note: the scoring was from 1 to 10.

We asked all individuals identified through the method described above to rank the 10 outcome areas generated from the literature in decreasing order of importance. We only allowed mutually exclusive categories to “force” respondents to provide a rank order. In addition, written surveys were administered to 13 programme managers during an International Planned Parenthood Federation meeting held in The Hague, the Netherlands, on 27 October 2011. We received 53 completed surveys (50% response rate). All responses in this Phase were anonymous.

Fig. 1 shows the mean scores resulting from the ranking of the outcome areas. Although some areas scored relatively low (e.g. prevention of mother-to-child transmission of HIV or reduction of human papillomavirus infection rates), we decided to include these lower-scoring items within other areas and this consolidation reduced the number of areas from 10 to 7 (Table 1).

Phase 2

In this phase, we divided people into groups based on their expertise in the seven outcome areas in Table 1. An analysis of the people who were asked to provide input (available from the corresponding author) showed that they were mainly from low- and middle-income countries and either academics, donors, staff members of United Nations and other international nongovernmental organizations (NGOs), government officials or staff members of national NGOs. For any given area, we aimed to have at least 10 respondents propose research topics. Table 1 shows the number of responses received. To facilitate the development of research questions, we prompted participants by asking them what issues need to be addressed within each outcome area, in the near (2015) or longer term (2020), through research of the following types:

  • epidemiological research (i.e. descriptive research, designed to measure burden of disease, explore risk factors and protective factors, or evaluate existing research interventions);
  • operations research (i.e. development research, designed to improve the deliverability, affordability, sustainability and scale-up of existing interventions);
  • discovery research: designed to develop new interventions.

During this exercise we asked the respondents to provide their names and contact information in case we needed to have their responses clarified. No limits were imposed on the research questions the respondents could suggest.

After receiving the responses, we synthesized the results in three steps. In the first step, two independent coders per area developed clearly-worded research questions from the respondents’ textual replies. In the second, one member of the team (who did not participate in the first step) harmonized the questions between the two coders. Third, one member of the team streamlined the questions, removed redundancies, repositioned those that belonged under different outcomes (e.g. abortion questions that appeared under contraception) and eliminated those that would not lead to valuable research outputs. The goal was to have a maximum of 40 questions per outcome area.

After the questions were synthesized, we created a web site where the respondents who generated questions could review them in their totality for a given outcome area and suggest rewording, removing or adding questions. This web site was viewed by 45 people from countries in Africa, Asia, Europe, Latin America and the Caribbean, North America, and Oceania. Individuals spent an average of 7.5 minutes on the site, which was monitored by Google Analytics®. We revised the questions based on the suggestions received.

Phase 3

In this phase, we selected five criteria for ranking the research questions generated in Phase 2. We based these criteria on previous applications of CHNRI processes1016 and on what made sense for adolescent sexual and reproductive health research.

The criteria were:

  • Clarity: Is the question well framed and are its end-points clear?
  • Answerability: Can the question generate important new knowledge in an ethical way?
  • Impact: Would the answer to this question result in an effective intervention?
  • Implementation: Would the answer to this question result in an intervention or a strategy with a strong likelihood of being affordable and sustainable in most low- and middle-income countries?
  • Equity: Would the answer to this question help to reduce inequity in disease burden over the next 10 years?

To diversify the set of rankings, we assigned potential respondents to the areas in which their expertise was strongest, as we did in Phase 2, and we also randomly assigned them to a second area. Using anonymous SurveyMonkey surveys, we invited 296 people to participate. Most of these people were on our previous list of experts and some were identified by a snowball technique. For each of the seven outcome areas we asked respondents to state whether the research question did or did not meet a given criterion (yes or no) or if they were undecided regarding this point.

Our goal was to get at least 17 responses per outcome area. This is thought to be the minimum number needed to achieve consensus at this stage (Igor Rudan, personal communication, May 2012). Table 1, third column, shows the number of Phase 3 respondents in each area.


The main results from this exercise come from Phase 3. For the analysis of the rankings, we exported all of the responses into an Excel spreadsheet. For each of the five criteria, we used the standard CHNRI scoring system: yes = 1; no = 0 and undecided = 0.5. In this way we developed a mean score on each criterion for each question, and by adding these scores and dividing by five we obtained each question’s mean overall score. We weighted all criteria equally.

In Table 2 we show the highest-ranking research questions by outcome area. We provide each question’s mean overall score and its score on each criterion (ranging from 0 to 1). In general we show the top five questions, but in one outcome area (sexually transmitted infections and infection with the human papillomavirus) we present the top six because two scores were tied (a full set of scores can be obtained from the corresponding author).

We found a high level of agreement on the most important research questions in each of the seven outcome areas, with total mean scores ranging from 0.84 to 0.97 (out of a possible 1.00). The scores on individual criteria differed depending on the research question, both within and across outcome areas.

In Phase 2, questions initially showed substantial overlap across different outcome areas, particularly contraception and abortion. However, in the final ranking of the questions these overlaps were minimal, although contraception was mentioned under three areas: maternal health, abortion and integration of family planning and HIV services.

Although we did not take the three prompting questions about research type into account when weighting the mean scores, two coders took note of the type of research needed to address each research question. Table 2 (second column) shows the type required to address the top-ranking research questions. If we consider the full set of questions, descriptive research was the type most frequently required, but development research was the type most commonly needed to address the five top-ranking questions in each outcome area (data not shown).


Using a modified version of the priority-setting method developed by the CHNRI, we sought input from nearly 300 experts in adolescent sexual and reproductive health to identify priority outcome areas and research questions. The experts we consulted, who included researchers, programme managers and donors, came from all parts of the world. The CHNRI process is rigorous; it gathers input from a wide range of sources and ultimately attains a high degree of consensus on research priorities.

A key limitation of our exercise is that some of the experts we approached failed to respond to our questions. Although we used several methods to try to generate responses, we cannot rule out the presence of non-response bias. Nonetheless, we are confident that the questions generated by our experts are valid, since during each Phase of our exercise we had a greater number of respondents than the minimum required by the CHNRI method. In addition, we used Google Analytics® and other methods to verify that we had correctly interpreted the input provided by the experts. We also used multiple coders to generate and frame the research questions, and in the final Phase of the study, when experts ranked the research questions, we randomized the respondents to different outcome areas and changed the order of the questions.

The outcome areas featured in this exercise have to do with the prevention of health problems stemming from adolescents’ sexual behaviour, which is often impulsive and unplanned, and with adolescents’ access to effective interventions, which various factors can hinder.23 The top-ranking research questions suggest a widespread impression that the definition of the problems affecting adolescents, and the delivery and assessment of specific interventions, need to be improved. There was also concern over the needs of particular subpopulations, such as adolescent boys and married adolescents, which reflects an awareness that adolescents do not comprise a homogeneous group and that they live in widely different contexts. Many of the top-ranking questions suggest that interest has shifted away from basic prevalence questions and towards questions dealing with the scale-up of existing interventions and the development of new ones.

We ranked questions by overall score, derived by averaging the scores for all five criteria. Although the total score provides a summary rating, in some circumstances the score for a particular criterion is viewed as more important than the overall score. If, for example, donors were reviewing the research questions, they might be more attentive to the scores given to impact and implementation than to the overall score.

Almost two decades ago, the Programme of Action of the International Conference on Population and Development focused specifically on the problems affecting adolescents. These included adolescent pregnancy, HIV transmission and gender inequality in the areas of reproductive health and rights, as well as the impact of these problems on poverty and development in general. This highlighted the stake young people have in the development process and the general relationship between health, population and development, a message that has been reinforced in recent international forums.24 Renewed attention is now being devoted to the review of the Programme of Action and to the Millennium Development Goals.

Research and programmatic evidence have provided reasons for both optimism and pessimism25,26 regarding the well-being of adolescents and efforts to address their reproductive health needs. The absolute and relative growth in the adolescent population is levelling off everywhere except in Africa,27 but the need to pay increased attention to the needs of adolescents in the area of sexual and reproductive health will continue. The crucial role of research for improving policies and programmes in this area is well known.28 Because adolescents are all different and live in dissimilar contexts, regional and national exercises will undoubtedly be needed to identify and prioritize the research most pressingly needed in a given society. Nonetheless, the results of the work we have performed can help global donors and programme managers in their efforts to prioritize funding for research on adolescent sexual and reproductive health. As a further impetus for research, we subsequently held a workshop in which researchers developed concept notes for the top-ranking questions (available from the corresponding author).


We would like to acknowledge Rajiv Bahl, Lucy MacNamara and Igor Rudan.

Competing interests:

None declared.