Background:
The monitoring and evaluation framework of the International Health Regulations (2005) (IHR-MEF) is made up of 4 instruments: mandatory States Parties Annual Reporting (SPAR) and 3 voluntary instruments, voluntary external evaluation, after-action review (AAR) and simulation exercises (SimEx). The latter 2 were proposed by the WHO Secretariat to address the recommendations of the IHR review committee in May 2015 that States Parties should move from exclusive self-evaluation to a combined approach of self-evaluation, peer review and voluntary external evaluation by both domestic and independent experts. The committee also recommended that States Parties urgently conduct in-depth reviews of significant disease outbreaks and public health events (PHEs).
Between 2016 and February 2019, WHO headquarters in collaboration with regional and country offices supported 45 AARs and 97 SimEx as part of IHR monitoring and evaluation. This article summarizes the trends, common strengths and gaps in emergency preparedness and response identified through AARs and SimEx in 2018.
Methods:
A desk review was conducted of 36 reports (19 AARs and 17 SimEx) submitted by Member States to WHO in 2018. As the reports had different structures, formats and methods, the analysis was based on where the AARs and SimEx were conducted, which events and functional areas were reviewed or tested and any common findings and benefits.
Results:
The reports were from 4 of the 6 WHO regions, the African region (AFR), the eastern Mediterranean Region (EMR), the European Region (EUR) and the Western Pacific region (WPR). Of the reports on SimEx, 47.1% (8/17) were from EUR, followed by AFR (29.4%; 5/17), EMR (17.6%; 3/17) and WPR (5.9%; 1/17). Of the AAR, 73.7% (14/19) were from AFR, 10.5% (2/19) from EMR and from both WPR and 5.3% (1/19) EUR. The majority (88.2%; 15/17) of the SimEx were table-top exercises. Only 1 functional exercise and 1 field or fullscale exercise were conducted. National exercises comprised 64.7% (11/17) of the SimEx, regional exercises comprised 29.4% (5/17) and global exercises comprised 5.9% (1/17). The scenarios of most SimEx (76.5%, 13/17) involved biological agents such as Ebola and cholera viruses, followed by natural disasters such as floods and earthquakes (23.5%, 4/17). The proportions were similar for AARs, with 81% (17/21) of the PHEs reviewed due to biological agents and 14% (3/21) to natural disasters. One AAR was conducted on chemical poisoning from food (Figure 1). The number of functional areas validated or tested in SimEx ranged from 2 to 7, communication ranking first (64.3%, 11/17), followed by coordination (47.1%, 8/17) and surveillance (41.2%, 7/17). The results were slightly different for AARs, with coordination ranking first (85.7%, 12/14), followed by risk communication (78.6%, 11/14) and surveillance (71.4%, 10/14). All the SimEx reports listed gaps and areas for improvement, the most common being lack of understanding of the roles and responsibilities in emergency response (29.4%, 5/17), followed by difficulties in coordination (23.5%, 4/17) and unfamiliarity with documents and standard operating procedures (23.5%, 4/17). A fourth major gap identified was lack of medical supplies and equipment (11.8%, 2/17).
Identification of strengths is equally important for maintaining and building on best practices. All the AAR reports listed strengths, but they were too broad and diverse for analysis. Only 35.3% (6/17) of the SimEx reports listed strengths, which were mainly related to the existence of plans and procedures (82.3%, 5/6). The date of the end of the event was mentioned in 52.6% (10/19) of the AAR reports. The interval between the end date and the date on which the AAR was conducted ranged from 25 to 279 days, with a median of 122 days, which is longer than the 90 days recommended by WHO. Anonymous feedback is often collected at the end of an AAR or SimEx, which provides an opportunity to determine what participants have learnt. When feedback was reported, the most common benefit reported was increased understanding of leadership and management structures, such as the incident management system (100%, 3/3). Other benefits included staff training and learning, engendering enthusiasm, knowledge, skills and willingness to participate in emergency response.
Discussion and conclusions:
AARs and SimEx were conducted unevenly among the WHO regions in 2018, and awareness of their benefits for evaluating functional IHR capacity should be increased through advocacy for timely AAR of actual events and use of SimEx for functional assessments in the absence of a recent PHE. Biological agents were the most common cause of the PHEs reviewed or simulated event. Table-top exercises were the most frequent type of SimEx, which may reflect the fact that such exercises are more cost-effective and less resource-intensive than operation-based exercises. Both SimEx and AAR are important learning tools and effective methods for informing stakeholders about best practices, challenges and the causes of gaps in preparedness. Although all the AAR and SimEx reports included clear recommendations, it is not known whether they have since been implemented and whether they improved emergency response capacity. Post-AAR and post-SimEx follow-up is recommended to validate any expected or assumed improvements. Other IHR-MEF components, including SPAR and voluntary evaluations, could be used to review or validate implementation of recommendations. The study was limited by the fact that few reports were available. This is due to the voluntary nature of these instruments and inconsistencies in the structure, format, methods and availability of key information. WHO has published new guidance for AARs and SimEx in the IHR-MEF,2 which provides strategic guidance and criteria for inclusion of AAR and SimEx and introduces a standardized minimum reporting template, with indicative timelines for AARs. This will improve analysis of the findings for better conclusions about the preparedness and IHR capacity of Member States in the future.
Author affiliations:
Country Health Emergency Preparedness and International Health Regulations, Health Emergencies Programme, World Health Organization, Geneva, Switzerland (corresponding author: Landry Ndriko Mayigane, mayiganel@who.int).
More language: WER24may2019-xxxiv-xxxvii-eng-fre