Serious AEFI

Interrater reliability of causality assessment for serious adverse events following immunization

An adverse event following immunization (AEFI) is defined as any untoward medical occurrence following immunization which does not necessarily have a causal relationship to the vaccine. The adverse event may be any unfavourable or unintended sign, abnormal laboratory finding, symptom or disease. Based on the advice from GACVS to review the causality assessment system, WHO commissioned a group of experts to develop a methodology and tools to assist health-care personnel in the assessment of causality of an adverse event and use of a vaccine. The causality assessment (CA) methodology and tool developed included an eligibility component for the assessment that reviews the diagnosis associated with the event and identifies the administered vaccines; a checklist that systematically guides users to gather available information to feed a decision algorithm; and a decision support algorithm that assists the assessors to arrive at a classification of the individual AEFI (manual classification). This revised methodology was endorsed by GACVS at its meeting in June 2012.¹⁴ A causality assessment manual and AEFI causality assessment software were developed (electronic classification). Final classification generated by the process includes 4 categories in which the event is either: (1) consistent; (2) inconsistent; (3) indeterminate with respect of causal association; or (4) unclassifiable. Feedback obtained from end users of this methodology included the importance of conducting a validation study regarding the interrater agreement of the classification.

To address this concern, an interrater study of causality assessment for serious AEFIs was conducted in April 2017 to evaluate the reliability (i.e. the degree to which an assessment tool produces consistent results between country evaluators) of the methodology and to compare the manual AEFI causality assessment method with the electronic method. The study was conducted using serious AEFI cases from India and Zimbabwe. Each country had 2 assessing teams and each team had 4 persons with expertise in paediatrics, epidemiology, pharmacovigilance and public health.

During the GACVS meeting in December 2017, members were presented with the methods and findings from the interrater study as well as proposed changes to the worksheets and manual. Overall, there was a good concordance of ratings with >80% agreement on cases between experts from both countries and >80% agreement between experts from the same countries using the manual or the electronic methods. Based on the data obtained from this study, it was concluded that the methodology was reliable and the electronic AEFI CA methodology was a suitable process for CA at country level and for comparison of the CA across countries. The CA appeared to be influenced by the quality of the case report and understanding of country specific processes of AEFI reporting, along with the experience of the assessors. A full scientific report is in preparation.

The results derived from the interrater study, as well as qualitative feedback from study participants, were used to further revise the worksheet and CA methodology by a GACVS working group. The Committee made several recommendations. To further refine the algorithm of the CA tool, GACVS recommended a systemic analysis of unclassifiable events; an analysis of the questions posed as part of the checklist to evaluate interrater agreement regarding the responses provided; the inclusion of additional AEFIs, such as seizures; and inclusion of additional evidence, if available, in the CA to support a potential causal association of an AEFI with vaccination. Based on the qualitative analysis from the evaluation exercise GACVS also provided suggestions to clarify some aspects of the methodology and supporting guidance documents. The Committee looked forward to the publication and online availability of the revised AEFI causality assessment manual; the updating of the AEFI causality assessment software with the revised inputs; and the translation into additional UN languages of both the manual and electronic tools.

¹⁴ See No. 30, 2012, pp. 284–286.

Full report of GACVS meeting of 6-7 December 2017, published in the WHO Weekly Epidemiological Record of 19 January 2018

Serious AEFI during primary infant vaccination series in South India

India is currently using pentavalent (DwPT-HepB-Hib) vaccine from local manufacturers. The vaccination series is carried out nationwide via a stepwise introduction process initiated in December 2011. To accompany this effort and address safety concerns on the potential of AEFI, the Indian authorities, with INCLEN Trust International, conducted a prospective dynamic cohort study in 2 southern districts (Kollam, Kerala and Coimbatore, Tamil Nadu). This study evaluated the association between routine pentavalent and oral poliovirus (OPV) vaccination and all-cause deaths and hospitalizations (referred to as serious AEFI) among infants after receipt of each of the 3 doses of vaccine in the primary immunization schedule. The 2 districts were selected on the basis of their low infant mortality (reducing the background of coincidental events), high coverage with the primary vaccination series, well established use of pentavalent vaccine, and robust primary health-care infrastructure.

The primary study objective compared all-cause death and hospitalization rates at 0–7 days versus 22–28 days after each vaccine dose administered. Secondary objectives included comparing all-cause death and hospitalization between 8–14 days and 15–21 days versus 22–28 days after vaccination. For infants with a delayed second or third dose of vaccine, the risk of serious AEFI was estimated for the period to the following dose, or at 24 weeks of age, or death, whichever occurred first. Infants were enrolled at the time of their first dose of pentavalent and OPV vaccines and followed weekly until 4 weeks after their third dose of pentavalent vaccine. In addition to collecting information on illnesses, data were collected for subsequent hospitalizations and deaths through weekly contact with the infant’s family. The information was entered electronically at interview. An International Advisory Group provided technical input on the protocol and statistical plan, reviewed study progress during several field visits, and reviewed preliminary study findings. In addition to analysing the project as a cohort study, secondary analyses will also include a self-controlled case series. Implementation of the study occurred between September 2014 and May 2016. In total over 30 000 infants were enrolled and followed weekly until 4 weeks after the third dose of vaccine. Data collection is complete and data analysis is ongoing, with plans for publication.

GACVS acknowledged the quality of the study implementation, the completeness of the follow-up and the thorough and timely data collection system. The primary analysis showed no safety concerns. This study will help to better characterize factors associated with untoward events temporally related with vaccination early in life and will provide a robust empirical basis to illustrate the coincidental occurrence of serious AEFI and quantify the frequency with which those events can be expected.

Full report of GACVS meeting of 15-16 June 2016, published in the WHO Weekly Epidemiological Record of 15 July 2016