Electronic monitoring of treatment adherence and validation of alternative adherence measures in tuberculosis patients: a pilot study
Jossy van den Boogaard a, Ramsey A Lyimo b, Martin J Boeree a, Gibson S Kibiki b & Rob E Aarnoutse c
a. Radboud University Nijmegen Medical Centre, University Centre for Chronic Diseases Dekkerswald, Nijmeegsebaan 31, 6560 AB, Groesbeek, Netherlands.
b. Kilimanjaro Clinical Research Institute, Kilimanjaro Christian Medical Centre, Moshi, Tanzania.
c. Department of Clinical Pharmacy, Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands.
Correspondence to Jossy van den Boogaard (e-mail: firstname.lastname@example.org).
(Submitted: 25 January 2011 – Revised version received: 04 May 2011 – Accepted: 04 May 2011 – Published online: 20 May 2011.)
Bulletin of the World Health Organization 2011;89:632-639. doi: 10.2471/BLT.11.086462
Non-adherence to treatment for tuberculosis is a major barrier to global tuberculosis control. To ensure adherence to treatment by tuberculosis patients, the direct observation of treatment by a trained supervisor is recommended.1 Initially, such directly observed treatment (DOT) was provided in health-care facilities only, but because of workload demands, several countries have started to involve community members in the provision of DOT.2 Studies have shown that community-based DOT is a cost-effective strategy that yields treatment outcomes similar to those obtained with facility-based DOT.3–6 However, community-based DOT has been criticized for being beyond the control of health-care providers and hence conducive to self-administered (unsupervised) treatment and non-adherence.6,7
The actual degree of adherence by patients on community-based DOT has not yet been assessed. Measuring adherence is difficult because most available direct and indirect measures have limitations. Direct adherence measures, such as tests to measure drug levels in plasma or urine, cover brief medication intake periods only. Indirect measures, such as pill counts and self-report questionnaires, cover longer periods but assume rather than prove the patient’s actual medication intake.8
A sophisticated indirect adherence measure is the Medication Event Monitoring System (MEMS). MEMS medication bottles contain a microelectronic chip that registers the date and time of every bottle opening. Assuming that bottle openings represent medication intake, MEMS provides a detailed profile of the patient’s adherence behaviour. MEMS is currently regarded as the gold standard to measure adherence.8 It has been used as such in a wide range of studies on adherence to antihypertensive and lipid-lowering therapy,9,10 therapy for neurologic and psychiatric disorders,11,12 post-transplantation immunosuppressive therapy13,14 and antiretroviral therapy.15–17 Few studies report on the use of MEMS to monitor adherence to tuberculosis treatment.18–21 Because of the high cost involved, MEMS is not feasible for use in routine practice in most settings with a high tuberculosis burden but could be used as a reference standard to validate simple and affordable measures that can be used in patients on community-based DOT.8,21
In this pilot study, we used MEMS to: (i) describe adherence rates among Tanzanian tuberculosis patients on community-based DOT and (ii) determine the validity of several direct and indirect adherence measures of potential use in resource-limited settings.
The study was conducted in the Kilimanjaro region of the United Republic of Tanzania, where the annual tuberculosis case notification rate is 178 per 100 000 population.6 The national tuberculosis programme empowers patients to choose between community- and facility-based DOT. Most patients opt for community-based DOT and those on facility-based DOT are mostly inpatients.6 Patients on community-based DOT have to select a treatment supporter from their community (usually a relative or spouse) who is instructed on how to provide daily DOT at home. Patients on community-based DOT are supposed to collect their medication once a week in the first two months of treatment and once every two weeks in the remaining four months. They should return medication blisters for pill counts and their clinic attendance is registered.22
Study design and procedures
This was a longitudinal pilot study in which treatment adherence among 50 patients on community-based DOT was monitored by MEMS throughout treatment. MEMS was used as a gold standard to validate several other adherence measures (single and in combinations) in this patient group. The adherence measures were selected for their applicability in the Tanzanian setting and included an isoniazid (INH) urine test, a urine colour test for rifampicin, the Brief Medication Questionnaire (BMQ), the Morisky scale, an adapted version of the AIDS Clinical Trials Group (ACTG) adherence questionnaire, pill counts and clinic attendance for medication refills.
The participants were recruited between February and May 2010 from one rural hospital, one urban hospital and two urban health centres. Considering the pilot nature of our study, we chose to enrol 50 patients only. Adult outpatients who consecutively presented at one of the study sites with newly diagnosed tuberculosis and who had chosen community-based DOT were eligible to participate. Eligible patients were informed about the study procedures by trained clinic staff and asked to sign an informed consent form. They were told that their tuberculosis medication would be provided in a medication bottle with a microelectronic chip that registers every bottle opening. They were asked not to use the MEMS bottle for other medication, to open it only to take out medication, and to bring the bottle to every medication refill visit. Patients were informed that they would be under routine treatment and care.
Drug dispensing nurses were responsible for providing patients with medication. Medication refill visits were conducted in accordance with routine practice, except that the medication blisters were cut into pieces to make them fit in the MEMS bottle. The nurses registered the dates of the patients’ clinic visits for medication refills and the number of tablets remaining at each visit. During the visits in weeks 4, 8, 12 and 16, the procedures deviated from routine practice. At these visits, patients submitted a sample of urine and were asked to fill out the BMQ and Morisky scale (in weeks 4, 8 and 12) or the ACTG adherence questionnaire (in week 16). At completion of treatment, the patients filled out a questionnaire about their experience with the use of MEMS bottles.
Treatment adherence measures
Medication Event Monitoring System
MEMS medication bottles (250-ml containers with a 38-mm MEMS 6 TrackCap, AARDEX Ltd, Sion, Switzerland) were used by all participants throughout treatment. MEMS data were used to calculate adherence rates (by dividing the number of days on which at least one bottle opening was registered by the total number of monitored days and multiplied by 100) and to differentiate between adherent and non-adherent patients for validation of the other adherence measures. For the latter purpose, commonly used adherence rate cut-off values of 100%, 95% and 80% were applied.8
Isoniazid urine test
The isoniazid urine test or IsoScreen test (GFC Diagnostics Ltd, Bicester, England) is based on the Arkansas method for the detection of INH in urine and supplied in a ready-to-use plastic testing device. The test was performed in accordance with the directions provided in the accompanying manual. The test result was negative when no colour change was observed after 5 minutes, positive when the colour changed to dark purple and equivocal when the colour turned green. Patients with at least one negative test were regarded as non-adherent.
Urine colour test
Prior to each INH urine test, urine colour was checked for the presence of rifampicin. Orange urine was scored as positive and yellow urine as negative. Patients with at least one negative test were categorized as non-adherent.
The Morisky scale is a self-report adherence measure with four questions about common barriers to adherence.23 We classified as non-adherent all patients who answered “yes” to at least one of the four questions in at least one of the three repeated tests.
Brief Medication Questionnaire
The BMQ consists of three sections (“screens”) with questions about adherence behaviour, the experienced effects of treatment and other factors that could affect adherence. The screens were scored as described elsewhere.24 Patients with a total score of 1 or more in at least one of the three repeated tests were classified as non-adherent.
AIDS Clinical Trials Group adherence questionnaire
The adapted ACTG adherence questionnaire was developed for patients infected with the human immunodeficiency virus (HIV) who participate in clinical trials25 and is available at www.ghdonline.org/uploads/ACTG_Adherence_Baseline_Questionnaire.pdf (last accessed: 12 May 2011). Our adapted version consisted of three multiple choice items corresponding to sections B (social support), C (possible reasons for non-adherence) and D (adherence behaviour) of the original baseline questionnaire. We scored any answer other than “never” to the questions in sections C and D or less than “somewhat satisfied” to the questions in section B as positive. A positive score was regarded as indicative of non-adherence.
Refill visits and pill counts
The patients’ clinic attendance for medication refills was registered and remaining tablets were counted at every refill visit. Patients who delayed at least once for a medication refill visit and those who had an incorrect number of tablets remaining at least once were classified as non-adherent.
MEMS data were analysed by using Powerview software (AARDEX Ltd, Sion, Switzerland). Periods of “pocket dosing” (i.e. taking out medication for later use) that were identified by the MEMS use questionnaire were excluded from the analysis as non-monitored periods. Statistical analysis was performed in SPSS version 16.0 (SPSS Inc., Chicago, United States of America). Means are presented with standard deviation (SD) and medians with interquartile range (IQR). Means were compared by using the Student t-test. The sensitivity, specificity, positive and negative predictive values and accuracy of single and combined adherence measures were calculated by using MEMS as the gold standard. For combined measures, non-adherent patients were those who were classified as non-adherent by at least one of the single measures in the combination.
The study was approved by the institutional review board of the Kilimanjaro Christian Medical Centre (Moshi, United Republic of Tanzania) and the National Institute for Medical Research (Dar es Salaam, United Republic of Tanzania).
Patient characteristics and treatment outcomes
We enrolled 31 male and 19 female patients. Their characteristics are summarized in Table 1. Six of the 22 patients who were co-infected with HIV used antiretroviral medication and 14 were on co-trimoxazole prophylaxis. Although all patients were on community-based DOT, seven had no formal treatment supporter.
Table 1. Patient characteristics and treatment outcomes among participants (n = 50) in a study of adherence to tuberculosis treatment, United Republic of Tanzania, 2010
Thirty-seven patients successfully completed treatment. Six patients died; all were HIV-positive. Three patients defaulted and four patients dropped out of the study (three were transferred to another region and one developed jaundice and his treatment had to be interrupted).
Treatment adherence according to MEMS
No MEMS data were available for three patients (one defaulter and two who died) because the medication bottle was not returned. For the other 47 patients, a total of 6871 treatment days were monitored by MEMS. On 194 monitored days the MEMS bottle was not opened; the median per patient was 2 days (IQR: 0–5). The mean adherence rate was 96.3% (SD: 7.7) and did not differ significantly between patients with and without a treatment supporter: 96.2% (SD: 8.2) and 97.1% (SD: 3.1), respectively (P = 0.79).
Adherences was less than 100% in 70% of all patients; less than 95% in 21% of them, and less than 80% in 2% (Table 2). Among the patients who completed the six-month treatment course, adherence was less than 100% in 73%, less than 95% in 19% and less than 80% in none, respectively.
Table 2. Median monitored treatment days and adherence rates, and proportions of patients who were less than 100%, 95% and 80% adherent,a as assessed by the Medication Event Monitoring System (MEMS), United Republic of Tanzania, 2010
Monthly adherence rates were fairly constant. In the group of patients who completed treatment, the median monthly adherence rate was 100% and the mean monthly adherence rate varied between 95.4% (SD: 7.3) in month 6 and 98.5% (SD: 2.7) in month 3.
Validation of adherence measures
For the validation of the alternative adherence measures only patients who completed the six-month treatment course (n = 37) were included. As shown in Fig. 1, the proportions of non-adherent patients identified by the different measures varied widely. Table 3 shows the sensitivity, specificity, positive and negative predictive value and accuracy of the adherence measures in terms of their ability to differentiate between adherent and non-adherent patients. The ACTG adherence questionnaire and urine colour test had the highest sensitivities but lowest specificities. The Morisky scale and refill visits had the highest specificities but lowest sensitivities. The sensitivities of most measures improved when the cut-off value to differentiate between adherent and non-adherent patients was lowered from 100% to 95% adherence, but the specificities dropped. The positive and negative predictive values were almost reversed by changing the cut-off value from 100% to 95%, reflecting the large difference in the proportions of patients categorized as non-adherent by using the former versus the latter cut-off value. A cut-off value of 80% could not be applied because none of the patients who completed treatment was less than 80% adherent.
Fig. 1. Non-adherence to tuberculosis treatment among patients who completed treatment (n = 37) as assessed by different adherence measures, United Republic of Tanzania, 2010
ACTG, AIDS Clinical Trials Group; BMQ, Brief Medication Questionnaire; INH, isoniazid; MEMS, Medication Event Monitoring System.
Table 3. Sensitivity, specificity, positive and negative predictive values, and accuracy of individual treatment adherence measures, alone and in combination, in a study of adherence to tuberculosis treatment, United Republic of Tanzania, 2010
The combination of pill counts and refill visits that is used in routine practice had moderate sensitivity and specificity. Its sensitivity and negative predictive value for the identification of patients who were less than 95% adherent improved by adding any of the other adherence measures except the Morisky scale (Table 3).
Patients’ experience with MEMS use
The questionnaire about the use of MEMS, which was filled out by the 37 patients who completed treatment, revealed that only one patient had correctly understood the purpose of MEMS despite verbal and written information at the onset of the study. Twenty-five patients (68%) stated that the white, bulky appearance of the MEMS bottle reminded them to take the medication. However, the other patients said that the use of MEMS had not influenced their adherence behaviour. Mean adherence rates did not differ between these two groups: 97.5% (SD: 3.0) and 97.3% (SD: 3.2), respectively (P = 0.86).
Eight patients occasionally opened the MEMS bottle to take out medication for later use (resulting in a total of 42 non-monitored treatment days). This usually occurred when patients did not want to take the bottle along on travel occasions.
This is the first study in which MEMS was used to assess treatment adherence rates in patients on community-based DOT over the full six-month tuberculosis treatment course. We observed high adherence rates in our pilot study of 50 Tanzanian patients. Almost 80% of the patients were more than 95% adherent and only one patient was less than 80% adherent. These findings do not confirm the concern that patients on community-based DOT are prone to become non-adherent, even though the study did reveal that some patients (i.e. those without a formal treatment supporter) turn community-based DOT into unsupervised treatment.
The participants’ demographic characteristics, such as the ratio of males to females and their treatment outcomes, were comparable to those of the general tuberculosis patient population in the Kilimanjaro region.6 This suggests that we studied a regionally representative patient sample. However, the adherence rates of our patients could have been biased by their participation in the study. Although we tried to deviate as little as possible from routine practice, the repeated adherence questionnaires and urine tests certainly made participants aware of our interest in their adherence behaviour. Two thirds of the patients felt that their adherence behaviour had been influenced by the use of MEMS, but their average adherence rate did not differ from those observed among patients who stated that MEMS had not influenced their behaviour. Findings from other studies suggest that when MEMS is used over long periods, its “interventional effect” disappears.26,27 Since our sample size was small, larger population studies should be conducted to assess the real impact of community-based DOT with and without formal treatment supporter on adherence rates. We did not aim to validate the concept of community-based DOT; studies for this purpose should have a different design.
The main objective of our study was to use MEMS as a reference standard to calculate the validity of several adherence measures whose use is feasible in patients on community-based DOT in resource-limited settings. The high adherence rates in the study population forced us to apply high adherence rate cut-off values to calculate the validity and reliability of the adherence measures. This resulted in wide gaps between the sensitivities and specificities of the measures.8 Combinations of measures were found to be more accurate than single measures in identifying as many true non-adherent patients as possible (reflected in high sensitivities and negative predictive values). The sensitivity and negative predictive value of the routinely used combination of pill counts and clinic attendance for medication refills improved substantially by adding a simple and cheap measure such as the ACTG adherence questionnaire, particularly at an adherence rate cut-off value of 95%.
The rifampicin urine colour test classified more patients as non-adherent than the INH urine test. Since the orange urine colouration caused by rifampicin is of short duration and may be absent altogether,28 it is likely that the urine colour test misclassified some patients with yellow urine as non-adherent. Such misclassifications are difficult to confirm in a study population with high adherence rates.
The adapted ACTG adherence questionnaire yielded more favourable responses than the other self-report measures. Differences in wording in the questionnaires may account for this.8 While patients had to answer either “yes” or “no” to the questions in the Morisky scale, they could answer “often”, “sometimes”, “rarely”, or “never” to comparable questions in the ACTG adherence questionnaire. This wider range of choice options may have evoked more honest replies. The ACTG adherence questionnaire (and to a lesser extent the BMQ) has the added advantage of disclosing factors that cause non-adherence in the individual patient. These factors could be used to design tailored interventions for promoting adherence among non-adherent patients on community-based DOT. We therefore suggest using the triple combination consisting of the ACTG adherence questionnaire, refill visits and pill counts to monitor treatment adherence by Tanzanian patients on community-based DOT. If the results are interpreted carefully, the combination seems valid and its use in routine practice appears feasible.
In conclusion, this study revealed high levels of treatment adherence by Tanzanian tuberculosis patients on community-based DOT. Although caution should be exercised in interpreting the results because this is a pilot study, the findings support the recommendation of using community-based DOT as an alternative to facility-based DOT in settings with overburdened health-care facilities. To monitor adherence among patients on community-based DOT in resource-limited settings where electronic monitoring is not feasible, combinations of simple and affordable adherence measures can be used. Supplementing pill counts and clinic attendance with a self-report measure such as the ACTG adherence questionnaire will help to identify potentially non-adherent patients who could benefit from tailored adherence-promoting interventions. Further studies in larger patient populations are needed to assess the adherence rates of patients on community-based DOT and to confirm the validity of simple and affordable adherence measures.
The KNCV Tuberculosis Foundation (the Netherlands) provided financial support for the study but was not involved in the design or conduct of the study, or in the decision to publish the results.
- An expanded DOTS framework for effective tuberculosis control (WHO/CDS/TB/2002.297). Geneva: World Health Organization; 2002.
- Maher D. The role of the community in the control of tuberculosis. Tuberculosis (Edinb) 2003; 83: 177-82 doi: 10.1016/S1472-9792(02)00066-5 pmid: 12758209.
- Lwilla F, Schellenberg D, Masanja H, Acosta C, Galindo C, Aponte J, et al., et al. Evaluation of efficacy of community-based vs. institutional-based direct observed short-course treatment for the control of tuberculosis in Kilombero district, Tanzania. Trop Med Int Health 2003; 8: 204-10 doi: 10.1046/j.1365-3156.2003.00999.x pmid: 12631309.
- Wandwalo E, Kapalata N, Egwaga S, Morkve O. Effectiveness of community-based directly observed treatment for tuberculosis in an urban setting in Tanzania: a randomised controlled trial. Int J Tuberc Lung Dis 2004; 8: 1248-54 pmid: 15527158.
- Floyd K, Skeva J, Nyirenda T, Gausi F, Salaniponi F. Cost and cost-effectiveness of increased community and primary care facility involvement in tuberculosis care in Lilongwe District, Malawi. Int J Tuberc Lung Dis 2003; 7: S29-37 pmid: 12971652.
- van den Boogaard J, Lyimo R, Irongo CF, Boeree MJ, Schaalma H, Aarnoutse RE, et al., et al. Community vs. facility-based directly observed treatment for tuberculosis in Tanzania’s Kilimanjaro Region. Int J Tuberc Lung Dis 2009; 13: 1524-9 pmid: 19919771.
- Frieden TR, Sbarbaro JA. Family observation of antituberculosis treatment. Lancet 2006; 367: 2055-6, author reply 2055-6 doi: 10.1016/S0140-6736(06)68913-7 pmid: 16798377.
- Farmer KC. Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clin Ther 1999; 21: 1074-90, discussion 1073 doi: 10.1016/S0149-2918(99)80026-5 pmid: 10440628.
- Zeller A, Ramseier E, Teagtmeyer A, Battegay E. Patients’ self-reported adherence to cardiovascular medication using electronic monitors as comparators. Hypertens Res 2008; 31: 2037-43 doi: 10.1291/hypres.31.2037 pmid: 19098375.
- Schwed A, Fallab CL, Burnier M, Waeber B, Kappenberger L, Burnand B, et al., et al. Electronic monitoring of compliance to lipid-lowering therapy in clinical practice. J Clin Pharmacol 1999; 39: 402-9 doi: 10.1177/00912709922007976 pmid: 10197299.
- Grosset KA, Bone I, Grosset DG. Suboptimal medication adherence in Parkinson’s disease. Mov Disord 2005; 20: 1502-7 doi: 10.1002/mds.20602 pmid: 16037924.
- Rivers PH, Ardagh-Walter N, Wright EC. Measurement of anticonvulsant adherence behaviour in the community using a Medication Events Monitoring System (MEMS) Health Care Anal 1998; 6: 308-16 doi: 10.1007/BF02678367 pmid: 10345937.
- Denhaerynck K, Schäfer-Keller P, Young J, Steiger J, Bock A, De Geest S. Examining assumptions regarding valid electronic monitoring of medication therapy: development of a validation framework and its application on a European sample of kidney transplant patients. BMC Med Res Methodol 2008; 8: 5- doi: 10.1186/1471-2288-8-5 pmid: 18284675.
- Russell CL, Owens S, Hamburger KQ, Thompson DA, Leach RR, Cetingok M, et al., et al. Medication adherence and older renal transplant patients’ perceptions of electronic medication monitoring. J Gerontol Nurs 2009; 35: 17-21 doi: 10.3928/00989134-20090903-06 pmid: 19772223.
- Melbourne KM, Geletko SM, Brown SL, Willey-Lessne C, Chase S, Fisher A. Medication adherence in patients with HIV infection: a comparison of two measurement methods. AIDS Read 1999; 9: 329-38 pmid: 12737122.
- Hugen PW, Langebeek N, Burger DM, Zomer B, van Leusen R, Schuurman R, et al., et al. Assessment of adherence to HIV protease inhibitors: comparison and combination of various methods, including MEMS (electronic monitoring), patient and nurse report, and therapeutic drug monitoring. J Acquir Immune Defic Syndr 2002; 30: 324-34 pmid: 12131570.
- Arnsten JH, Demas PA, Farzadegan H, Grant RW, Gourevitch MN, Chang CJ, et al., et al. Antiretroviral therapy adherence and viral suppression in HIV-infected drug users: comparison of self-report and electronic monitoring. Clin Infect Dis 2001; 33: 1417-23 doi: 10.1086/323201 pmid: 11550118.
- Fallab-Stubi CL, Zellweger JP, Sauty A, Uldry C, Iorillo D, Burnier M. Electronic monitoring of adherence to treatment in the preventive chemotherapy of tuberculosis. Int J Tuberc Lung Dis 1998; 2: 525-30 pmid: 9661817.
- Starr M, Sawyer S, Carlin J, Powell C, Newman R, Johnson P. A novel approach to monitoring adherence to preventive therapy for tuberculosis in adolescence. J Paediatr Child Health 1999; 35: 350-4 doi: 10.1046/j.1440-1754.1999.00371.x pmid: 10457290.
- Ailinger RL, Black PL, Lima-Garcia N. Use of electronic monitoring in clinical nursing research. Clin Nurs Res 2008; 17: 89-97 doi: 10.1177/1054773808316941 pmid: 18387881.
- Ruslami R, van Crevel R, van de Berge E, Alisjahbana B, Aarnoutse RE. A step-wise approach to find a valid and feasible method to detect non-adherence to tuberculosis drugs. Southeast Asian J Trop Med Public Health 2008; 39: 1083-7 pmid: 19062699.
- Manual of the National Tuberculosis and Leprosy Programme in Tanzania. Fifth edition. Dar es Salaam: Ministry of Health; 2010.
- Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care 1986; 24: 67-74 doi: 10.1097/00005650-198601000-00007 pmid: 3945130.
- Svarstad BL, Chewning BA, Sleath BL, Claesson C. The Brief Medication Questionnaire: a tool for screening patient adherence and barriers to adherence. Patient Educ Couns 1999; 37: 113-24 doi: 10.1016/S0738-3991(98)00107-4 pmid: 14528539.
- Chesney MA, Ickovics JR, Chambers DB, Gifford AL, Neidig J, Zwickl B, et al., et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG). AIDS Care 2000; 12: 255-66 doi: 10.1080/09540120050042891 pmid: 10928201.
- Bova CA, Fennie KP, Knafl GJ, Dieckhaus KD, Watrous E, Williams AB. Use of electronic monitoring devices to measure antiretroviral adherence: practical considerations. AIDS Behav 2005; 9: 103-10 doi: 10.1007/s10461-005-1685-0 pmid: 15812617.
- Wagner GJ, Ghosh-Dastidar B. Electronic monitoring: adherence assessment or intervention? HIV Clin Trials 2002; 3: 45-51 doi: 10.1310/XGXU-FUDK-A9QT-MPTF pmid: 11819185.
- Whitfield R, Cope GF. Point-of-care test to monitor adherence to anti-tuberculous treatment. Ann Clin Biochem 2004; 41: 411-3 doi: 10.1258/0004563041731637 pmid: 15333195.