Bulletin of the World Health Organization

Long-running telemedicine networks delivering humanitarian services: experience, performance and scientific output

Richard Wootton a, Antoine Geissbuhler b, Kamal Jethwani c, Carrie Kovarik d, Donald A Person e, Anton Vladzymyrskyy f, Paolo Zanaboni a & Maria Zolfo g

a. Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North Norway, N-9038 Tromsø, Norway.
b. Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland.
c. Center for Connected Health, Boston, United States of America (USA).
d. Department of Dermatology, University of Pennsylvania, Philadelphia, USA.
e. Pacific Island Health Care Project, Tripler Army Medical Center, Honolulu, USA.
f. Donetsk National Medical University, Donetsk, Ukraine.
g. Clinical Sciences Department, Institute of Tropical Medicine, Antwerp, Belgium.

Correspondence to Richard Wootton (e-mail: r_wootton@pobox.com).

(Submitted: 17 November 2011 – Revised version received: 01 March 2012 – Accepted: 01 March 2012.)

Bulletin of the World Health Organization 2012;90:341-347D. doi: 10.2471/BLT.11.099143

Introduction

Telemedicine (i.e. medicine practised at a distance) has been used to improve health care delivery in a wide range of applications. To date, most of the work has taken place in industrialized countries and there is relatively little experience in the developing world.1 Telemedicine’s fundamental benefit is in improving access to care, and in the developing world such access is often poor.1 Thus, telemedicine may provide a useful way to reduce inequities and strengthen health systems in developing countries.

In 2005, the World Health Organization (WHO) established a global observatory for e-health to monitor the development of information and communications technologies (ICT) for health care – including telemedicine – and to provide reliable information and guidance on best practices, policies and standards. According to a recent survey, telemedicine has progressed far less in lower-income countries than in high-income countries both in terms of the proportion of countries with established services and the proportion offering pilot telemedicine services.2 Nonetheless, several telemedicine networks around the world deliver humanitarian services on a routine basis, many to low-income countries. These networks provide tele-consultations for physicians and other health professionals needing advice about the clinical management of difficult cases, and some also provide education. Store-and-forward methods (e.g. e-mail) are often used for communication because they are generally cheaper and more convenient, but real-time methods (e.g. video links) are also used when required. Telemedicine networks delivering humanitarian services may be of interest to decision-makers considering wider implementation. Existing networks employ different organizational models and provide different kinds of services, and what represents best practice is unclear. Furthermore, data about network activities and performance are lacking.

To ensure effective and appropriate use of telemedicine in resource-limited settings, implementation must be guided by more and better evidence.3 The objective of the present paper is to summarize the experience gained so far with long-running telemedicine networks delivering humanitarian services; it looks at general information, network performance and scientific output. Long-running networks were selected for study because lack of programme sustainability is a commonly reported problem in telemedicine.

Methods

The work was conducted in three stages: (i) identifying relevant telemedicine networks; (ii) collecting information about their activities, and (iii) summarizing the resulting data.

Network selection

Long-running telemedicine networks delivering humanitarian services were identified. In this context, “long-running” was defined as having existed for 5 years or more. “Telemedicine” was defined as clinical and educational work at a distance. “Humanitarian services” were defined as actions designed to save lives, alleviate suffering and maintain human dignity delivered unconditionally (i.e. without seeking payment from their recipients). These networks were initially identified by one of the authors based on personal knowledge of the field.

A contact person at each telemedicine network was approached and asked if they knew of any other long-running telemedicine networks. This produced a list of nine networks, all of which were contacted. Two of the nine telemedicine networks were excluded from further consideration: one because it had apparently ceased operating when contacted and another because it never replied. The seven long-running telemedicine networks covered by this paper are shown in Table 1. Respondents, who were always the network coordinators, were invited to co-author this paper in recognition of their contribution to data collection, subsequent data analysis and the preparation of this paper.

Data collection

In September 2011, a questionnaire was sent to each contact person to collect basic information about the networks. This was followed in October 2011 by a second questionnaire designed to collect more detailed information regarding network performance and the evidence base. Each questionnaire contained 20 questions based on the framework for network performance evaluation proposed by Wootton et al.4 This framework takes into account the perspectives of the three main user groups – requesters (i.e. physicians requesting advice on the management of difficult cases), coordinators (i.e. network managers) and experts (i.e. those who respond to the electronic referrals) – and it identifies five performance measures of relevance to each of the three user groups. A third questionnaire was sent in February 2012 to obtain information about governance, liability and confidentiality.

Data summary

From the responses to the questionnaire, a list of the scientific papers describing the work of each network was compiled. Only papers listed in Medline and dealing with telemedicine services were included in the analysis. To summarize their contents, one author extracted information from each published paper on the following: (i) study design; (ii) effectiveness; (iii) quality; (iv) economics; (v) provision of access to care, and (vi) sustainability.

A second author, working independently, then graded each point. Study design was assessed using the nine-point scale traditionally employed to measure the strength of the evidence.5 For convenience, the nine levels were converted into a quality score (1 = poor, 2 = average, 3 = good), where levels I–III were considered good, IV–VI average, and VII–IX poor. The other five topics were graded subjectively and assigned a quality score based on the amount of information provided about them (0 = none, 1 = poor, 2 = average, 3 = good). Finally, the results were reviewed by all authors and changed by consensus where necessary. Hence, the papers from each network were reviewed by all the other networks, and this increased the consistency of the results.

Results

General information

The seven networks studied had been in operation from 5 to 15 years (median: 11 years), as shown in Table 2, available at: http://www.who.int/bulletin/volumes/90/5/11-099143. All the networks provide clinical tele-consultations for humanitarian purposes, and five of them were also involved in some form of education.

All networks used store-and-forward (asynchronous) methods for delivering tele-consultations, which confirms that real-time techniques are probably not appropriate in the present context. In addition, one telemedicine network used videoconferencing for consultations pertaining to trauma cases, which usually require an immediate response. Four of the networks offered tele-consultations in all clinical specialties, whereas the other three networks focused on delivering specialist services in areas such as traumatology, orthopaedics, neurosurgery, dermatology and management of patients with human immunodeficiency virus (HIV) infection or acquired immunodeficiency syndrome (AIDS). The source of funding for equipment acquisition and maintenance varied between telemedicine networks but was generally a mix of charitable funds and national or local support.

Four of the five networks delivering tele-education provided detailed information about their activities. Three networks delivered tele-education via asynchronous methods such as computer-based learning or web-based discussion forums. Two networks also used synchronous tele-education delivered through videoconferencing. Educational activities were offered by all the telemedicine networks delivering tele-consultations in specialty areas. Consistent with the clinical activities, tele-education was offered for trauma, dermatology and HIV/AIDS case management.

Organizational models were investigated through questions regarding the three main user groups. The number of requesters gives an idea of the size of each network and ranged from 10 to over 500. The number of requesters was loosely associated with the range of clinical services provided but not with the duration of network operation. The number of referring sites ranged from 4 to 399, and the number of countries ranged from 1 to 58. The number of sites and countries where requesters were based was roughly proportional to the number of requesters. Methods of requester accreditation – the licensing necessary to perform a clinical consultation – were different between the seven telemedicine networks, as detailed in Table 2. Three of the networks had a formal requester accreditation process; others used a more informal mechanism, consisting, for example, of personally knowing the physicians involved. One network claimed to have no requester accreditation process at all.

The smallest network had a total of 15 experts and the largest had 513. The number of sites where these were located varied greatly, from 1 to 502, and the number of countries ranged from 1 to 22. Differences in organizational models were evidenced by the large differences in the number of physicians using the telemedicine networks. One network indicated that not all requesters and experts were active because of constant staff turnover. This was not surprising, especially for big networks registering hundreds of physicians. All seven telemedicine networks had experts based in other industrialized countries. Two networks also had experts located in the countries where the requesters were based, and one network had experts from other developing countries. All the telemedicine networks had a process for accrediting the experts, as detailed in Table 2. Methods of accreditation included state licensure, formal credentials, clinical experience and training. Finally, in six telemedicine networks the experts were working as volunteers. Only two networks paid experts for the time they spent delivering tele-consultations.

The management of requests and the selection of the experts responsible for answering them were done by a coordinator in six of the seven networks. Thus, the coordinator appears to be a key element of the networks’ organizational model. In one network, this activity was performed entirely by the requesters themselves, whereas in another network the requesters were supported in the process by a coordinator. The coordinators and the experts were funded differently; in three of the seven networks coordinators were volunteers, whereas in the other four networks they were paid for their time.

Network performance

Network activity ranged from 50–400 tele-consultations per year. In 2010, the networks managed an average of 209 cases. In 2006–2011, three networks showed a positive trend over the 5-year period, three showed a negative trend, and one showed stable activity. On average, network activity increased by about 10% each year.

The average time to first reply to a request is an important performance measure affecting the value of the clinical tele-consultations to the referrer. Estimates from the last 12 months of operation showed that most networks took an average of 24 hours (range: 5.6–72 hours) to provide a first reply to a request. Unanswered replies to requests in 2010 were very few, particularly compared with network activity. Moreover, some requests could not be processed; in some cases, for instance, inadequate images were submitted and the referrer was unable to provide satisfactory alternatives. Overall, almost all requests were answered promptly, i.e. within 48 hours. Another performance measure is the possibility of a dialogue between requesters and experts. This feature was available for all seven telemedicine networks. On the other hand, the experts were not always informed of individual patient outcomes. In 2010, the number of cases managed by a network whose individual outcomes were fed back to experts varied from none in some networks to all in others.

Little quantitative data were available on the educational activities conducted by the telemedicine networks since their establishment. Data from the RAFT [Réseau en Afrique Francophone pour la Télémédecine] network in sub-Saharan Africa showed an increase in the number of hours of tele-education delivered each year. The telemedicine network of the Institute of Tropical Medicine (ITM) in Antwerp, Belgium, offered online learning and web-based discussion forums, but contact hours could not be quantified.

Scientific output

By the time of the survey, the seven networks had published a total of 59 papers; 44 that dealt with telemedicine and that were indexed in Medline were included in the present study.6–49 The scientific output represented by the 44 papers is summarized in Table 3. Information about study design was available for all papers. In particular, 29 papers represented non-controlled clinical series, with the number of patients observed ranging from a dozen to about 2000. The remaining 15 papers were anecdotes or case reports. Thus, all papers provided evidence classified as poor in strength. Evidence pertaining to programme sustainability and improved access to care was provided by more than half of the papers. One fourth of them also covered quality and effectiveness, while only a few provided any evidence on network economics. Overall, the quality of the scientific output was poor to average.

Other factors

Other factors relevant to the operation of each telemedicine network are summarized in Table 4, available at: http://www.who.int/bulletin/volumes/90/5/11-099143. Although governance varied across the seven networks as a function of differences in organizational models and contexts, medical liability and patient confidentiality were handled in similar ways. The factors concerning sustainability were generally different between networks, and included institutional anchoring, organizational models, technical and clinical solutions, clinical quality and benefits to patients, exit strategy, technology and funding. Uncertainties in funding were a common risk factor. Others had to do with the availability of coordinators, the training of experts, a lack of infrastructure and equipment and similar problems. Questionnaire respondents gave their views about the future plans of their telemedicine networks. These were similar across networks and had to do primarily with expansion to other countries and with the engagement of additional experts. Collaboration with other telemedicine networks was also listed as a feasible and useful plan for the future.

Discussion

The present study summarizes the experience gained to date with existing telemedicine networks that deliver humanitarian services. All of the seven well established, long-standing networks studied provided reasonable evidence that they were improving access to care in the developing world. However, the overall quality of the scientific output emanating from these networks is still rather weak. This applies to study design and to the evaluation of other important parameters, including effectiveness, service quality and economics. Stronger evidence is therefore needed to increase the appropriate use, scale and impact of telemedicine in resource-limited settings.50 This need for stronger evidence underscores that more and better evaluations need to be conducted.3 Given their size and relative success, long-running telemedicine networks should be the subject of controlled evaluations in future.

Another relevant finding from the study pertains to network performance. Measuring the performance of a telemedicine network is essential for understanding whether the network is working as intended or having the desired effect. By adapting a recently developed framework for network performance evaluation,4 we documented the seven telemedicine networks’ performance and the services they provided. Differences in services and performance could be explained by different organizational models.

Notwithstanding the use of different organizational models, clinical case load was strikingly similar across networks: all seven networks were providing only a few hundred tele-consultations annually. This activity level may stem from the fact that the networks are run by a single individual or a small number of committed enthusiasts or “clinical champions”. Since the present networks collectively appear to meet only a tiny fraction of the potential demand from the developing world,51 one may wonder why their activity levels are not increasing rapidly. Although the reasons could be many, small-scale organizational models may be one. Future work might therefore be directed at investigating new organizational models that would facilitate large-scale network operation. Improved collaboration between existing networks may prove beneficial as well, since it would attenuate the lack of resources reported by some networks and improve sustainability.

The findings of the present study have two main implications. First, telemedicine networks delivering humanitarian services appear to be sustainable – at least as operated to date – and they deliver clinically useful services. Second, the evidence summarized in this paper, albeit weak, may be useful to decision-makers. It may, for instance, encourage ministries of health in developing countries to establish, support or join similar telemedicine networks.

The present study has several limitations. For one thing, the list of networks studied may not be exhaustive; other long-running telemedicine networks around the world may also be delivering humanitarian services. However, we believe that we have covered the main active networks. Furthermore, the study was not a systematic review and the assessment of the quality of the scientific studies emanating from the networks was necessarily subjective. Moreover, we examined only successful networks (and arguably, experience from unsuccessful networks may be equally informative). Finally, the experience of the telemedicine networks was reviewed by people responding on behalf of the networks and may reflect reporting bias.

The present study emphasizes the need to generate stronger evidence and more and better evaluations of telemedicine networks and their effectiveness in improving outcomes and access to health care. Future research should address these topics. Nonetheless, the present study provides reasonable grounds for supporting the future expansion of telemedicine networks offering humanitarian services in developing countries.

Acknowledgements

We are grateful to our colleagues in the various networks for their willingness to share the data about network operation.

Competing interests:

None declared.

References

Share
Email Twitter Facebook Google Delicious LinkedIn
More...