Bulletin of the World Health Organization

Use of text messages to communicate clinical recommendations to health workers in rural China: a cluster-randomized trial

Yaolong Chen a, Kehu Yang a, Tao Jing b, Jinhui Tian a, Xiping Shen a, Changchun Xie c, Bin Ma a, Yali Liu a, Liang Yao a & Xiaoyuan Cao d

a. Evidence-Based Medicine Centre, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, Gansu Province, China.
b. Pathogenic Biology Institute, Lanzhou University, Gansu Province, China.
c. Department of Environmental Health, University of Cincinnati, Ohio, United States of America (USA).
d. Health Bureau of Gansu Province, Gansu Province, China.

Correspondence to Kehu Yang (e-mail: yangkh@lzu.edu.cn).

(Submitted: 04 July 2013 – Revised version received: 05 November 2013 – Accepted: 12 November 2013 – Published online: 25 March 2014.)

Bulletin of the World Health Organization 2014;92:474-481. doi: http://dx.doi.org/10.2471/BLT.13.127076


Health workers in rural China do not receive systematic, qualified medical education and training1,2 because, unlike their urban counterparts, they face constraints such as inadequacies in transport and funding and they are largely unaware of the need for education.3,4 Gansu Province has 16 853 health workers (including family physicians, nurses, public health practitioners, pharmacists, midwives and laboratory technicians) in 1333 township health centres, distributed across 14 regions.5 Most of the health centres are located in remote mountainous areas, and thus providing continuing medical education to these health workers is a major challenge.6

Mobile text messages have been used to improve health outcomes in a wide range of contexts because of their low cost and convenience.7,8 For instance, text messages have been used in health programmes for smoking cessation,9 disease management1012 and weight reduction13 and to improve adherence to medication14 and attendance at health-care appointments.15 Since Chinese mobile phone users send high volumes of messages – 79 billion messages, equivalent to 73 per user – in September 2012 alone16 – we hypothesized that such messages could be useful for communicating medical information to rural health workers in China.

In general, text messages seem to be effective for communicating information in a health-care context and have been well accepted by users.17,18 Research also indicates that text messages could serve as a powerful tool for behaviour change,1921 both in developed and developing countries.22 However, so far research has focused almost exclusively on the sending of health messages to patients rather than to health workers, and on the use of messages as patient reminders rather than for the delivery of evidence-based information. In this study, we tested whether text messages sent to rural health workers containing evidence-based recommendations could improve knowledge and influence prescribing medical practice.


Study design, participants and recruitment

The study was undertaken in the Gansu province in north-western China from 17 October to 25 December 2011. It was designed as a “before” and “after” randomized controlled trial. The intervention group was sent text messages on the management of viral infections affecting the upper respiratory tract and otitis media, and the control group was given the same messages in the context of a regular continuing medical education programme. In preparation for the trial, we undertook several surveys and conducted two pilot studies in seven health centres in Gaolan county, Gansu province, between November 2009 and April 2011. Information on these pilot studies, which were conducted to choose the best content and delivery of the text messages and to conduct a power analysis for the trial, can be obtained from the corresponding author on request.

To be eligible for recruitment individuals had to be health workers in a township health centre in Gansu province. The term “health worker” was used broadly to include family physicians, nurses, public health practitioners, pharmacists, midwives and laboratory technicians. Only physicians could prescribe drugs, but other health workers were also sent text messages because of their potential influence on physician behaviour. In addition, the pilot studies showed that confining text messages to physicians made other health workers feel excluded. Health workers who did not own a mobile phone or whose mobile phone could not receive text messages were excluded from the trial.

Sample size

The power calculations were based on the results of two pilot studies and the formula outlined by Donner and Klar23 for cluster randomized trials with a binary study outcome. The analysis of variance estimator of the intra-cluster correlation coefficient was calculated as 0.15. We initially calculated the minimum sample size to be 76 health centres with a total of 742 participants, allowing for a 40% loss-to-follow-up. However, in light of higher rates of loss to follow-up in the pilot studies, we increased the number of health centres to 100 to improve statistical power.


We used the health centre as the unit of randomization. A cluster design was used to avoid biases arising from the possible conveyance of information by members of the intervention group to members of the control group if both were located at the same health centre. Randomization was done in two stages. First, with the help of the health administration department of Gansu province, we sent invitation letters to all 1333 health centres in Gansu province. By the deadline, 163 health centres had agreed to participate in our study. From these centres we randomly selected 100 for inclusion in the trial; we then used a computer-generated random sequence to select the clusters for intervention. To minimize the potential for selection bias, cluster allocation was masked from statisticians until the analyses were completed.

The intervention

For the main trial, we created 18 text messages. Of these messages, 16 contained evidence-based recommendations for the management of the infections affecting the upper respiratory tract and middle ear that are most common in rural Gansu – the common cold, influenza, pharyngitis, tonsillitis – and of otitis media, a frequent complication of upper respiratory infections. The recommendations were mainly sourced from Clinical evidence24 and the Cochrane Library.25,26 Senior physicians from the First Hospital of Lanzhou University revised the language of the recommendations to ensure that health workers in rural areas could understand the messages clearly. All text messages were within 280 Chinese characters in length, the maximum for most mobile phones in China.

The intervention was carried out from 15 November to 24 December 2011. A computer sent the text messages to the intervention group three times a week (on Tuesdays, Thursdays and Saturdays) at 20:30. The control group received the recommendations through the traditional method, a one-day training programme delivered by two senior physicians from the First Hospital of Lanzhou University, held on 3 December 2011.

Data collection

Health workers were interviewed by telephone and asked questions to test their knowledge of disease management – for the five selected acute respiratory conditions – before and after the intervention and the traditional training workshop. The difference between the intervention and control group in the percentage point change in average test score was the main study outcome. A secondary outcome was the difference between the intervention and control group, expressed in percentage points, in the average number of antibiotic and steroid prescriptions issued by family physicians.

Telephone surveys were conducted on 5 and 6 November 2011 (before the intervention) and on 24 and 25 December 2011 (after the intervention) using a computer-assisted telephone interviewing system based on random dialing. Participants were asked 10 multiple-choice questions on the appropriate treatment of the selected diseases and complications. All questions were scored as one point per correct response and zero points for an incorrect response. We assumed that scores reflected health workers’ ability to identify the appropriate action when confronted with a specific medical problem. Scores were averaged as a percentage of questions answered correctly at both the individual and cluster level. An additional questionnaire was administered to record participant satisfaction with both the intervention and the educational methods used in the control group.

To assess family physicians’ prescription practices, random sampling using a computer-generated randomization procedure was used to select 10 health centres in each cluster for the collection of prescriptions. Investigators then collected drug prescriptions for upper respiratory infections in these health centres from 1 December 2011 to 28 February 2012. As a comparator, they also obtained the prescriptions issued over the same period one year before the trial (i.e. from 1 December 2010 to 28 February 2011). Prescriptions for upper respiratory infections were chosen for the trial because: (i) viral infections affecting the upper respiratory tract are very common in rural China, especially during late autumn and winter; (ii) health workers at township health centres often inappropriately prescribe antibiotics and steroids for these viral infections.27,28

Statistical analysis

Analysis was by intention to treat. All statistical analyses were conducted using SAS software, version 9.2 (SAS Institute, Cary, USA). At the cluster level, we calculated the average knowledge score for each cluster (i.e. health centres) and used it as the outcome. An independent t-test was conducted to compare the difference in average scores between the intervention and control groups, with a 95% CI of the average score difference. At the individual level, a linear mixed model (mixed procedure in SAS) was performed to evaluate the intervention effect. The cluster was chosen as a random effect to account for the dependence of individuals within the same cluster. The model contained the study groups (intervention versus control), sex and baseline score as fixed effects. Missing values were entered by the cluster mean input method.29 Sensitivity analysis was performed by analysing the observed outcomes only. Statistical significance was defined as P < 0.05.

Ethics and consent

The trial was registered with the Chinese Clinical Trial Register on 15 August 2009 (registration number ChiCTR-TRC-09000488) and received approval by the Chinese Ethics Committee of Registering Clinical Trials (ChiECRCT-2012026). Informed consent was obtained via telephone survey and all calls were recorded automatically by the computer-assisted telephone interviewing system.


Of the 1333 health centres invited to participate in the trial, 163 health centres agreed, and of these 100 were chosen at random and allocated either to the intervention group (490 health workers at 52 health centres) or the control group (487 health workers at 48 health centres) (Fig. 1).

Fig. 1. Flowchart showing the selection of participants for study on the use of text messaging to communicate clinical recommendations to health workers, Gansu province, China, 2011
Fig. 1. Flowchart showing the selection of participants for study on the use of text messaging to communicate clinical recommendations to health workers, Gansu province, China, 2011

The first telephone survey to assess knowledge of the recommendations before the intervention was successfully conducted with 348 people in the intervention group, and 349 in the control group. The second telephone survey to assess knowledge after the intervention was successfully completed with 301 people in the intervention group, and 332 in the control group. An analysis of baseline characteristics showed no statistically significant differences between the two groups (Table 1).

After receiving text messages, the average score in the intervention group increased significantly more than in the control group, both at the cluster and the individual level (Table 2). In subgroup analyses, family physicians’ scores in the intervention group increased significantly more than scores in the control group, both individually and at the cluster level (Table 2).

In the intervention group, no change in the prescription of antibiotics was found; however, prescriptions for steroids fell by 21 percentage points (Table 3). In the control group, prescriptions for antibiotics and steroids increased by 17 and 11 percentage points, respectively.

During the follow-up survey on attitudes towards the text messages containing evidence-based recommendations, one third of the health workers in the intervention group reported that they frequently adopted the recommendations in their clinical decision-making and 95% wanted to continue receiving the text messages (Fig. 2).

Fig. 2. Attitudes towards text messaging as a means of communicating clinical recommendations to health workers,a Gansu province, China, 2011
Fig. 2. Attitudes towards text messaging as a means of communicating clinical recommendations to health workers,<sup>a</sup> Gansu province, China, 2011
a The number of respondents varies slightly for each question because some respondents failed to respond to all questions.
b The degree to which the recommendation is credible or evidence-based.


This study shows that compared with traditional methods of medical education, text messages are more effective in leading to a greater understanding of recommendations, especially for family physicians, a result that was shown by changes in prescribing practices.

Several reasons explain the success of text messages in transmitting medical information. First, for the majority of health workers, text messages were the only way they obtained the latest and best clinical knowledge. In our pilot studies, we found that continuing medical education in Gansu Province consisted primarily of training sessions hosted by higher-level health departments.6 However, due to constraints in time and resources, such training sessions happen infrequently, and only reach a small number of health workers throughout the province. Thus, 80% of family physicians relied on medical textbooks to guide the diagnosis and treatment of patients, most of which contained outdated and incorrect recommendations.5

Second, compared to textbooks and printed learning materials, text messages were easier to carry, retrieve and remember. Moreover, our pilot study showed that, of alternative means of communicating medical information, such as television, radio, newspapers, or blackboards in health centres, health workers ranked text messages as their preferred method.30

Third, text messages were tailored to the local disease context and edited on the basis of feedback to suit health workers’ clinical needs. The slight difference in the results at the individual and cluster level could be due to minimal texting between health workers in the same health centre.

Text messages delivered during the intervention were perhaps the first time that some health workers became aware of evidence-based recommendations, given limited opportunities for continued medical education. Yet research has shown that medical education and physicians’ knowledge of the latest recommendations can have a direct influence on the prescription of antibiotics.31 In our study, text messages may have prevented family physicians from prescribing antibiotics and steroids for viral infections affecting the upper respiratory tract. This is of critical importance, since the use of antibiotics has increased at an average annual rate of around 15% in China from 2000 to 2011, 32,33 a finding supported by the prescribing practices of the control group in this study.

Health workers reported that the biggest advantage of using text messages was the ease in receiving and retrieving information. Preliminary research found that health workers had limited time to study medical information, with 62% of health workers having less than 3 hours per week to read medical literature.6 Health workers also reported a preference for information delivery platforms that were more convenient and easier to use. Text messages are limited to only 280 characters, however, which prevent the dissemination of highly detailed recommendations. This weakness could be overcome by increasing the frequency that text messages are sent. An open-access database for health workers that included detailed information on the treatment of each disease could further resolve this issue. Text messages received high scores for their validity and applicability, which suggested that recommendations should be both evidence-based and suited to the local disease context. Nearly all participants in the intervention group (95%) wanted to continue receiving text messages.

A major benefit of using text messages is the cost-effectiveness, which is a key consideration in resource-poor settings. Each text message costs approximately 0.1 Yuan (United States dollars, US$ 0.016) to send. In this study, total expenditure on text messages for the intervention group was less than 1000 Yuan (US$ 160.64), or less than 2 Yuan (US$ 0.32) per health worker. In comparison, the one-day training for the control group cost an average of 560 Yuan (US$ 89.96) per health worker, for printed materials, accommodation and transportation costs. This amounts to a 280-fold difference per person. While not discounting the advantages of traditional medical education, such as the face-to-face interactions, discussions, and communication between trainees and trainers, the use of text messages provides an effective low-cost alternative that can reach a larger audience of health workers more frequently.

In our study, we assessed the effectiveness of text messages as tools to both increase knowledge of evidence-based recommendations, and positively affect physician practices. The main strengths of this study include the pragmatic design, the large numbers of participants, and the subjective and objective outcomes tested. All recommendations sent to health workers came from evidence-based resources, such as the Clinical Evidence and the Cochrane Library. Recommendations were further reviewed and modified by senior physicians from the First Hospital of Lanzhou University to ensure readability and utility. The cluster-randomized trial was preceded by pilot studies conducted at seven health centres over the course of two years. These pilot studies assessed the viability and applicability of text messages for continued medical education, and found that using text messages as a knowledge translation tool could change physician knowledge and behaviour.

However, our results should be considered within the limitations of the study. First, although we evaluated the long-term effects (i.e. one year) of the intervention in our pilot study, only the short-term effects (i.e. three months) were evaluated by telephone survey in the main trial. Future studies should address the long-term utility of text messages as a tool of knowledge translation. Second, the causal relationship between text messages and physicians’ behaviour change remains ambiguous, and could not be fully addressed in this study. Third, although health workers’ scores were higher, on average, after the intervention, their scores remained poor. This suggests that text messaging may be an improvement over the traditional educational method but that its role in continuing medical education needs to be researched further. Fourth, the complexities of behaviour change might not have been fully captured by this study. We assumed that prescriptions were reflective of behaviour, and that physicians were important loci of change, given their authoritative role in health centres. Future studies could build on our findings by developing them through behaviour change theories.34

On the basis of our pilot studies and this cluster-randomized trial, our findings showed that text messages offer a convenient, inexpensive, and effective method to disseminate evidence-based recommendations with the effect of increasing rural health workers’ clinical knowledge and positively impacting their prescription practices.


We thank Xiaojuan Xiao, Zehao Wang and Qi Wang for their great help in data collection and telephone surveys. We thank Hairong Bao and Xiaoju Liu for their work in reviewing and revising the text messages. We are also thankful to Emilio Dirlikov and Yngve Falck-Ytter for their comments and revisions on earlier drafts of our trial.


This study was funded by grants from the China Medical Board, Grant No. 09-944. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests:

None declared.