Evaluate activities in regards to principles
Evaluating how WHO communications fulfil the principles of effective communications practice
Assessments related to the principles of effective communication may be conducted for particular products; for example, issuing more rapid press releases (timeliness). Alternatively, they may be applicable to ongoing practices; for example, writing communications products targeted to the public in plain language (increasing understanding).
Assessment steps
Varied methods and tools for setting goals, developing objectives, creating indicators, and measuring progress can be found in the resources of the Framework’s Evaluation section. The seven-step process presented below is suggested as one possible model that may be used to improve communication in relation to the principles set out in the Framework (Fig.1).
1. Identify an activity or product to improve
Identify an objective for a communication practice or product by better aligning planned activities or materials with one or more of the principles of effective communication (i.e. aim to make it more relevant to audiences, easier for people to understand, etc.
Communicators are encouraged to start evaluation with realistic objectives. For example, it is important to remember that few individual communication interventions can move an audience immediately from awareness to behaviour change. However, communicators can show incremental improvements in products and processes along a range of communications goals (such as increasing awareness, enhancing understanding of health risks, or nurturing the skills needed for audiences to under take an action based on WHO’s information and advice). These are examples of objectives related to each principle in the Framework.
- Increase the number of people who see the programme’s feature stories (Accessible).
- Get more staff at WHO to walk the stairs, instead of taking the elevators (Actionable).
- Increase the number of WHO Twitter followers who share the Organization’s health messages with their networks (Credible).
- Raise the percentage of staff who sign up for training after receiving a notice of capacity-building opportunities (Relevant).
- Decrease the amount of time needed to clear press releases (Timely).
- Make the language in messages targeted at the public readable at the high school level (Understandable).
2. Identify a tactic to improve performance
Using respective sections of the Framework (timeliness, relevance, etc.) identify a tactic that may best suit the products or interventions chosen to be implemented in order to meet the communications objectives.
3. Create indicators to measure improvement
Create precise indicators that would measure the communications performance, both before and after new tactics have been introduced for a set period of time. Indicators are used to measure communications outputs and show if a tactic has had desired results. Indicators should be:
These are examples of indicators that may be used.
- The number of feature stories posted or linked across all three levels of WHO’s websites (headquarters, regional, and country). (Accessible).
- The number of people walking the stairs during lunchtime, 12:00-14:00. (Actionable).
- The number of WHO Twitter followers to share the Organization’s health messages with their networks. (Credible).
- The number of staff signing up for training using forms included in notices for capacity-building activities. (Relevant).
- Amount of time from sending a draft press release for clearance to its distribution to the news media. (Timely).
- Scores on automated reading level assessments of communication materials. (Understandable).
4. Conduct a baseline assessment
Conduct a baseline assessment to identify how well the unit is currently performing on these indicators. Also called “formative research”, baseline assessments are carried out before a new tactic or intervention is used to improve performance.
The baseline uses the same method of measuring that will be later used by communicators to find out if a new intervention has improved performance. These are examples of baseline assessments.
- Check current website traffic over a set period to monitor the number of views on a sample of website feature stories. (Accessible).
- Put monitors in the stairways for a set period of time to count the number of people who take the stairs. (Actionable).
- Count the number WHO health messages that are shared by followers over a set period of time. (Credible).
- Count the number of staff who sign up for training after receiving the training advertisement. Compared to the number of those who signed up independently of the advertisement. (Relevant).
- Track the time it takes to send numberous drafts of press releases through clearance to dissemination to the press and/or posting on relevant websites. (Timely).
- Using computer based assessment software, assess the reading level of current communication materials targeted to the public. (Understandable).
5. Refine indicators
Refine the objectives from the baseline assessment. What would be considered a success given current performance levels and proposed intervention? These are examples of refined objectives.
- A 100% increase in the number of people who see a programme’s feature stories. (Accessibility).
- Get 20% more staff walking the stairs, instead of taking the elevators. (Actionable).
- A 10% increase in the number of WHO Twitter followers who share the Organization’s health messages with their networks. (Credibility).
- A raise in the percentage of staff from 5% to 10% signing up for training after receiving a notice of capacity-building opportunities. (Relevance).
- A 20% decrease of the amount of time needed to clear press releases. (Timeliness).
- Ensure 100% of WHO materials targeted to lay audiences readable at the high school level. (Understandable)
6. Perform new tactic(s)
Using selected tactics, establish new practices and sustain them for a period of time necessary to find out if they help communicators make progress in meeting the predefined objectives. Examples of tactics can include the following.
- Work with colleagues at all levels of WHO on feature stories so they are promoted and linked across websites at all levels of the Organization. (Accessible).
- Set up posters at point-of-decision locations, i.e. elevator doors, explaining the benefits of stair climbing. (Actionable).
- Include call to action for WHO Twitter followers to share the Organization’s health messages with their networks. (Credible).
- Insert testimonials from staff who took the capacity-building training and benefited from it. (Relevant).
- Set up expedited clearance protocol, such as back-ups for clearance points who may be on leave, time limits for “no-response”. (Timely).
- Follow the guidelines for high school reading level in all outreach materials such as the number of syllables per word, length of sentences, amount of white space, etc.). (Understandable)
7. Measure progress
After using new tactics for a set period of time, repeat measures to compare results against the baseline. Depending upon degree of success communicators may decide to further explore the following areas.
- Continue the new practice based on good results, measuring periodically to sustain or improve performance.
- Keep on using the new tactic and measure after it has had more time to achieve impact. Some practices take longer to show results.
- Try a different approach. Communicators may need to try modified or different tactics before finding the one that works best for their context, objectives and primary audiences.
- Use the evaluation results to inform communications planning for next steps and future outreach.
- Share the results with relevant technical staff as rationale for using effective communication tactics. Also share them with other communication colleagues who can benefit from the evaluation and testing.
