Using evidence within HIA
Bringing together different types of evidence
At some point, an individual or a group must decide what the main evidence messages are. This is one of the most challenging components of any HIA.
Usually the different forms of evidence support one another, but sometimes findings can be conflicting. Evidence can conflict in the:
- direction of findings, for example one type of evidence reports a positive health effect, while another reports a negative effect on health.
- size of the health effect, for example one type of evidence may report a small, or statistically insignificant effect, while another type of evidence may report a significant health effect.
When bringing the different types of evidence together, questions and issues that can be considered are:
What are your values regarding evidence?
People value different types of evidence differently. Within the clinical sciences, the general view is that systematic reviews are the most robust and reliable markers of effectiveness, closely followed by a well-designed meta-analyses. Both rely heavily on data from randomised controlled trials, and give less weight to case-control studies and observational studies. Yet relying exclusively on systematic reviews and meta-analyses within HIA leads to significant limitations. Another, broader approach is to value all types of evidence, including experiential evidence of members of the public.
Combining evidence is challenging
As with all areas dealing with evidence, there are challenges in trying to balance different types of conflicting evidence within HIA. In some areas such as qualitative research, researchers may argue about the threshold of what is considered good quality work. There is no universally applicable method for systematically combining or reviewing qualitative findings, although a number of projects have developed interesting methodological approaches in this area. Nor is there any clear or agreed method for combining non-traditional forms of evidence – such as the study of public opinion – with evidence from more traditional studies to provide a comprehensive assessment of the effectiveness of different interventions. For the time being, the methods described below are among those used at present by HIA practitioners to overcome such difficulties.
How do HIA practitioners combine evidence?
For some practitioners, one of the main reasons for using HIA is that it is a participatory approach that values the views of the community. If this is to be done seriously, then all types of evidence must be valued during the assessment of a proposal. Within the HIA process, the views of the public can sit alongside other evidence such as expert opinion and scientific data, with each being presented and valued in an appropriate manner. Two options for considering the evidence are:
- deciding what type of evidence will be given the highest weighting. Considering the diversity of stakeholder views is needed, but an attempt to provide a clear evidence-base of the potential health impacts will assist the development of a single set of recommendations. The steering group have an important role in making the decisions around what types of evidence are valued. It is useful if the steering committee have discussed such issues well before they are presented with any potentially challenging decisions.
- weighting evidences equally. This will mean that a wider range of evidence can move into the next stage of the HIA, ‘formulating recommendations’, but this is not necessarily a problem. Recommendations that cover all possible decisions can be provided from this evidence. The recommendations can acknowledge conflicting evidence and can outline stakeholder views on the evidence. This approach can work as HIA does not make decisions, it provides information in a clear and transparent way for decision makers. An HIA example that used this way of developing recommendations was the HIA of a proposal to substitute chopped tyres for some of the coal used in a cement kiln in Rugby, England.
Presenting the evidence in a user-friendly form
The evidence gathered is usually presented to a number of different audiences:
Consideration of the different realities of each of these groups is required when determining how evidence will be presented. What is suitable for one group is unlikely to be suitable for another.
- Steering group
- Participants of appraisal workshops