Language about evidence is often vague. In the following examples, it is impossible to tell from the terms used (in italics) just what the evidence is or what makes it ‘high level’, ‘limited’ and so on, especially when the sentences are also laden with other jargon:
Despite a lack of high-level evidence in its favour, laparoscopic adrenalectomy has practically replaced open adrenalectomy in the management of benign adrenal lesions.
Although there is modest high-quality evidence of traditional ‘clinical outcomes’ from self-management programs, these programs are strongly endorsed by consumers.
In contrast to commonly held opinion, there is limited strong evidence that lifestyle factors are a dominant factor in the pathophysiology of …
To date, there is no clear evidence of an increased cancer risk in medical radiation workers exposed to current levels of radiation doses.
Some evidence suggests that changes in snowmelt may also increase the risk of forest fires.
Mounting evidence from diverse archives suggests the occurrence of an abrupt ‘cold snap’ just before the onset of the prolonged cooler late Holocene.
Scientific evidence for warming of the climate system is unequivocal.
This is less of a problem when statements about evidence occur alongside data that explain what the evidence is, or set some parameters about the intended meaning. But when such statements are used in summary information (such as an abstract, literature review, plain-English summary or media release), they can give an inaccurate or misleading message. For example, does modest high-quality mean a few (how many?) high-quality studies that showed a modest effect, or some less-than-high-quality studies that showed a large effect?
To improve the clarity of evidence statements, 4 strategies are helpful:
- Replace the word evidence with a description of the studies behind it.
- Replace imprecise quality adjectives (eg good, poor) with a description of the study design.
- Replace misleading quantity adjectives (eg no, little, some) with a description of the study results.
- Define or grade evidence against a predefined standard.
The following examples are from clinical medicine, but the same principles apply in other disciplines.
Replace the word evidence with a description of the studies behind it
To improve reader comprehension, replace the word evidence with a description of the research or studies that provide the evidence. Using adjectives such as good-quality or well-designed to describe the studies themselves has more meaning than vague terms such as moderate evidence.
Replace imprecise quality adjectives with a description of study design
Avoid vague language by cutting out as many imprecise adjectives as possible (eg high-level, good, consistent, reasonable, compelling, poor, insufficient). Instead, describe what you mean (eg X is more effective than Y). Shift the quality descriptors onto the studies or the size of the effect (eg well-designed randomised controlled trial, large effect):
A meta-analysis of 6 large, well-designed randomised controlled trials showed that X was more effective than Y for reducing Z pain.
Replace misleading quantity adjectives with a description of study results
Avoid adjectives that describe the quantity of evidence. The term no evidence is particularly misleading because it could mean many high-quality studies that show no statistically significant effect, or no studies at all, or anything in between:
- If you mean that there have not been any studies, make this clear and admit that you do not know the answer
Do not say:
There is no evidence that X is better/more effective/superior/more beneficial than Y. [which implies that studies have been done and they have shown no beneficial effect]
Say:
We do not know how X compares with Y because there have been no studies.
- If you mean that a lot of evidence is available, but it does not show a statistically significant effect, clearly state this
Do not say:
There is no evidence that X is effective for Z.
Say:
A large well-designed clinical trial showed no statistically significant effect of X for Z.
- If you mean that there are some studies, but they are inconsistent and do not lead to firm conclusions, clearly state this
Do not say:
There is no conclusive evidence that X is effective for Z.
Try something like:
Four small randomised controlled trials showed inconsistent results, so it is impossible to draw any conclusions about whether X is more or less effective than Y. Further well-designed trials are needed to answer this question.
Although no evidence is the most striking example of this problem, the same reasoning applies to any descriptors that refer to the amount of evidence (eg little evidence, some evidence, much evidence). For example, little evidence could mean 1 study that showed a large effect or several studies that showed marginal effects, and so on.
Examples of how to apply these strategies are shown in the following table.
Strategies for precise wording for evidence statements
|
Type of studya
|
Results
|
Example evidence statement
|
|
Several well-designed studies
|
Positive effect (effective/beneficial)
|
Three well-designed clinical trials showed that X decreases [outcome] compared with Y.
Three good-quality studies showed that X is effective [beneficial] for …
|
|
Several well-designed studies
|
Negative effect
(harmful)
|
Three well-designed clinical trials showed that X increases [outcome] compared with Y.
Three good-quality studies showed that X is less effective [harmful] than Y for …
|
|
Several well-designed studies
|
No significant effectb
|
Three well-designed clinical trials showed no significant difference between X and Y.
Three good-quality studies showed no significant difference between X and Y.
Do not say ‘there is no effect’ – ‘no effect’ is impossible to prove in science, and studies that do not show a statistically significant effect may have wide confidence intervals that include biologically or clinically important effects.c
Do not say ‘there is no evidence’. There is evidence – it just shows no significant effect!
|
|
1 well-designed study
|
Positive or negative effect
|
One well-designed trial has shown that X is more effective (or less effective) than Y … but this finding needs to be confirmed by more trials.
|
|
2 or more well-designed studies with conflicting results
|
At least 1 study shows a positive effect; at least 1 study shows no significant effect
|
Two good-quality studies show conflicting [inconsistent] results for the relative effectiveness of X and Y (or ‘about whether X reduces [outcome] …’).
|
|
1 or more poorly designed studies
|
Positive or negative effect
|
One small [poorly designed] clinical trial showed that X was better [worse] than Y. Larger [better-designed] trials are needed to check this result.
Do not say ‘larger trials are needed to confirm this result’ because it is not known whether the finding will be confirmed or refuted until the larger trials are done.
|
|
1 or more poorly designed studies
|
No significant effect
|
Several small studies have shown that X may not be effective for …, but larger [better-designed] trials are needed to further investigate this effect.
There have not been enough well-designed clinical trials to either support or refute …
|
|
No studies
|
Not applicable
|
There have been no studies of the effect of …
The effect of X … is not known.
Do not say ‘there is no evidence for …’. This can be confused with a negative or not significant result.
|
a The general term study refers to any research studies (eg clinical trials, ecological studies, other scientific research). Well-designed means
a protocol that eliminates bias (eg randomised allocation to groups, blinded measurement of outcomes) and with enough subjects or
measurements to allow accurate statistical analysis. Descriptors in the evidence statements will vary according to the type of research.
b The term significant is used here as shorthand for statistically significant.
c An effect can be statistically significant but not be of real biological or clinical importance (eg pain might increase but remain within a level
that does not affect daily life).
Define or grade evidence against a standard
To take the guesswork out of the meaning of adjectival statements about scientific evidence, some organisations have defined the adjectives they use. Use these, if appropriate in your field:
- The International Agency for Research on Cancer has defined several categories of evidence that support the carcinogenicity of agents in humans and animals.
- The National Health and Medical Research Council (in Australia) and most other major health agencies worldwide have adopted approaches to grading the evidence that health advice draws on.
- The United Nations Intergovernmental Panel on Climate Change (IPCC) has published definitions of adjectival terms relating to likelihood of an outcome.
IPCC likelihood scale
|
Term
|
Likelihood of outcome (% probability)
|
|
Virtually certain
|
99–100
|
|
Very likely
|
90–100
|
|
Likely
|
66–100
|
|
About as likely as not
|
33–66
|
|
Unlikely
|
0–33
|
|
Very unlikely
|
0–10
|
|
Exceptionally unlikely
|
0–1
|
Source: Guidance note for lead authors of the IPCC Fifth Assessment Report on consistent treatment of uncertainties
