Writing about evidence

Many disciplines, such as science and economics, are based on evidence.

In the past 2 decades, the term evidence-based practice has been used to refer to practice or decision making that is based on a body of evidence (eg scientific studies) that has been systematically collected and critically assessed. This terminology was pioneered in the fields of health and medicine, but has also been applied to the environment, agriculture, social sciences, education and economics.

Although evidence-based practice has added much rigour to the analysis of data in these fields, it has led to an overreliance on the word evidence in technical and general publications, including practice guidelines (such as clinical practice or environmental management), government reporting and the media. What constitutes evidence? What constitutes evidence that is reliable enough to guide practice? There is no clear-cut, simple definition, so use of this term can muddy the waters.

To avoid confusion, 3 golden rules stand out: 

Download our quick guide for easy reference: The language of evidence .

Types of evidence

Evidence can be quantitative or qualitative:

  • Quantitative evidence is based on data that can be put into numbers. This includes quantitative measurements and statistics that can be analysed with respect to past outcomes, and used to build models to predict future outcomes. The data are often generated by carefully designed comparisons of outcomes in 2 or more controlled systems. This is the preferred evidence in disciplines such as science, medicine and economics. Claims made in these fields usually require meaningful quantitative supporting evidence to be credible.
  • Qualitative evidence is based on data that can be put into words, such as narratives or comparisons based on observations and descriptions. This includes identifying themes across a set of events or observations, and seeing where there may be connections or commonalities. In some fields of research, such as social science, comparative quantitative studies are difficult to do, and qualitative studies such as surveys or case studies provide the most robust form of evidence.
  • Qualitative examples are supporting pieces of information, such as quotes and extracts from relevant texts, experts and other research. Claims made in a humanities, arts or similar context require supporting information of this sort to be persuasive.

In some fields, mixed methods research combines elements from qualitative and quantitative methods to provide evidence. For example, a researcher might conduct a multiple-choice survey (quantitative) and then ask for further comments in free-form boxes or interview people in person about their responses (qualitative).

The table shows the types of evidence that can be considered for quantitative and qualitative research, and the supporting information that should be provided.

Examples of types of evidence and key information to include when writing about such evidence

Methodology

Examples of evidence and supporting information for various fields

Quantitative

Primary evidence:

  • experimental data
  • observed data (measurement or scoring with human observation or using instruments)

Secondary evidence:

  • official or public data and statistics (archives, records, census data, maps, contracts)
  • reviews and reports

Supporting information:

  • details of data, including
    • context (experimental or observational)
    • sample size
    • sample method (eg random, stratified, volunteer)
    • statistical significance of calculated statistics
  • discussion of dependent and independent variables, carefully defined and differentiated
  • discussion of correlation, and clear articulation of significance and possible causation

Qualitative

Primary evidence:

  • direct observations (in a real-world or experimental context)
  • surveys and interviews (groups or individuals)
  • transcripts; audio or video recordings
  • personal items and artefacts (eg photographs, objects, letters, art works)
  • eyewitness accounts, autobiographies, diaries, records kept by agencies or other organisations (eg speeches, meeting minutes)
  • contemporaneous accounts (eg newspaper reports)

Secondary evidence:

  • journal articles
  • histories and biographies
  • textbooks
  • documentaries

Supporting information:

  • identification of primary vs secondary evidence
  • identification of selection method for sample group from which data were collected (eg random, stratified, volunteer)
  • explanation of factors that may influence information provided, such as
    • language ability: can subjects express themselves fully? Are documents translated from the original?
    • environmental context: are subjects likely to be candid? Is groupthink a factor?
Return to top

Quality of evidence

Within each field of study, some study designs or types of information are considered to be more reliable sources of evidence than others. For example, in clinical medicine, randomised controlled trials are the gold standard because their study design inherently eliminates bias that would otherwise make the results less reliable (eg because of the well-known placebo effect). Such trials can be well or less well conducted. To write accurately about evidence in a specific discipline, it is important to be familiar with the hierarchy of evidence in your discipline and what defines a well-conducted study or investigation. 

Return to top

Be specific and accurate

Language about evidence is often vague. In the following examples, it is impossible to tell from the terms used (in italics) just what the evidence is or what makes it ‘high level’, ‘limited’ and so on, especially when the sentences are also laden with other jargon:

Despite a lack of high-level evidence in its favour, laparoscopic adrenalectomy has practically replaced open adrenalectomy in the management of benign adrenal lesions.

Although there is modest high-quality evidence of traditional ‘clinical outcomes’ from self-management programs, these programs are strongly endorsed by consumers.

In contrast to commonly held opinion, there is limited strong evidence that lifestyle factors are a dominant factor in the pathophysiology of

To date, there is no clear evidence of an increased cancer risk in medical radiation workers exposed to current levels of radiation doses.

Some evidence suggests that changes in snowmelt may also increase the risk of forest fires.

Mounting evidence from diverse archives suggests the occurrence of an abrupt ‘cold snap’ just before the onset of the prolonged cooler late Holocene.

Scientific evidence for warming of the climate system is unequivocal.

This is less of a problem when statements about evidence occur alongside data that explain what the evidence is, or set some parameters about the intended meaning. But when such statements are used in summary information (such as an abstract, literature review, plain-English summary or media release), they can give an inaccurate or misleading message. For example, does modest high-quality mean a few (how many?) high-quality studies that showed a modest effect, or some less-than-high-quality studies that showed a large effect?

To improve the clarity of evidence statements, 4 strategies are helpful:

  • Replace the word evidence with a description of the studies behind it.
  • Replace imprecise quality adjectives (eg good, poor) with a description of the study design.
  • Replace misleading quantity adjectives (eg no, little, some) with a description of the study results.
  • Define or grade evidence against a predefined standard.

The following examples are from clinical medicine, but the same principles apply in other disciplines.

Replace the word evidence with a description of the studies behind it

To improve reader comprehension, replace the word evidence with a description of the research or studies that provide the evidence. Using adjectives such as good-quality or well-designed to describe the studies themselves has more meaning than vague terms such as moderate evidence.

Replace imprecise quality adjectives with a description of study design

Avoid vague language by cutting out as many imprecise adjectives as possible (eg high-level, good, consistent, reasonable, compelling, poor, insufficient). Instead, describe what you mean (eg X is more effective than Y). Shift the quality descriptors onto the studies or the size of the effect (eg well-designed randomised controlled trial, large effect):

A meta-analysis of 6 large, well-designed randomised controlled trials showed that X was more effective than Y for reducing Z pain.

Replace misleading quantity adjectives with a description of study results

Avoid adjectives that describe the quantity of evidence. The term no evidence is particularly misleading because it could mean many high-quality studies that show no statistically significant effect, or no studies at all, or anything in between:

  • If you mean that there have not been any studies, make this clear and admit that you do not know the answer
Do not say:
There is no evidence that X is better/more effective/superior/more beneficial than Y. [which implies that studies have been done and they have shown no beneficial effect]
Say:
We do not know how X compares with Y because there have been no studies.
  • If you mean that a lot of evidence is available, but it does not show a statistically significant effect, clearly state this

Do not say:
There is no evidence that X is effective for Z.
Say:
A large well-designed clinical trial showed no statistically significant effect of X for Z.

  • If you mean that there are some studies, but they are inconsistent and do not lead to firm conclusions, clearly state this

Do not say:
There is no conclusive evidence that X is effective for Z.
Try something like:
Four small randomised controlled trials showed inconsistent results, so it is impossible to draw any conclusions about whether X is more or less effective than Y. Further well-designed trials are needed to answer this question.

Although no evidence is the most striking example of this problem, the same reasoning applies to any descriptors that refer to the amount of evidence (eg little evidence, some evidence, much evidence). For example, little evidence could mean 1 study that showed a large effect or several studies that showed marginal effects, and so on.

Examples of how to apply these strategies are shown in the following table.

Strategies for precise wording for evidence statements

Type of studya

Results

Example evidence statement

Several well-designed studies

 

Positive effect (effective/beneficial)

Three well-designed clinical trials showed that X decreases [outcome] compared with Y.

Three good-quality studies showed that X is effective [beneficial] for …

Several well-designed studies

Negative effect
(harmful)

Three well-designed clinical trials showed that X increases [outcome] compared with Y.

Three good-quality studies showed that X is less effective [harmful] than Y for …

Several well-designed studies

No significant effectb

Three well-designed clinical trials showed no significant difference between X and Y.

Three good-quality studies showed no significant difference between X and Y.

Do not say ‘there is no effect’ – ‘no effect’ is impossible to prove in science, and studies that do not show a statistically significant effect may have wide confidence intervals that include biologically or clinically important effects.c

Do not say ‘there is no evidence’. There is evidence – it just shows no significant effect!

1 well-designed study

Positive or negative effect

One well-designed trial has shown that X is more effective (or less effective) than Y … but this finding needs to be confirmed by more trials.

2 or more well-designed studies with conflicting results

At least 1 study shows a positive effect; at least 1 study shows no significant effect

Two good-quality studies show conflicting [inconsistent] results for the relative effectiveness of X and Y (or ‘about whether X reduces [outcome] …’).

1 or more poorly designed studies

Positive or negative effect

One small [poorly designed] clinical trial showed that X was better [worse] than Y. Larger [better-designed] trials are needed to check this result.

Do not say ‘larger trials are needed to confirm this result’ because it is not known whether the finding will be confirmed or refuted until the larger trials are done.

1 or more poorly designed studies

No significant effect

Several small studies have shown that X may not be effective for …, but larger [better-designed] trials are needed to further investigate this effect.

There have not been enough well-designed clinical trials to either support or refute …

No studies

Not applicable

There have been no studies of the effect of …

The effect of X … is not known.

Do not say ‘there is no evidence for …’. This can be confused with a negative or not significant result.

a  The general term study refers to any research studies (eg clinical trials, ecological studies, other scientific research). Well-designed means
    a protocol that eliminates bias (eg randomised allocation to groups, blinded measurement of outcomes) and with enough subjects or
    measurements to allow accurate statistical analysis. Descriptors in the evidence statements will vary according to the type of research.

b  The term significant is used here as shorthand for statistically significant.

c  An effect can be statistically significant but not be of real biological or clinical importance (eg pain might increase but remain within a level
    that does not affect daily life).

Define or grade evidence against a standard

To take the guesswork out of the meaning of adjectival statements about scientific evidence, some organisations have defined the adjectives they use. Use these, if appropriate in your field:

  • The International Agency for Research on Cancer has defined several categories of evidence that support the carcinogenicity of agents in humans and animals.
  • The National Health and Medical Research Council (in Australia) and most other major health agencies worldwide have adopted approaches to grading the evidence that health advice draws on.
  • The United Nations Intergovernmental Panel on Climate Change (IPCC) has published definitions of adjectival terms relating to likelihood of an outcome.

IPCC likelihood scale

Term

Likelihood of outcome (% probability)

Virtually certain

                     99–100

Very likely

                     90–100

Likely

                     66–100

About as likely as not          

                      33–66

Unlikely

                       0–33

Very unlikely

                       0–10

Exceptionally unlikely

                       0–1

Source: Guidance note for lead authors of the IPCC Fifth Assessment Report on consistent treatment of uncertainties

Return to top

Avoid biased language

Language about evidence is often emotive. Consciously or subconsciously, writers may use language that reveals their own beliefs or hopes. This can creep into writing in many subtle ways:

The benefits of eating Vegemite sandwiches to reduce stress levels have not been studied.

The use of the positive word benefits suggests that eating Vegemite sandwiches is beneficial. The writer thinks that, when the research is in, it will confirm their belief in the benefit. But, if there have not been any studies, we do not know if it is beneficial, neutral or harmful. This is the most common sort of bias in scientific literature. Contrast it with an unbiased version:

We don’t know the effect of Vegemite sandwiches on stress levels.

Another pitfall is revealed here:

There is not enough evidence to favour red roses over white roses for those hoping for romantic success.

The use of the positive word favour influences the meaning of the sentence – because there is ‘not enough evidence’, favour red roses could just as easily be favour white roses. It is better to say:

It is not known whether sending red or white roses is more likely to help you to achieve romantic success.

Another example:

The available evidence suggests that advice to stay active has small beneficial effects for patients with acute simple low back pain, and little or no effect for patients with sciatica.

Available implies that there is ‘unavailable’ evidence that might say something different. Non-existing (or potential, possible, future, further …) evidence is just that, so do not imply that it exists and might be favourable for the issue under consideration.

Return to top

User login

... or purchase now

An individual subscription is only A$60 per year

Group and student discounts may apply

Australian manual of scientific style Start communicating effectively

Purchase