As researchers, we hope that when politicians are making decisions about policy, they use our research to help ground their thinking. In Canada, CIHR (Canadian Institutes of Health Research), encourages researchers to make their work accessible, and has specific grant applications focused on Knowledge Translation (KT).

Part of the trouble with evaluating whether or not politicians use research when making decisions about public health is the sheer scope of public health. While we may consider very direct issues, such as vaccinations, one could argue that macro level issues such as agriculture, crime and town planning all affect public health (I briefly mentioned the latter in a previous post).

The problem is further exacerbated by how complicated the literature on public health is. While you can make decisions about drug effectiveness using randomized controlled trial data, public health research can include surveys, cohort/case-control studies and cost-effectiveness analyses. Even expert opinion can be considered a valid resource when used properly.

Now, knowing how complicated this all is, you can still try to draw some conclusions. First, is research evidence used? Secondly, if it is, what kind of evidence is used when making decisions? Third, how do they use evidence? Fourth, what else do they use when making a decision? And finally, what stops them from using research evidence?

More after the jump.

What did they do?

To address this study, they conducted a systematic review of the literature. They selected studies that were done in countries with universal health care (Europe, Canada, Australia and New Zealand) and were done after 1980. They used a multitude of sources, including the Cochrane Collaboration, MEDLINE, SCOPUS, PsychInfo and others.

What did they find?

They initially identified 4154 articles. Of these, 4095 were removed as either duplicates or because they were out of the scope of this review, 40 did not meet the inclusion criteria and one was published twice. This left 18 studies.

They found that there was little evidence linking research evidence and public health decision making. Only 63% of Ontario health staff reported using at least one systematic review in their decision making process in the previous 2 years, and only 28% used academic research. However, these numbers were reported in two separate studies and so you cannot decide whether this is an exception or the norm.

Very few studies looked at how policy makers used research evidence. While they agreed on what constituted evidence, a surprising result was that managers were likely to make a decision then find evidence to justify it. When asked why they didn’t use research, several themes emerged, including the lack of certainty in research, poor local applicability and a lack of focus on the social determinants of health.

So what does this all mean?

There isn’t much quantitative evidence detailing how research is used by policy makers. The evidence that does exist is not comprehensive, and doesn’t paint a good picture of what happens in the decision making process. There is a lot of qualitative research, but this has been done in either small or non-representative samples, and so it is difficult to draw conclusions from these data.

And what now?

As researchers, we need to work in tandem with policy makers to make sure that policies are based on evidence. This can include being more proactive in knowledge translation, as well as relying on summaries of the current state of knowledge (similar to Cochrane reviews). The barriers revealed in this study need to be examined, and where possible, addressed.


The article is available for free on the PLOS ONE website here:
Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, & Capewell S (2011). The use of research evidence in public health decision making processes: systematic review. PloS one, 6 (7) PMID: 21818262