As researchers, we hope that when politicians are making decisions about policy, they use our research to help ground their thinking. In Canada, CIHR (Canadian Institutes of Health Research), encourages researchers to make their work accessible, and has specific grant applications focused on Knowledge Translation (KT).
Part of the trouble with evaluating whether or not politicians use research when making decisions about public health is the sheer scope of public health. While we may consider very direct issues, such as vaccinations, one could argue that macro level issues such as agriculture, crime and town planning all affect public health (I briefly mentioned the latter in a previous post).
The problem is further exacerbated by how complicated the literature on public health is. While you can make decisions about drug effectiveness using randomized controlled trial data, public health research can include surveys, cohort/case-control studies and cost-effectiveness analyses. Even expert opinion can be considered a valid resource when used properly.
Now, knowing how complicated this all is, you can still try to draw some conclusions. First, is research evidence used? Secondly, if it is, what kind of evidence is used when making decisions? Third, how do they use evidence? Fourth, what else do they use when making a decision? And finally, what stops them from using research evidence?
More after the jump.
What did they do?
To address this study, they conducted a systematic review of the literature. They selected studies that were done in countries with universal health care (Europe, Canada, Australia and New Zealand) and were done after 1980. They used a multitude of sources, including the Cochrane Collaboration, MEDLINE, SCOPUS, PsychInfo and others.
What did they find?
They initially identified 4154 articles. Of these, 4095 were removed as either duplicates or because they were out of the scope of this review, 40 did not meet the inclusion criteria and one was published twice. This left 18 studies.
They found that there was little evidence linking research evidence and public health decision making. Only 63% of Ontario health staff reported using at least one systematic review in their decision making process in the previous 2 years, and only 28% used academic research. However, these numbers were reported in two separate studies and so you cannot decide whether this is an exception or the norm.
Very few studies looked at how policy makers used research evidence. While they agreed on what constituted evidence, a surprising result was that managers were likely to make a decision then find evidence to justify it. When asked why they didn’t use research, several themes emerged, including the lack of certainty in research, poor local applicability and a lack of focus on the social determinants of health.
So what does this all mean?
There isn’t much quantitative evidence detailing how research is used by policy makers. The evidence that does exist is not comprehensive, and doesn’t paint a good picture of what happens in the decision making process. There is a lot of qualitative research, but this has been done in either small or non-representative samples, and so it is difficult to draw conclusions from these data.
And what now?
As researchers, we need to work in tandem with policy makers to make sure that policies are based on evidence. This can include being more proactive in knowledge translation, as well as relying on summaries of the current state of knowledge (similar to Cochrane reviews). The barriers revealed in this study need to be examined, and where possible, addressed.
The article is available for free on the PLOS ONE website here: http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0021704
Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, & Capewell S (2011). The use of research evidence in public health decision making processes: systematic review. PloS one, 6 (7) PMID: 21818262
Mr Epidemiology 
October 1, 2011 at 9:00 pm
It annoyed me greatly when I discovered how health policy is made in Australia, since in studying public health I decided that evidence-based policy was the field I wanted to get into. I should have known better than to believe anything had changed since I first encountered the policy arm of the local state health department in 1986. Working in a different field, my boss nevertheless wanted some population figures on a condition he was studying, in order to see if it was being catered to fairly on a population basis. He contacted the Policy Division and discovered they knew nothing about the prevalence or incidence, but nevertheless had written policies, which the health department followed. It transpired that the job in that department was “policy writing” and that all of the people in it were either sociologists or social workers. They based policy on philosophical grounds (according to the fashion of the day, I concluded), with no reference to assessment of published evidence or local conditions. They had not even heard of the federal department for census and statistics. We were rather shocked and immediately organised a multi-pronged survey of the population to ascertain the prevalence and estimated incidence of the condition of interest. More recently, having completed an MPH course, I hunted around for some mentoring or “apprentice” type of work to see if I felt comfortable getting into policy work. Then I discovered that these days policy seemed to be developed out of two main arms: lobbyists on behalf of small groups in the population [eg. physically disabled,; sports people & exercise physiologists; private health insurance companies in a mainly public system, etc]; and: “policy wonks”. The wonks seem to be silver-tongued individuals with various biases who do use some statistics to support their work, but still follow in the footsteps of the old-style policy writers. They don’t obviously belong to lobby groups, but I can see which ones are covertly representing others. Sure, there are policy studies groups attached to various universities, think tanks and private institutes, but I could not see a clear path for their expert advice to be taken directly on board by politicians without the wonks or the lobbyists getting equal or unequal “talk time”. I’ve tucked myself into the state health policy group of a political party more to see what happens than in the expectation that policy-making will ever consider proper evidence. Of course my “position” is unpaid and I can’t fathom where I’m ever going to be able to get a job in this area! Systematic reviews and epidemiological surveys don’t stand a chance in the grass roots of policy-making here!
My blog: http://healthforhumans.blogspot.com