A UBC Okanagan professor is suggesting government policy makers and advisors need to do a rethink when it comes to giving validity to reports coming across their desks.
Carey Doberstein, an assistant professor of political science a the Kelowna campus, recently published an experimental study of public sector workers and determined that many give a written report or study purported to be from a university more credibility than one from a think-tank or advocacy group.
Doberstein conducted a randomized controlled survey experiment involving British Columbia public service staff, asking them to read and assess the credibility of various policy studies. For half of the respondents, the authorship of the studies was randomly switched but the content remained the same.
Doberstein then compared the average credibility assessments between the control and experimental groups.
害羞草研究所淭here were systematic and at times extraordinarily large differences between the credibility assessments provided by these policy professionals on precisely the same policy studies, when the only part I changed was the label of who wrote it,害羞草研究所� said Doberstein.
害羞草研究所淚rrespective of the content and just by virtue of presenting it as written by an academic, the report suddenly becomes more credible in the eyes of bureaucrats.害羞草研究所�
The results surprised him, in part due to the magnitude of the differences observed. For one report, originally authored by the Fraser Institute, the credibility skyrocketed among study participants when they read the same document thinking it came from a university academic.
Another policy study, this time written by a university economist, received very high credibility assessments in the control group. But when authorship was changed to be purportedly written by the Canadian Centre for Policy Alternatives think-tank, its credibility plummeted dramatically.
害羞草研究所淧ut simply, the think-tank affiliation was a significant drag on the perceived credibility of their report and analysis,害羞草研究所� said Doberstein.
The same was true for reports said to be written by research-based advocacy groups.
"Some may interpret this finding positively,害羞草研究所� he said. 害羞草研究所淭hat analysts in government are skeptical of reports or studies that emerge from think tanks or advocacy organizations offering analysis and conclusions that tend to align with the organization害羞草研究所檚 obvious ideological position.害羞草研究所�
Yet Doberstein says having a report害羞草研究所檚 credibility increase simply by changing the name of the source is concerning as it can appear that policy-relevant research contained within its pages is being ignored by government policy advisors.
害羞草研究所淲e expect public servants to objectively examine the research evidence available to them,害羞草研究所� he said. 害羞草研究所淗owever, it seems many are taking shortcuts, and in essence giving academics a free pass.害羞草研究所�
And while this study examined the biases among policymakers in BC, Doberstein notes similar results were observed his subsequent replication experiment involving provincial policy analysts in Ontario, Saskatchewan and Newfoundland.
Doberstein害羞草研究所檚 study was recently published in Policy Studies Journal.
