The silver bullet reversed: the impact of evidence on policymaker attention

Author:
Leire Rincón García University of Barcelona and IBEI, Spain

Search for other papers by Leire Rincón García in
Current site
Google Scholar
Close
Restricted access
Get eTOC alerts
Rights and permissions Cite this article

Background:

Despite the growing attention given to the political process of evidence-based policymaking (EBPM), we still know little about how evidence is processed at the early stages of the policymaking process, especially at the agenda-setting stage. Whether and when political elites pay attention to evidence-based information is crucial to the study of EBPM but also essential to the well-functioning of democracy.

Aims and objectives:

The aim of this paper is to cover this gap, by asking whether evidence increases policymaker attention to policy proposals. The working hypothesis is that everything else being constant, evidence should increase policy-maker attention.

Methods:

To test this hypothesis, this paper relies on a field experiment embedded in a real-life fundraising campaign of an advocacy organisation targeted at the Members of the European Parliament (MEPs). The field experiment is embedded in a real-life fundraising campaign of an advocacy organisation targeted at the Members of the European Parliament (MEPs).

Findings:

Results show that information type matters to policy-maker attention, but evidence is not effective in this respect. Findings also suggest that there are no important differences between political groups and, crucially, that previous policy support does not have an impact on policy-maker attention. This paper shows that that while evidence is essential to the policy process, ideas are key to attract policymakers’ attention at the individual level in the absence of prior demand.

Discussion and conclusion:

Overall, findings suggest that empirical information is not a quick pass for policy-maker attention. In this context, other types of information and framing are likely to make a difference. Future studies should analyse how framing may alter political elites’ predisposition to attend empirical evidence.

  • Baesler, J.E. and Burgoon, J.K. (1994) The temporal effects of story and statistical evidence on belief change, Communication Research, 21(5): 582602.

    • Search Google Scholar
    • Export Citation
  • Barker, D.C. (2005) Values, frames, and persuasion in presidential nomination campaigns, Political Behavior, 27(4): 37594.

  • Baumgartner, F.R., De Boef, S. and Boydstun, A. (2008) The Decline of the Death Penalty and the Discovery of Innocence, Cambridge: Cambridge University Press.

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R. and Jones, B. (2015) The Politics of Information: Problem Definition and the Course of Public Policy in America, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • BIG Report (2009) Basic Income Grant Pilot Project Assessment Report, ISBN: 978-99916-842-4-6.

  • Borrás, S. and Højlund, S. (2015), Evaluation and policy learning, European Journal of Political Research, 54: 99-120, doi:10.1111/1475-6765.12076.

    • Search Google Scholar
    • Export Citation
  • Bundi, P. (2016) What Do We Know About the Demand for Evaluation? Insights From the Parliamentary Arena, American Journal of Evaluation, 37(4): 522541, doi: 10.1177/1098214015621788.

    • Search Google Scholar
    • Export Citation
  • Bouwen, P. (2002) Corporate lobbying in the European Union: the logic of access, Journal of European Public Policy, 9(3): 36590.

  • Bouwen, P. (2004) Exchanging access goods for access: a comparative study of business lobbying in the European Union Institutions, European Journal of Political Research, 43(3): 33769.

    • Search Google Scholar
    • Export Citation
  • Butler, D.M. and Broockman, D.E. (2011) Do politicians racially discriminate against constituents? A field experiment on state legislators, American Journal of Political Science, 55(3): 46377.

    • Search Google Scholar
    • Export Citation
  • Butler, D.M. (2014) Representing the Advantaged: How Politicians Reinforce Inequality, Cambridge: Cambridge University Press.

  • Butler, D., Volden, C., Dynes, A. and Shor, B. (2017) Ideology and learning in policy diffusion: experimental evidence, American Journal of Political Science, 61(1): 3749.

    • Search Google Scholar
    • Export Citation
  • Cairney, P. (2016) The Politics of Evidence-Based Policy Making, London: Palgrave Macmillan.

  • Cairney, P. and Zahariadis, N. (2016) Multiple streams approach: a flexible metaphor presents an opportunity to operationalize agenda setting processes, in N. Zahariadis and M. Buckman (eds) Handbook of Public Policy and Agenda-Setting, Cheltenham: Edward Elgar.

    • Search Google Scholar
    • Export Citation
  • Ceron, A. and Negri, F. (2016) The ‘social side’ of public policy: monitoring online public opinion and its mobilization during the policy cycle, Policy & Internet, 8(2): 13147.

    • Search Google Scholar
    • Export Citation
  • Chaiken, S. and Trope, Y. (eds) (1999) Dual-Process Theories in Social Psychology, New York: Guilford Press.

  • Chaqués-Bonafont, L. (2019) The agenda setting capacity of global networks, in D. Stone and K. Moloney (eds) Oxford Handbook of Global Policy and Transnational Administration, Oxford: Oxford University Press.

    • Search Google Scholar
    • Export Citation
  • Dahlstrom, M.F. (2010) The role of causality in information acceptance in narratives: an example from science communication, Communication Research, 37(6): 85775.

    • Search Google Scholar
    • Export Citation
  • Dalia Research (2017) 31% of Europeans Want Basic Income as Soon as Possible, https://daliaresearch.com/31-of-europeans-want-basic-income-as-soon-as-possible/.

    • Search Google Scholar
    • Export Citation
  • Druckman, J.N. and Bolsen, T. (2011) Framing, motivated reasoning, and opinions about emergent technologies, Journal of Communication, 61(4): 65988.

    • Search Google Scholar
    • Export Citation
  • Duchon, D., Dunegan, K.J. and Barton, S.L (1989) Framing the problem and making decisions: the facts are not enough, IEE Transactions on Engineering Management, 36(1): 2527.

    • Search Google Scholar
    • Export Citation
  • Entman, R.M. (1993) Framing: toward clarification of a fractured paradigm, Journal of Communication, 43(4): 5158.

  • Forget, E. (2011) The town with no poverty: the health effects of a Canadian guaranteed annual income field experiment, Canadian Public Policy, 37(3): 283305.

    • Search Google Scholar
    • Export Citation
  • Gaines, B.J., Kuklinski, J.H., Quirk, P.J., Peyton, B. and Verkuilen, J. (2007) Same facts, different interpretations: partisan motivation and opinion on Iraq, Journal of Politics, 69(4): 95774.

    • Search Google Scholar
    • Export Citation
  • Gastel, B. (1983) Presenting Science to the Public, Philadelphia, PA: iSi Press.

  • Gerber, A.S. and Green, D.P. (2000) The effects of canvassing, telephone calls, and direct mail on voter turnout: a field experiment, American Political Science Review, 94(3): 65363.

    • Search Google Scholar
    • Export Citation
  • Gerber, A.S. and Green, D.P. (2012) Field Experiments: Design, Analysis, and Interpretation, New York: W. W. Norton.

  • Gilardi, F. (2010) Who learns from what in policy diffusion processes?, American Journal of Political Science, 54(3): 65066.

  • Gilardi, F. and Wasserfallen, F. (2017) Policy Diffusion: Mechanisms and Practical Implications, Working Paper, https://www.fabriziogilardi.org/resources/papers/Gilardi-Wasserfallen-2017.pdf.

    • Search Google Scholar
    • Export Citation
  • Gray, V. (1973) Innovation in the States: a diffusion study, American Political Science Review, 67(04): 117485.

  • Hall, R. and Deardorff, A.V. (2006) Lobbying as legislative subsidy, American Political Science Review, 100(1): 6984.

  • Hardie, J. and Cartwright, N. (2012) Evidence-Based Policy: A Practical Guide to Doing It Better, Oxford: Oxford University Press.

  • Heinze, T. (2011) Mechanism-based thinking on policy diffusion: a review of current approaches in political science, KFG Working Papers, 34, https://www.polsoz.fu-berlin.de/en/v/transformeurope/publications/working_paper/wp/wp34/WP_34_Heinze.pdf.

    • Search Google Scholar
    • Export Citation
  • Holmes, A. (2017) 31% of Europeans Want Basic Income as Soon as Possible, Dalia, May 3, https://daliaresearch.com/31-of-europeans-want-basic-income-as-soon-as-possible/.

    • Search Google Scholar
    • Export Citation
  • Jacob, S., Speer, S., and Furubo, J.-E. (2015) The institutionalization of evaluation matters: Updating the international atlas of evaluation 10 years later, Evaluation, 21: 631.

    • Search Google Scholar
    • Export Citation
  • Jones, B.D. (2017) Behavioral rationality as a foundation for public policy studies, Cognitive Systems Research, 43: 6375.

  • Jones, B.D. and Baumgartner, F. (2005) The Politics of Attention, Chicago, IL: University of Chicago Press.

  • Kahan, D., Braman, D., Slovic, P., Gastil, J. and Cohen, G. (2008) Cultural cognition of the risks and benefits of nanotechnology, Nature Nanotechnology, 43: 6375.

    • Search Google Scholar
    • Export Citation
  • Kangas, O.E., Niemelä, M. and Varjonen, S. (2014) When and why do ideas matter? The influence of framing on opinion formation and policy change, European Political Science Review, 6(1): 7392.

    • Search Google Scholar
    • Export Citation
  • Kazoleas, D. (1993) The impact of argumentativeness on resistance to persuasion, Journal of Human Communication, 20(1): 11837.

  • Kingdon, J.W. (1984) Agendas, Alternatives, and Public Policies, Boston, MA: Little, Brown.

  • Kopfman, J.E., Smith, S.W., Ah Yun, J.K. and Hodges, A. (1998) Affective and cognitive reactions to narrative versus statistical evidence organ donation messages, Journal of Applied Communication Research, 26(3): 279300.

    • Search Google Scholar
    • Export Citation
  • Kunda, Z. (1990) The case for motivated reasoning, Psychological Bulletin, 108(3): 48098.

  • Lakoff, G. (2004) Don't Think of an Elephant! Know Your Values and Frame the Debate, White River Junction, VT: Chelsea Green.

  • Liu, B.S. and Ditto, P.H. (2013) What Dilemma? Moral evaluation shapes factual belief, Social Psychological and Personality Science, 4(3): 31623.

    • Search Google Scholar
    • Export Citation
  • Lodge, M., and Taber, C.S. (2000) Three steps toward a theory of motivated political reasoning, in A. Lupia, M. McCubbins and S. Popkin (Eds.), Elements of reason: Cognition, choice, and the bounds of rationality, New York: Cambridge University Press, pp. 183-213.

    • Search Google Scholar
    • Export Citation
  • Lomas, J. and Brown, D. (2009) Research and advice giving: a functional view of evidence-informed policy advice in a Canadian ministry of health, Milbank Quarterly, 87(4): 90326.

    • Search Google Scholar
    • Export Citation
  • Lord, C.G., Ross, L. and Lepper, M.R. (1979) Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence, Journal of Personality and Social Psychology, 37(11): 2098109.

    • Search Google Scholar
    • Export Citation
  • Mondak, J. (1993) Source cues and policy approval: the cognitive dynamics of public support for the Reagan agenda, American Journal of Political Science, 37(1): 186212.

    • Search Google Scholar
    • Export Citation
  • Nisbet, M.C. and Mooney, C. (2007) Science and society: framing science, Science, 316(5821): 56.

  • Nutley, S., Walter, I. and Davies, H. (2007) Using Evidence: How Research Can Inform Public Services, Bristol: Policy Press.

  • Obinger, H., Schmitt, C. and Starke, P. (2013) Policy diffusion and policy transfer in comparative welfare state research, Social Policy and Administration, 47(1): 11129.

    • Search Google Scholar
    • Export Citation
  • O’Brien, D. (2013) Drowning the deadweight in the rhetoric of economism: what sport policy, free swimming, and EMA tell us about public services after the crash, Public Administration, 91(1): 6982.

    • Search Google Scholar
    • Export Citation
  • Reinard, J.C. (1988) The empirical study of the persuasive effects of evidence: the status after fifty years of research, Human Communication Research, 15(1): 359.

    • Search Google Scholar
    • Export Citation
  • Richardson, L. and John, P. (2012) Who listens to the grass roots? A field experiment on informational lobbying in the UK, British Journal of Politics and International Relations, 14(4): 595612.

    • Search Google Scholar
    • Export Citation
  • Scheufele, D.A. and Lewenstein, B.V. (2005) The public and nanotechnology: how citizens make sense of emerging technologies, Journal of Nanoparticle Research, 7: 65967.

    • Search Google Scholar
    • Export Citation
  • Sevenans, J. (2017) Why Political Elites Respond to the Media: the Micro-Level Mechanisms Underlying Political Agenda-Setting Effects, PhD Thesis, Antwerpen: Universiteit Antwerpen.

    • Search Google Scholar
    • Export Citation
  • Sevenans, J., Walgrave, S. and Gwendolyn, J.E. (2016) How political elites process information from the news: the cognitive mechanisms behind behavioral political agenda-setting effects, Political Communication, 33(4): 60527.

    • Search Google Scholar
    • Export Citation
  • Sevenans, J., Walgrave, S. and Vos, D. (2015) Political elites’ media responsiveness and their individual political goals: a study of national politicians in Belgium, Research and Politics, 2(3): 17.

    • Search Google Scholar
    • Export Citation
  • Sides, J. (2015) Stories or science? Facts, frames, and policy attitudes, American Politics Research, 44(3): 387414.

  • Simon, H.A. (1985) Human Nature in Politics: The Dialogue of Psychology with Political Science, 79(2): 293304.

  • Speer, S., Pattyn, V., and DePeuter, B. (2015) The growing role of evaluation in parliaments: Holding governments accountable, International Review of Administrative Sciences, 81: 3757.

    • Search Google Scholar
    • Export Citation
  • Stone, D.A. (1989) Causal stories and the formation of policy agendas, Political Science Quarterly, 104(2): 281300.

  • Taber, C.S., Cann, D. and Kucsova, S. (2009) The motivated processing of political arguments, Political Behavior, 31(2): 13755.

  • Tal, A. and Wansink, B. (2014) Blinded with science: trivial graphs and formulas increase persuasiveness and belief in product efficacy, Public Understanding of Science, 25(1): 11725.

    • Search Google Scholar
    • Export Citation
  • Taylor, S.E. and Thompson, S.C. (1982) Stalking the elusive ‘vividness’ effect, Psychological Review, 89(2): 15581.

  • Tufte, E. (2001) The Visual Display of Quantitative Information, Cheshire, CT: Graphics Press.

  • Vis, B. (2019) Heuristics and political elites’ judgment and decision-making, Political Studies Review, 17(1): 4152.

  • Vries, C., Dinas, E. and Solaz, H. (2016) You have got mail! How intrinsic and extrinsic motivations shape legislator responsiveness, Working paper 140, IHS Political Science Series.

    • Search Google Scholar
    • Export Citation
  • Walgrave, S., Sevenans, J., Van Camp, K. and Loewen, P. (2018) What draws politicians’ attention? An experimental study of issue framing and its effect on individual political elites, Political Behavior, 40(3): 54769.

    • Search Google Scholar
    • Export Citation
  • Walter, I., Nutley, S.M. and Davies, H.T.O. (2005) What works to promote evidence-based practice? A cross-sector review, Evidence & Policy, 1(3): 33564.

    • Search Google Scholar
    • Export Citation
  • Ward, V., House, A. and Hamer, S. (2009) Knowledge brokering: exploring the process of transferring knowledge into action, BMC Health Services Research, 16(9): 12.

    • Search Google Scholar
    • Export Citation
  • Weiss, C.H. (1995) Have we learned anything new about the use of evaluation? American Journal of Evaluation, 19(1): 2133.

  • Wood, M. (2015) Depoliticisation, resilience and the herceptin post-code lottery crisis: holding back the tide, British Journal of Politics and International Relations, 17(4): 64464.

    • Search Google Scholar
    • Export Citation
  • Zahariadis, N. (2016) Setting the agenda on agenda setting: definitions, concepts, and controversies, in N. Zahariadis and M. Buckman (eds) Handbook of Public Policy and Agenda-Setting, Cheltenham: Edward Elgar.

    • Search Google Scholar
    • Export Citation
  • Zebregs, S., Van den Putte, B., Neijens, P. and De Graaf, A. (2014) The differential impact of statistical and narrative evidence on beliefs, attitude, and intention: a meta-analysis, Health Communication, 30(3): 28289.

    • Search Google Scholar
    • Export Citation
Leire Rincón García University of Barcelona and IBEI, Spain

Search for other papers by Leire Rincón García in
Current site
Google Scholar
Close

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 408 79 45
Full Text Views 597 383 2
PDF Downloads 107 53 3

Altmetrics

Dimensions

You are not currently authorised to access the full text of this chapter or article.
Access options
To access the full chapter or article then please choose one of the options below.
Purchase
Pay to access content (PDF download and unlimited online access)
Other access options
Redeem Token
Institutional Login
Log in via Open Athens or Shibboleth. Please contact your librarian if you need any help.
Login with Institutional Access
Personal Login
Login to your BUP account with your individual credentials.
Login with BUP account

Institutional librarians can find more information about free trials here

Evidence & Policy
A journal of research, debate and practice