Are responses to official consultations and stakeholder surveys reliable guides to policy actors’ positions?

Authors:
Karin Ingold University of Berne, Switzerland

Search for other papers by Karin Ingold in
Current site
Google Scholar
Close
,
Frédéric Varone University of Geneva, Switzerland

Search for other papers by Frédéric Varone in
Current site
Google Scholar
Close
,
Marlene Kammerer University of Berne, Switzerland

Search for other papers by Marlene Kammerer in
Current site
Google Scholar
Close
,
Florence Metz ETH Zurich, Switzerland

Search for other papers by Florence Metz in
Current site
Google Scholar
Close
,
Lorenz Kammermann University of Berne, Switzerland

Search for other papers by Lorenz Kammermann in
Current site
Google Scholar
Close
, and
Chantal Strotz University of Lucerne, Switzerland

Search for other papers by Chantal Strotz in
Current site
Google Scholar
Close
Restricted access
Get eTOC alerts
Rights and permissions Cite this article

Policy positions are used extensively to explain coalition formation, advocacy success and policy outputs, and government consultations and stakeholder surveys are seen as important means of gathering data about policy actors’ positions. However, we know little about how accurately official consultations and stakeholder surveys reflect their views. This study compares advocacy organisations’ publicly stated positions in their responses to official consultations with their positions expressed in confidential surveys conducted by the authors. It compares three decision-making processes in Switzerland – in energy, climate and water protection – to analyse responses via two different types of data gathering methods. The results show a substantial divergence between official and private expressions of policy positions. Specific types of policy actors (losers), instruments (persuasive measures) and subsystems (collaborative network) produce more divergent positions. This has important methodological implications for comparative policy studies that use different data gathering methods and focus on different policy domains.

  • Ansell, C. and Gash, A. (2007) Collaborative governance in theory and practice, Journal of Public Administration Research and Theory, 18(4): 54371. doi: 10.1093/jopart/mum032

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R. and Jones, B.D. (1991) Agenda dynamics and policy subsystems, Journal of Politics, 53(4): 104474. doi: 10.2307/2131866

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R., Berry, J.M., Hojnacki, M., Kimball, D.C. and Leech, B.L. (2009) Lobbying and policy change: Who wins, who loses, and why, Chicago, IL: University of Chicago.

    • Search Google Scholar
    • Export Citation
  • Belli, R.F., Traugott, M.W., Young, M and McGonagle, K.A. (1999) Reducing vote overreporting in surveys: Social desirability, memory failure, and source monitoring, Public Opinion Quarterly, 90108.

    • Search Google Scholar
    • Export Citation
  • Benoit, K. and Laver, M. (2009) Party policy in modern democracies: Transferred to digital print, Routledge Research in Comparative Politics, 19 i.e. 18, London: Routledge, www.loc.gov/catdir/enhancements/fy0654/2006014032-d.html

    • Search Google Scholar
    • Export Citation
  • Bidwell, D. (2016) The effects of information on public attitudes toward renewable energy, Environment and Behavior, 48(6): 74368. doi: 10.1177/0013916514554696

    • Search Google Scholar
    • Export Citation
  • Bräuninger, T., Debus, M. and Müller, J. (2013) Estimating policy positions of political actors across countries and time, Working Paper 153, Mannheim: Mannheimer Zentrum für Europäische Sozialforschung.

    • Search Google Scholar
    • Export Citation
  • Budge, I. and Klingemann, H.-D. (2010) Estimates for parties, electors, and governments, 1945– 1998, Mapping policy preferences, 1, Oxford: Oxford University Press.

    • Search Google Scholar
    • Export Citation
  • Bundi, P., Varone, F., Gava, R., and Widmer, T. (2018) Self-selection and misreporting in legislative surveys, Political Science Research and Methods, 6(4): 77189. doi: 10.1017/psrm.2016.35

    • Search Google Scholar
    • Export Citation
  • Bunea, A. (2012) Issues, preferences and ties: determinants of interest groups’ preference attainment in the EU environmental policy, Journal of European Public Policy, 20(4): 55270. doi: 10.1080/13501763.2012.726467

    • Search Google Scholar
    • Export Citation
  • Bunea, A. and Ibenskas, R. (2015) Quantitative text analysis and the study of EU lobbying and interest groups, European Union Politics, 16(3): 42955. doi: 10.1177/1465116515577821

    • Search Google Scholar
    • Export Citation
  • Cairney, P. and Jones, M.D. (2016) Kingdon’s multiple streams approach: what is the empirical impact of this universal theory?, Policy Studies Journal, 44(1): 3758. doi: 10.1111/psj.12111

    • Search Google Scholar
    • Export Citation
  • Calanni, J.C., Siddiki, S.N., Weible, C.M. and Leach, W.D. (2015) Explaining coordination in collaborative partnerships and clarifying the scope of the belief homophily hypothesis, Journal of Public Administration Research and Theory, 25(3), 90127. doi: 10.1093/jopart/mut080

    • Search Google Scholar
    • Export Citation
  • Coleman, J.S. (1974) Power and structure of society, New York: Norton.

  • Crowne, D.P. and Marlowe, D. (1960) A new scale of social desirability independent of psychopathology, Journal of Consulting Psychology, 24(4): 34954.

    • Search Google Scholar
    • Export Citation
  • DeMaio, D.J. (1984) Social desirability and survey measurement: A review, in C.E. Turner and E. Martin (ed), Surveying subjective phenomena, Thousand Oaks, CA: Sage, 25782.

    • Search Google Scholar
    • Export Citation
  • Dermont, C., Ingold, K., Kammermann, L. and Stadelmann-Steffen, I. (2017) Bringing the policy making perspective in: a political science approach to social acceptance, Energy Policy, 108: 35968. doi: 10.1016/j.enpol.2017.05.062.

    • Search Google Scholar
    • Export Citation
  • Fischer, M. (2014) Coalition structures and policy change in a consensus democracy, Policy Studies Journal, 42(3): 34465. doi: 10.1111/psj.12064

    • Search Google Scholar
    • Export Citation
  • Fischer, M., Ingold, K., Sciarini, P. and Varone, F. (2016) Dealing with bad guys: actor- and process-level determinants of the ‘devil shift’ in policy making, Journal of Public Policy, 36(2): 309334. doi: 10.1017/S0143814X15000021

    • Search Google Scholar
    • Export Citation
  • Guerrero, A.M., Bodin, Ö., McAllister, R.R.J., Wilson, K.A. (2015) Achieving social-ecological fit through bottom-up collaborative governance: an empirical investigation, Ecology and Society, 20(4). doi: 10.5751/ES-08035-200441

    • Search Google Scholar
    • Export Citation
  • Groves, R.M., Fowler, Jr, F.J., Couper, M.P., Lepkowski, J.M., Singer, E. and Tourangeau, R. (2011) Survey Methodology, 561.

  • Heikkila, T. and Gerlak, A.K. (2013) Building a conceptual approach to collective learning: lessons for public policy scholars, Policy Studies Journal, 41(3): 484512. doi: 10.1111/psj.12026

    • Search Google Scholar
    • Export Citation
  • Henry, A.D. (2011) Ideology, power, and the structure of policy networks, Policy Studies Journal, 39(3): 36183. doi: 10.1111/j.1541-0072.2011.00413.x

    • Search Google Scholar
    • Export Citation
  • Hill, M.J. and Varone, F. (2017) The public policy process (7th edn), London and New York: Routledge Taylor & Francis.

  • Howlett, M. and Lejano, R.P. (2013) Tales from the crypt, Administration & Society, 45(3): 35781. doi: 10.1177/0095399712459725

  • Ingold, K. (2011) Network structures within policy processes: coalitions, power, and brokerage in Swiss climate policy, Policy Studies Journal, 39(3): 43559. doi: 10.1111/j.1541-0072.2011.00416.x

    • Search Google Scholar
    • Export Citation
  • Ingold, K. and Christopoulos, D. (2015) The network of political entrepreneurs: a case study of Swiss climate policy, In I.N. Aflaki (ed), Entrepreneurship in the Polis, Farnham: Ashgate.

    • Search Google Scholar
    • Export Citation
  • Jourdain, C., Hug, S. and Varone, F. (2016) Lobbying across venues: an issue-tracing approach, State Politics & Policy Quarterly, 17(2): 12753. doi: 10.1177/1532440016672272

    • Search Google Scholar
    • Export Citation
  • Kahneman, D. and Tversky, A. (eds) (2009) Choices, values, and frames, 10 printing,  Cambridge: Cambridge University Press.

  • Kammermann, L. and Strotz, C. (2014) Akteure und Koalitionen in der Schweizer Energiepolitik nach Fukushima, Master Thesis, Bern: University of Bern.

    • Search Google Scholar
    • Export Citation
  • Klingemann, H.-D. (2008) Estimates for parties, electors, and governments in Eastern Europe, European Union and OECD 1990–2003, Mapping policy preferences, 2, Oxford: Oxford University Press.

    • Search Google Scholar
    • Export Citation
  • Klüver, H. (2009) Measuring interest group influence using quantitative text analysis, European Union Politics, 10(4): 53549. doi: 10.1177/1465116509346782

    • Search Google Scholar
    • Export Citation
  • Knill, C. and Tosun, J. (2012) Public policy: A new introduction, Basingstoke: Palgrave Macmillan.

  • Kutner, M.H., Nachtsheim, C., Neter, J. and Li, W. (2005) Applied linear statistical models (5th edn), Operations and Decision Sciences series, Boston, MA: McGraw-Hill Irwin.

    • Search Google Scholar
    • Export Citation
  • Landry, R. and Varone, F (2005) Choice of policy instruments: confronting the deductive and the interactive approaches, in P Eliadis, MM Hill, M Howlett (eds) Designing government: From instruments to governance, Montreal and Kingston, ON: McGill-Queen’s University Press.

    • Search Google Scholar
    • Export Citation
  • Lasswell, H.D. (1956) The decision process, College Park, MD: University of Maryland Press.

  • Laumann, E.O. and Knoke, D. (1987) The organizational state: Social choice in national policy domains, Madison, WI: University of Wisconsin Press (WIS-Edition).

    • Search Google Scholar
    • Export Citation
  • Leach, W.D. and Sabatier, P.A. (2005) To trust an adversary: integrating rational and psychological models of collaborative policymaking, American Political Science Review, 99(4): 491503. doi: 10.1017/S000305540505183X

    • Search Google Scholar
    • Export Citation
  • Leach, W.D., Weible, C.M., Vince, S.R., Siddiki, S.N. and Calanni, J.C. (2014) Fostering learning through collaboration: knowledge acquisition and belief change in marine aquaculture partnerships, Journal of Public Administration Research and Theory, 24(3): 591622. doi: 10.1093/jopart/mut011

    • Search Google Scholar
    • Export Citation
  • Linder, S. and Peters, G. (1989) Instruments of government: perceptions and contexts, Journal for Public Policy, 9(1): 3558. doi: 10.1017/S0143814X00007960

    • Search Google Scholar
    • Export Citation
  • Markard, J.O., Suter, M. and Ingold, K. (2016) Socio-technical transitions and policy change: advocacy coalitions in Swiss energy policy, Environmental Innovation and Societal Transitions, 18: 21537. doi: 10.1016/j.eist.2015.05.003

    • Search Google Scholar
    • Export Citation
  • Marks, G., Hooghe, L., Steenbergen, M.R. and Bakker, R. (2007) Crossvalidating data on party positioning on European integration, Electoral Studies, 26(1): 2338. doi: 10.1016/j.electstud.2006.03.007

    • Search Google Scholar
    • Export Citation
  • Metz, F. (2017) From network structure to policy design in water protection: A comparative perspective on micropollutants in the Rhine River Riparian countries, Cham: Springer International Publishing.

    • Search Google Scholar
    • Export Citation
  • Montpetit, É. and Lachapelle, E. (2015) Can policy actors learn from academic scientists?, Environmental Politics, 24(5): 66180. doi: 10.1080/09644016.2015.1027058

    • Search Google Scholar
    • Export Citation
  • Moyson, S. (2017) Cognition and policy change: the consistency of policy learning in the advocacy coalition framework, Policy and Society, 36(2): 32044. doi: 10.1080/14494035.2017.1322259

    • Search Google Scholar
    • Export Citation
  • Sabatier, P., Hunter, S. and McLaughlin, S. (1987) The devil shift: perceptions and misperceptions of opponents, Western Political Quarterly, 40(3): 44976. doi: 10.2307/448385

    • Search Google Scholar
    • Export Citation
  • Sabatier, P.A. (ed) (1999) Theories of the policy process, Boulder, CO: Westview Press.

  • Sabatier, P.A. and Jenkins-Smith, H.C. (1993) Policy change and learning: An advocacy coalition approach, Boulder, CO: Westview Press.

  • Sager, F. (2009) Governance and coercion, Political Studies, 57(3): 53758. doi: 10.1111/j.1467-9248.2008.00743.x

  • Schneider, A. and Ingram, H. (1993) Social construction of target populations: implications for politics and policy, American Political Science Review, 87(2): 33447. doi: 10.2307/2939044

    • Search Google Scholar
    • Export Citation
  • Slapin, J.B. and Proksch, S.-O. (2008) A scaling model for estimating time-series party positions from texts, American Journal of Political Science, 52(3): 70522. doi: 10.1111/j.1540-5907.2008.00338.x

    • Search Google Scholar
    • Export Citation
  • Steenbergen, M.R. and Jones, B.S. (2002) Modeling multilevel data structures, American Journal of Political Science, 46(1): 21837 . doi: 10.2307/3088424

    • Search Google Scholar
    • Export Citation
  • Tourangeau, R. and Yan, T. (2007) Sensitive questions in surveys, Psychological Bulletin, 133(5): 859883 .

  • Varone, F., Ingold, K. and Jourdain, C. (2017a) Defending the status quo across venues and coalitions: evidence from California interest groups, Journal of Public Policy, 37(1): 126. doi: 10.1017/S0143814X16000179

    • Search Google Scholar
    • Export Citation
  • Varone, F., Ingold, K., Jourdain, C. and Schneider, V. (2017b) Studying policy advocacy through social network analysis, European Political Science, 16(3): 32236. doi: 10.1057/eps.2016.16

    • Search Google Scholar
    • Export Citation
  • Vedung, E. (1998) Policy instruments: typologies and theories, In M.-L. Bemelmans-Videc, R.C. Rist, E. Vedung (eds) Carrots, sticks and sermons: Policy instruments and their evaluation, New Brunswick and London: Transaction Publishers

    • Search Google Scholar
    • Export Citation
  • Weible, C.M., Sabatier, P.A. and McQueen, K. (2009) Themes and variations: taking stock of the advocacy coalition framework, Policy Studies Journal, 37(1): 12140. doi: 10.1111/j.1541-0072.2008.00299.x

    • Search Google Scholar
    • Export Citation
  • Weible, C.M., Sabatier, P.A., Jenkins-Smith, H.C., Nohrstedt, D., Henry, A.D. and deLeon, P. (2011) A quarter century of the advocacy coalition framework: an introduction to the special issue, Policy Studies Journal, 39(3): 34960. doi: 10.1111/j.1541-0072.2011.00412.x

    • Search Google Scholar
    • Export Citation
  • Weible, C., Ingold, K., Nohrstedt, D., Henry, A.D. and Jenkins-Smith, H.C. (2019) Sharpening Advocacy Coalitions, Policy Studies Journal, First published: 28 June 2019. doi: 10.1111/psj.12360

    • Search Google Scholar
    • Export Citation
  • Wilder, M. (2015) What is a policy paradigm? Overcoming epistemological hurdles in cross-dispciplinary conceputal adaptation, In J. Hogan (ed) Policy paradigms in theory and practice: Discourses, ideas and anomalies in public policy dynamics, Studies in the Political Economy of Public Policy, London: Palgrave Macmillan UK, pp. 1942.

    • Search Google Scholar
    • Export Citation
Karin Ingold University of Berne, Switzerland

Search for other papers by Karin Ingold in
Current site
Google Scholar
Close
,
Frédéric Varone University of Geneva, Switzerland

Search for other papers by Frédéric Varone in
Current site
Google Scholar
Close
,
Marlene Kammerer University of Berne, Switzerland

Search for other papers by Marlene Kammerer in
Current site
Google Scholar
Close
,
Florence Metz ETH Zurich, Switzerland

Search for other papers by Florence Metz in
Current site
Google Scholar
Close
,
Lorenz Kammermann University of Berne, Switzerland

Search for other papers by Lorenz Kammermann in
Current site
Google Scholar
Close
, and
Chantal Strotz University of Lucerne, Switzerland

Search for other papers by Chantal Strotz in
Current site
Google Scholar
Close

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 1848 997 144
Full Text Views 238 15 0
PDF Downloads 160 13 0

Altmetrics

Dimensions

Policy & Politics
Advancing knowledge in public and social policy