The extent of use of surveys in policymaking: the case of Hong Kong

Authors:
Christian Joy Pattawi Cruz University of the Philippines, Philippines

Search for other papers by Christian Joy Pattawi Cruz in
Current site
Google Scholar Close
,
Kira Matus Hong Kong University of Science and Technology, Hong Kong SAR

Search for other papers by Kira Matus in
Current site
Google Scholar Close
, and
Stuart Gietel-Basten Hong Kong University of Science and Technology, Hong Kong SAR

Search for other papers by Stuart Gietel-Basten in
Current site
Google Scholar Close
Article Category:
Research Article
Copyright:
© Authors 2024
Online Publication Date:
23 Nov 2023
Pages:
261–282
Publisher:
Policy Press
Volume/Issue:
Volume 20: Issue 3
DOI:
https://doi.org/10.1332/17442648Y2023D000000010
Restricted access
Get eTOC alerts
Rights and permissions Cite this article
Buy article/chapter (
GBP  £25.00
)

Background:

There are limited studies focused on examining specific types of evidence, like surveys beyond the US and territories with unicameral legislatures and unique contexts.

Aims and objectives:

To measure the extent of survey research being used as evidence in policymaking in Hong Kong.

Methods:

Through document analysis, this study screened and examined Hong Kong Legislative Council documents utilised to enact 569 bills from 2000 to 2022.

Findings:

About 25% of bills utilised surveys as evidence, with differences across 18 policy areas. Health services recorded the highest percentage of survey use in legislation. In the Hong Kong legislature, surveys are primarily used to understand policy issues better. Mode of data collection, sample size, response rates, and representativeness of surveys are not commonly discussed in legislative documents.

Discussion and conclusion:

The study findings reaffirm previous research on the limited utilisation of survey evidence in policymaking in Hong Kong, an Asian context with a unicameral legislation and colonial history. The importance of survey evidence was highlighted in policy areas that directly impact the public, such as healthcare. The findings also highlight the important role of politics in investigating the use of surveys as research evidence for policymaking.

  • Apollonio, D.E. and Bero, L.A. (2017) Interpretation and use of evidence in state policymaking: a qualitative analysis, BMJ Open, 7(2): e012738. doi: 10.1136/bmjopen-2016-012738

    • Search Google Scholar
    • Export Citation

  • Behague, D., Tawiah, C., Rosato, M., Télésphore, S. and Morrison, J. (2009) Evidence-based policy-making: the implications of globally-applicable research for context-specific problem-solving in developing countries, Social Science & Medicine, 69(10): 1539–46. doi: 10.1016/j.socscimed.2009.08.006

    • Search Google Scholar
    • Export Citation

  • Boehm, M., Bowman, D. and Zinn, J.O. (2013) Survey research and the production of evidence for social policy, Social Policy and Society, 12(2): 309–18. doi: 10.1017/s1474746412000668

    • Search Google Scholar
    • Export Citation

  • Bowen, S. and Zwi, A.B. (2005) Pathways to ‘evidence-informed’ policy and practice: a framework for action, PLoS Medicine, 2(7): e166. doi: 10.1371/journal.pmed.0020166

    • Search Google Scholar
    • Export Citation

  • Burns, P.B., Rohrich, R.J. and Chung, K.C. (2011) The levels of evidence and their role in evidence-based medicine, Plastic and Reconstructive Surgery, 128(1): 305–10. doi: 10.1097/PRS.0b013e318219c171

    • Search Google Scholar
    • Export Citation

  • Cairney, P. (2019) The UK government’s imaginative use of evidence to make policy, British Politics, 14: 1–22. doi: 10.1057/s41293-017-0068-2

    • Search Google Scholar
    • Export Citation

  • Cheung, D.Y.T., Wang, M.P., Ho, S.Y., Lee, P.H., Kwong, A.C.S., Lai, V.W.Y. and Lam, T.H. (2018) Report on Tobacco Control Policy-related Survey 2017. Hong Kong Council on Smoking and Health. COSH Report No. 24. https://www.smokefree.hk/uploadedFile/COSHRN_E24.txt

    • Search Google Scholar
    • Export Citation

  • Department of Justice (2012) How Legislation is Made in Hong Kong: A Drafter’s View of the Process, Hong Kong: Law Drafting Division, Department of Justice. https://www.doj.gov.hk/en/publications/pdf/drafter_book_2012_e.pdf.

    • Search Google Scholar
    • Export Citation

  • Desmarais, B.A. and Hird, J.A. (2014) Public policy’s bibliography: the use of research in regulatory impact analyses, Regulation & Governance, 8(4): 497–510. doi: 10.1111/rego.12041

    • Search Google Scholar
    • Export Citation

  • Dillman, D.A., Smyth, J.D. and Christian, L.M. (2014). Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, Hoboken, NJ: John Wiley & Sons. doi: 10.2307/41061275

    • Search Google Scholar
    • Export Citation

  • Duncan, S. and Oliver, S. (2017) Editorial, Research for All, 1(1): 1–5. doi: 10.18546/RFA.01.1.01

  • El-Jardali, F., Lavis, J.N., Ataya, N., Jamal, D., Ammar, W. and Raouf, S. (2012) Use of health systems evidence by policymakers in eastern Mediterranean countries: views, practices, and contextual influences, BMC Health Services Research, 12: 200. doi: 10.1186/1472-6963-12-200

    • Search Google Scholar
    • Export Citation

  • Ellen, M.E., Lavis, J.N., Horowitz, E. and Berglas, R. (2018) How is the use of research evidence in health policy perceived? A comparison between the reporting of researchers and policy-makers, Health Research Policy and Systems, 16(1): 64. doi: 10.1186/s12961-018-0345-6

    • Search Google Scholar
    • Export Citation

  • Evans, J.R. and Mathur, A. (2018) The value of online surveys: a look back and a look ahead, Internet Research, 28(4): 854–87. doi: 10.1108/intr-03-2018-0089

    • Search Google Scholar
    • Export Citation

  • Farley-Ripple, E.N., Oliver, K. and Boaz, A. (2020) Mapping the community: use of research evidence in policy and practice, Humanities and Social Sciences Communications, 7: 83. doi: 10.1057/s41599-020-00571-2

    • Search Google Scholar
    • Export Citation

  • Glasziou, P. and Chalmers, I. (2018) Research waste is still a scandal: an essay by Paul Glasziou and Iain Chalmers, BMJ, 363: k4645. doi: 10.1136/bmj.k4645

    • Search Google Scholar
    • Export Citation

  • Gollust, S.E., Kite, H.A., Bennong, S.J., Callanan, R.A., Weisman, S.R. and Nanney, M.S. (2014) Use of research evidence in state policymaking for childhood obesity prevention in Minnesota, American Journal of Public Health, 104(10): 1894–1900. doi: 10.2105/AJPH.2014.302137

    • Search Google Scholar
    • Export Citation

  • Haselswerdt, J. and Rigby, E. (2021) What do advocates want from policy research? Evidence from elite surveys, Evidence & Policy, 17(3): 385–403. doi: 10.1332/174426420x16000978959673

    • Search Google Scholar
    • Export Citation

  • Hastak, M., Mazis, M.B. and Morris, L.A. (2001) The role of consumer surveys in public policy decision making, Journal of Public Policy & Marketing, 20(2): 170–85. https://www.jstor.org/stable/30000586

    • Search Google Scholar
    • Export Citation

  • Haynes, A., Turner, T., Redman, S., Milat, A.J. and Moore, G. (2015) Developing definitions for a knowledge exchange intervention in health policy and program agencies: reflections on process and value, International Journal of Social Research Methodology, 18(2): 145–59. doi: 10.1080/13645579.2014.921457

    • Search Google Scholar
    • Export Citation

  • Head, B.W. (2010) Reconsidering evidence-based policy: key issues and challenges, Policy and Society, 29(2): 77–94. doi: 10.1016/j.polsoc.2010.03.001

    • Search Google Scholar
    • Export Citation

  • Head, B.W., Ferguson, M., Cherney, A. and Boreham, P. (2014) Are policy-makers interested in social research? Exploring the sources and uses of valued information among public servants in Australia, Policy and Society, 33(2): 89–101. doi: 10.1016/j.polsoc.2014.04.004

    • Search Google Scholar
    • Export Citation

  • Henderson, G., Hertel-Fernandez, A., Mildenberger, M. and Stokes, L.C. (2021) Conducting the heavenly chorus: constituent contact and provoked petitioning in Congress, Perspectives on Politics, 21(1): 191–208. doi: 10.1017/S1537592721000980

    • Search Google Scholar
    • Export Citation

  • Herbst, S. (1993) Numbered Voices: How Opinion Polling Has Shaped American Politics, Chicago, IL: University of Chicago Press.

  • Hong Kong Legislative Council (2022) Issues in particular policy areas, https://www.legco.gov.hk/en/education/teach/legco_panel_papers.html.

  • Jakobsen, M.W., Lau, C.J., Skovgaard, T., Hämäläinen, R.M. and Aro, A.R. (2018) Use of research evidence in policymaking in three Danish municipalities, Evidence & Policy, 14(4): 589–611. doi: 10.1332/174426417x14982331542543

    • Search Google Scholar
    • Export Citation

  • Jamieson, K.H. et al. (2023) Protecting the integrity of survey research, PNAS Nexus, 2(3): 1–10. doi: 10.1093/pnasnexus/pgad049

  • Kastellec, J.P. and Leoni, E.L. (2007) Using graphs instead of tables in political science, Perspectives on Politics, 5(4): 755–71. http://www.jstor.org/stable/20446574.

    • Search Google Scholar
    • Export Citation

  • Lavis, J.N., Oxman, A.D., Lewin, S. and Fretheim, A. (2009) SUPPORT Tools for evidence-informed health policymaking (STP), Health Research Policy and Systems, 7: I1. doi: 10.1186/1478-4505-7-S1-I1

    • Search Google Scholar
    • Export Citation

  • Lee, F. (2020) Solidarity in the Anti-Extradition Bill movement in Hong Kong, Critical Asian Studies, 52(1): 18–32. doi: 10.1080/14672715.2020.1700629

    • Search Google Scholar
    • Export Citation

  • Makkar, S.R., Williamson, A., Turner, T., Redman, S. and Louviere, J. (2015) Using conjoint analysis to develop a system of scoring policymakers’ use of research in policy and program development, Health Research Policy and Systems, 13: 35. doi: 10.1186/s12961-015-0022-y

    • Search Google Scholar
    • Export Citation

  • Masood, S., Kothari, A. and Regan, S. (2020) The use of research in public health policy: a systematic review, Evidence & Policy, 16(1): 7–43. doi: 10.1332/174426418x15193814624487

    • Search Google Scholar
    • Export Citation

  • Mayne, R., Green, D., Guijt, I., Walsh, M., English, R. and Cairney, P. (2018) Using evidence to influence policy: Oxfam’s experience, Palgrave Communications, 4: 122. doi: 10.1057/s41599-018-0176-7

    • Search Google Scholar
    • Export Citation

  • McHugh, M.L. (2012) Interrater reliability: the kappa statistic, Biochemia Medica, 22(3): 276–82. doi: 10.11613/BM.2012.031

  • Mitchell, J. (2007) The use (and misuse) of surveys in policy analysis, in F. Fischer and G.J. Miller (eds) Handbook of Public Policy Analysis, New York: Routledge, pp 369–79. doi: 10.4324/9781315093192

    • Search Google Scholar
    • Export Citation

  • Mosley, J.E. and Gibson, K. (2017) Strategic use of evidence in state-level policymaking: matching evidence type to legislative stage, Policy Sciences, 50: 697–719. doi: 10.1007/s11077-017-9289-x

    • Search Google Scholar
    • Export Citation

  • Oliver, K. and Boaz, A. (2019) (2019) Transforming evidence for policy and practice: creating space for new conversations, Palgrave Communications, 5: 60. doi: 10.1057/s41599-019-0266-1

    • Search Google Scholar
    • Export Citation

  • Oliver, K. and Cairney, P. (2019) The dos and don’ts of influencing policy: a systematic review of advice to academics, Palgrave Communications, 5: 21. doi: 10.1057/s41599-019-0232

    • Search Google Scholar
    • Export Citation

  • Onwujekwe, O., Uguru, N., Russo, G., Etiaba, E., Mbachu, C., Mirzoev, T. and Uzochukwu, B. (2015) Role and use of evidence in policymaking: an analysis of case studies from the health sector in Nigeria, Health Research Policy and Systems, 13: 46. doi: 10.1186/s12961-015-0049-0

    • Search Google Scholar
    • Export Citation

  • Orton, L., Lloyd-Williams, F., Taylor-Robinson, D., O’Flaherty, M. and Capewell, S. (2011) The use of research evidence in public health decision making processes: systematic review, PLoS One, 6(7): e21704. doi: 10.1371/journal.pone.0021704

    • Search Google Scholar
    • Export Citation

  • Ouimet, M., Beaumier, M., Cloutier, A., Côté, A., Montigny, É., Gélineau, F., Jacob, S. and Ratté, S. (2023) Use of research evidence in legislatures: a systematic review, Evidence & Policy, Early view, 1–18. doi: 10.1332/174426421x16656568731041

    • Search Google Scholar
    • Export Citation

  • Parkhurst, J.O. and Abeysinghe, S. (2016) What constitutes ‘good’ evidence for public health and social policy-making? From hierarchies to appropriateness, Social Epistemology, 30(5–6): 665–79. doi: 10.1080/02691728.2016.1172365

    • Search Google Scholar
    • Export Citation

  • Rickinson, M., de Bruin, K., Walsh, L. and Hall, M. (2017) What can evidence-use in practice learn from evidence-use in policy?, Educational Research, 59(2): 173–89. doi: 10.1080/00131881.2017.1304306

    • Search Google Scholar
    • Export Citation

  • Salajan, A., Tsolova, S., Ciotti, M. and Suk, J.E. (2020) To what extent does evidence support decision making during infectious disease outbreaks? A scoping literature review, Evidence & Policy, 16(3): 453–75. doi: 10.1332/174426420x15808913064302

    • Search Google Scholar
    • Export Citation

  • Schlaufer, C. (2018) The narrative uses of evidence, Policy Studies Journal, 46(1): 90–118. doi: 10.1111/psj.12174

  • Scott, I. and Gong, T. (2015) Evidence-based policy-making for corruption prevention in Hong Kong: a bottom-up approach, Asia Pacific Journal of Public Administration, 37(2): 87–101. doi: 10.1080/23276665.2015.1041222

    • Search Google Scholar
    • Export Citation

  • Shaxson, L. (2005) Is your evidence robust enough? Questions for policy makers and practitioners, Evidence & Policy, 1(1): 101–111. doi: 10.1332/1744264052703177

    • Search Google Scholar
    • Export Citation

  • Sohn, J. (2018) Navigating the politics of evidence-informed policymaking: strategies of influential policy actors in Ontario, Palgrave Communications, 4: 49. doi: 10.1057/s41599-018-0098-4

    • Search Google Scholar
    • Export Citation

  • Strydom, W.F., Funke, N., Nienaber, S., Nortje, K. and Steyn, M. (2010) Evidence-based policymaking: a review, South African Journal of Science, 106(5–6): 1–8.

    • Search Google Scholar
    • Export Citation

  • Tseng, V. and Coburn, C. (2019) Using evidence in the US, in A. Boaz, H. Davies, A. Fraser and S. Nutley (eds) What Works Now? Evidence Informed Policy and Practice, Bristol: Policy Press, pp 351–68.

    • Search Google Scholar
    • Export Citation

  • Walugembe, D.R., Kiwanuka, S.N., Matovu, J.K.B., Rutebemberwa, E. and Reichenbach, L. (2015) Utilization of research findings for health policy making and practice: evidence from three case studies in Bangladesh, Health Research Policy and Systems, 13: 26. doi: 10.1186/s12961-015-0015-x

    • Search Google Scholar
    • Export Citation

  • Wardle, M. and Mainelli, M. (2022) The global financial centres index 31, https://www.longfinance.net/programmes/financial-centre-futures/global-financial-centres-index/gfci-publications/global-financial-centres-index-31/.

    • Search Google Scholar
    • Export Citation

  • Weible, C.M. and Ingold, K. (2018) Why advocacy coalitions matter and practical insights about them, Policy & Politics, 46(2): 325–43. doi: 10.1332/030557318X15230061739399

    • Search Google Scholar
    • Export Citation

  • Weiss, C.H. (1979) The many meanings of research utilization, Public Administration Review, 39(5): 426–31. doi: 10.2307/3109916

  • World Health Organization (2021) Evidence, Policy, Impact: WHO Guide for Evidence-Informed Decision-Making, Geneva: World Health Organization, https://apps.who.int/iris/bitstream/handle/10665/350994/9789240039872-eng.pdf?sequence=.

    • Search Google Scholar
    • Export Citation

  • Wu, Y. and Shen, F. (2020) Negativity makes us polarized: a longitudinal study of media tone and opinion polarization in Hong Kong, Asian Journal of Communication, 30(3–4): 199–220. doi: 10.1080/01292986.2020.1784968

    • Search Google Scholar
    • Export Citation

  • Yanovitzky, I. and Weber, M. (2020) Analysing use of evidence in public policymaking processes: a theory-grounded content analysis methodology, Evidence & Policy, 16(1): 65–82. doi: 10.1332/174426418x15378680726175

    • Search Google Scholar
    • Export Citation

  • Zhang, Y., Loh, C., Louie, P.K.K., Liu, H. and Lau, A.K.H. (2018) The roles of scientific research and stakeholder engagement for evidence-based policy formulation on shipping emissions control in Hong Kong, Journal of Environmental Management, 223: 49–56. doi: 10.1016/j.jenvman.2018.06.008

    • Search Google Scholar
    • Export Citation
Christian Joy Pattawi Cruz University of the Philippines, Philippines

Search for other papers by Christian Joy Pattawi Cruz in
Current site
Google Scholar Close
,
Kira Matus Hong Kong University of Science and Technology, Hong Kong SAR

Search for other papers by Kira Matus in
Current site
Google Scholar Close
, and
Stuart Gietel-Basten Hong Kong University of Science and Technology, Hong Kong SAR

Search for other papers by Stuart Gietel-Basten in
Current site
Google Scholar Close

cpcruz5@up.edu.ph
kmatus@ust.hk
sgb@ust.hk

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 1031 1031 106
Full Text Views 315 315 19
PDF Downloads 280 280 26

Altmetrics

Dimensions

Cover Evidence & Policy
Evidence & Policy
A journal of research, debate and practice