Evidence & Policy
A journal of research, debate and practice

Policy actors’ perceptions of qualitative research in policymaking: the case of higher education rulemaking in the United States

Author:
Rebecca S. Natow Hofstra University, USA

Search for other papers by Rebecca S. Natow in
Current site
Google Scholar
Close
Restricted access
Get eTOC alerts
Rights and permissions Cite this article

Background:

As calls for evidence-based policymaking become increasingly common, qualitative research has much to offer the policy community. However, policymakers frequently evidence a preference for quantitative research. By discounting the importance of qualitative research in the policymaking process, resulting policies and their target populations miss out on the benefits that qualitative research uniquely offers.

Aims and objectives:

The purpose of this study was to examine how qualitative research has been perceived and used in the US government’s rulemaking process for creating higher education regulations.

Methods:

This qualitative case study included data from semi-structured interviews with 34 policy actors involved in higher education rulemaking, rulemaking documents, and research reports cited in several key higher education regulations.

Findings:

Many policy actors viewed qualitative research favourably, but qualitative studies have seldom been cited in higher education rulemaking. Several respondents discussed validity concerns and some policymakers’ misunderstandings regarding qualitative methods. Moreover, storytelling can influence policy actors’ perspectives about the content of policies, and qualitative research was viewed as effective at identifying compelling stories. Thus, narratives derived from qualitative research may provide an opportunity for qualitative researchers to have their work considered in policymaking processes.

Discussion and conclusion:

Qualitative research faces challenges with gaining visibility and influence in the development of regulatory policy. However, this study has shown that qualitative research has the potential to be both useful and persuasive to policymakers. Studies that discuss relevant stories may be particularly compelling.

  • Altheide, D.L. and Johnson, J.M. (2011) Reflections on interpretive adequacy in qualitative research, SAGE Handbook of Qualitative Research, vol 4, Los Angeles, CA: SAGE, pp 58194.

    • Search Google Scholar
    • Export Citation
  • Asen, R., Gurke, D., Solomon, R., Conners, P. and Gumm, E. (2011) ‘The research says’: definitions and uses of a key policy term in federal law and local school board deliberations, Argumentation and Advocacy, 47(4): 195213.

    • Search Google Scholar
    • Export Citation
  • Association of Private Sector Colleges and Universities v. Duncan (2012) Memorandum Opinion, Civil Action 11–1314 (RC), D.D.C.

  • Belfield, C.R., Bowden, A.B. and Rodriguez, V. (2018) Evaluating regulatory impact assessments in education policy, American Journal of Evaluation, 40(3): 33553.

    • Search Google Scholar
    • Export Citation
  • Bowers, J. and Testa, P.F. (2019) Better government, better science: the promise of and challenges facing the evidence-informed policy movement, Annual Review of Political Science, 22(1): 52142.

    • Search Google Scholar
    • Export Citation
  • Bozeman, H., Mingo, M. and Hershey-Arista, M. (2017) Summary report for the 2017 gainful employment focus groups, report for the US department of education, Rockville, MD: Westat, https://www2.ed.gov/about/offices/list/ope/summaryrpt2017gefocus317.pdf.

    • Search Google Scholar
    • Export Citation
  • Carter, C.L. (2009) Consumer Protection in the States: A 50-state Report On Unfair and Deceptive Acts and Practices Statutes, Report, Boston, MA: National Consumer Law Center, https://www.nclc.org/images/pdf/udap/report_50_states.pdf.

    • Search Google Scholar
    • Export Citation
  • Clemens, R.F. and Tierney, W.G. (2017) The utility of qualitative research to inform public policy, in P.A. Pasque and V.M. Lechuga (eds) Qualitative Inquiry in Higher Education Organization and Policy Research, New York: Routledge, pp 2747.

    • Search Google Scholar
    • Export Citation
  • Corbin, J. and Strauss, A. (2008) Basics of Qualitative Research, 3rd edn, Los Angeles, CA: SAGE.

  • Creswell, J.W. and Poth, C.N. (2018), Qualitative Inquiry and Research Design: Choosing Among Five Approaches, 4th edn, Thousand Oaks, CA: SAGE.

    • Search Google Scholar
    • Export Citation
  • Delgado, R. (1989) Storytelling for oppositionists and others: a plea for narrative, Michigan Law Review, 87(8): 241141.

  • Denzin, N.K. (2009) The elephant in the living room: or extending the conversation about the politics of evidence, Qualitative Research, 9(2): 13960.

    • Search Google Scholar
    • Export Citation
  • Denzin, N.K., Lincoln, Y.S. and Giardina, M.D. (2006) Disciplining qualitative research, International Journal of Qualitative Studies in Education, 19(6): 76982.

    • Search Google Scholar
    • Export Citation
  • Donmoyer, R. (2012a) Can qualitative researchers answer policymakers’ What-Works Question?, Qualitative Inquiry, 18(8): 66273.

  • Donmoyer, R. (2012b) Two (very) different worlds: the cultures of policymaking and qualitative research, Qualitative Inquiry, 18(9): 798807.

    • Search Google Scholar
    • Export Citation
  • Dougherty, K.J. and Natow, R.S. (2020) Performance-based funding for higher education: how well does neoliberal theory capture neoliberal practice?, Higher Education, 80(3): 45778.

    • Search Google Scholar
    • Export Citation
  • Evans, D. (2003) Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions, Journal of Clinical Nursing, 12(1): 7784.

    • Search Google Scholar
    • Export Citation
  • Executive Order No. 12866, 1993 comp., Title 3 C.F.R. 638.

  • Fielding, N.G. (2020) Critical qualitative research and impact in the public sphere, Qualitative Inquiry, 26(2): 14252.

  • Gibbs, G.R. (2018) Analyzing Qualitative Data, 2nd edn, Thousand Oaks, CA: SAGE.

  • Gibton, D. (2016) Researching Education Policy, Public Policy, and Policymakers: Qualitative Methods and Ethical Issues, New York: Routledge.

    • Search Google Scholar
    • Export Citation
  • Gray, G. and Jones, M.D. (2016) A qualitative narrative policy framework? Examining the policy narratives of US campaign finance regulatory reform, Public Policy Administration, 31(3): 193220.

    • Search Google Scholar
    • Export Citation
  • Gross, J.P., Cekic, O., Hossler, D. and Hillman, N. (2009) What matters in student loan default: a review of the research literature, Journal of Student Financial Aid, 39(1): 1929.

    • Search Google Scholar
    • Export Citation
  • Haynes, A., Rowbotham, S.J., Redman, S., Brennan, S., Williamson, A. and Moore, G. (2018) What can we learn from interventions that aim to increase policy-makers’ capacity to use research? A realist scoping review, Health Research Policy & Systems, 16(1): 31.

    • Search Google Scholar
    • Export Citation
  • Henig, J. (2012) The politics of data use, Teachers College Record, 114(11): 132.

  • Herriott, R.E. and Firestone, W.A. (1983) Multisite qualitative policy research: optimizing description and generalizability, Educational Researcher, 12(2): 1419.

    • Search Google Scholar
    • Export Citation
  • Kaplan, R.G., Riedy, R., Van Horne, K. and Penuel, W. (2019) Going on a statewide listening tour: involving education leaders in the process of research to enhance the practical value of qualitative research, Evidence & Policy, 15(2): 17996.

    • Search Google Scholar
    • Export Citation
  • Kerrigan, M.R. and Johnson, A.T. (2019) Qualitative approaches to policy research in education: contesting the evidence-based, neoliberal regime, American Behavioral Scientist, 63(3): 28795.

    • Search Google Scholar
    • Export Citation
  • Kerwin, C.M. and Furlong, S.R. (2011) Rulemaking: How Government Agencies Write Law and Make Policy, 4th edn, Washington, DC: CQ Press.

  • Kugler, A. (2014) Labor market analysis and labor policymaking in the nation’s capital, ILR Review, 67(Suppl 3): 594607.

  • Lather, P. (2004) This is your father’s paradigm: government intrusion and the case of qualitative research in education, Qualitative Inquiry, 10(1): 1534.

    • Search Google Scholar
    • Export Citation
  • Lather, P. (2006) Foucauldian scientificity: rethinking the nexus of qualitative research and educational policy analysis, International Journal of Qualitative Studies in Education, 19(6): 78391.

    • Search Google Scholar
    • Export Citation
  • Lee, H., Lindquist, J.D., and Acito, F. (1997) Managers’ evaluation of research design and its impact on the use of research: An experimental approach, Journal of Business Research, 39(3): 23140.

    • Search Google Scholar
    • Export Citation
  • Locock, L. and Boaz, A. (2019) Drawing straight lines along blurred boundaries: qualitative research, patient and public involvement in medical research, co-production and co-design, Evidence & Policy, 15(3): 40921.

    • Search Google Scholar
    • Export Citation
  • Lovell, M., Guthrie, J., Simpson, P. and Butler, T. (2018) Navigating the political landscape of Australian criminal justice reform: senior policy-makers on alternatives to incarceration, Current Issues in Criminal Justice, 29(3): 22741.

    • Search Google Scholar
    • Export Citation
  • Lune, H. and Berg, B.L. (2017) Qualitative Research Methods for the Social Sciences, 9th edn, New York: Pearson.

  • Malterud, K. (2001) Qualitative research: standards, challenges, and guidelines, The Lancet, 358(9280): 48388.

  • Maxwell, J.A. (2004) Causal explanation, qualitative research, and scientific inquiry in education, Educational Researcher, 33(2): 311.

    • Search Google Scholar
    • Export Citation
  • Maxwell, J.A. (2013) Qualitative Research Design: An Interactive Approach, 3rd edn, Thousand Oaks, CA: SAGE.

  • McAleese, S. and Kilty, J.M. (2019) Stories matter: reaffirming the value of qualitative research, The Qualitative Report, 24(4): 82245.

    • Search Google Scholar
    • Export Citation
  • Merriam, S.B. and Tisdell, E.J. (2016) Qualitative Research: A Guide to Design and Implementation, San Francisco, CA: Jossey-Bass.

  • Miles, M.B., Huberman, A.M. and Saldaña, J. (2019) Qualitative Data Analysis: A Methods Sourcebook, 4th edn, Thousand Oaks, CA: SAGE.

  • National Research Council (2012) Using Science as Evidence in Public Policy, Washington, DC: National Academies Press.

  • Natow, R.S. (2017) Higher Education Rulemaking: The Politics of Creating Regulatory Policy, Baltimore, MD: Johns Hopkins University Press.

    • Search Google Scholar
    • Export Citation
  • Natow, R.S. (2020) Research utilization in higher education rulemaking: a multi-case study of research prevalence, sources, and barriers, Education Policy Analysis Archives, 28(5): 132.

    • Search Google Scholar
    • Export Citation
  • Nutley, S.M., Walter, I. and Davies, H.T.O. (2007) Using Evidence: How Research Can Inform Public Services, Bristol: Policy Press.

  • Office of the Federal Register (2011) A Guide to the Rulemaking Process, www.federalregister.gov/uploads/2011/01/the_rulemaking_process.pdf.

    • Search Google Scholar
    • Export Citation
  • Oliver, K., Innvar, S., Lorenc, T., Woodman, J. and Thomas, J. (2014) A systematic review of barriers to and facilitators of the use of evidence by policymakers, BMC Health Services Research, 14(1): 2.

    • Search Google Scholar
    • Export Citation
  • Ospina, S.M., Esteve, M. and Lee, S. (2018) Assessing qualitative studies in public administration research, Public Administration Review, 78(4): 593605.

    • Search Google Scholar
    • Export Citation
  • Pelesh, M.L. (1994) Regulations under the higher education amendments of 1992: a case study in negotiated rulemaking, Law and Contemporary Problems, 57(4): 15170.

    • Search Google Scholar
    • Export Citation
  • Posti-Ahokas, H. (2013) Empathy-based stories capturing the voice of female secondary school students in Tanzania, International Journal of Qualitative Studies in Education, 26(10): 127792.

    • Search Google Scholar
    • Export Citation
  • Postsecondary National Policy Institute (n.d.) Overview, https://pnpi.org/wp-content/uploads/2019/08/UpdatedPNPI_OnePager_Doc.pdf.

  • Program Integrity: Gainful Employment, Final Regulations (2014) 79 Fed. Reg. 64890.

  • Program Integrity: Gainful Employment, Final Regulations (2019) 84 Fed. Reg. 31392.

  • Purcell-Gates, V. (2000) The Role of Qualitative and Ethnographic Research in Educational Policy, Opinion paper, Newark, NJ: International Reading Association.

    • Search Google Scholar
    • Export Citation
  • Ritchie, J. and Spencer, L. (1994) Qualitative data analysis for applied policy research, in A. Bryman and R.G. Burgess (eds) Analyzing Qualitative Data, New York: Routledge, pp 17394.

    • Search Google Scholar
    • Export Citation
  • Rivera, L.A. and Tilcsik, A. (2019) Scaling down inequality: rating scales, gender bias, and the architecture of evaluation, American Sociological Review, 84(2): 24874.

    • Search Google Scholar
    • Export Citation
  • Saiani, P.P. (2018) Doing sociology in the age of ‘evidence-based research’: scientific epistemology versus political dominance, The American Sociologist, 49(1): 8097.

    • Search Google Scholar
    • Export Citation
  • Saldaña, J. (2016) The Coding Manual for Qualitative Researchers, 3rd edn, Los Angeles, CA: SAGE.

  • Sallee, M.W. and Flood, J.T. (2012) Using qualitative research to bridge research, policy, and practice, Theory Into Practice, 51(2): 13744.

    • Search Google Scholar
    • Export Citation
  • Savin-Baden, M. and Major, C.H. (2013) Qualitative Research: The Essential Guide to Theory and Practice, New York: Routledge.

  • Scott, G.A. (2011) Postsecondary Education: Student Outcomes Vary at For-Profit, Nonprofit, and Public Schools. Report to Congressional Requesters, Report no. GAO-12–143, Washington, DC: US Government Accountability Office.

    • Search Google Scholar
    • Export Citation
  • Sinkovics, R.R. and Alfoldi, E.A. (2012) Progressive focusing and trustworthiness in qualitative research, Management International Review, 52(6): 81745.

    • Search Google Scholar
    • Export Citation
  • Srivastava, P. and Hopwood, N. (2009) A practical iterative framework for qualitative data analysis, International Journal of Qualitative Research Methods, 8(1): 7684.

    • Search Google Scholar
    • Export Citation
  • Student Assistance General Provisions, Federal Family Education Loan Program, and William D. Ford Federal Direct Loan Program: Final Regulations (2015) 80 Fed. Reg. 67204.

    • Search Google Scholar
    • Export Citation
  • Student Assistance General Provisions, Federal Perkins Loan Program, Federal Family Education Loan Program, William D. Ford Federal Direct Loan Program, and Teacher Education Assistance for College and Higher Education Grant Program: Final Regulations (2016) 81 Fed. Reg. 75926.

  • Student Assistance General Provisions, Federal Family Education Loan Program, and William D. Ford Federal Direct Loan Program: Final Regulations (2019) 84 Fed. Reg. 49788.

  • Sunstein, C.R. (2013) The office of information and regulatory affairs: myths and realities, Harvard Law Review, 126(7): 183878.

  • Suri, H. (2011) Purposeful sampling in qualitative research synthesis, Qualitative Research Journal, 11(2): 6375.

  • Teacher Preparation Issues: Final Regulations (2016) 81 Fed. Reg. 75494.

  • Torgerson, D.J. and Torgerson, C.J. (2003) Avoiding bias in randomised controlled trials in educational research, British Journal of Educational Studies, 51(1): 3645.

    • Search Google Scholar
    • Export Citation
  • Torrance, H. (2008) Building confidence in qualitative research: engaging the demands of policy, Qualitative Inquiry, 14(4) 50727.

  • Truong, D.H. (2000) Daubert and judicial review: how does an administrative agency distinguish valid science from junk science, Akron Law Review, 33(3): 36590.

    • Search Google Scholar
    • Export Citation
  • US Department of Education (2002) No Child Left Behind: A Desktop Reference, Washington, DC: US Department of Education.

  • US Department of Education (2003) Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide, Washington, DC: US Department of Education.

    • Search Google Scholar
    • Export Citation
  • Weiss, C.H. (1979) The many meanings of research utilization, Public Administration Review, 39(5): 42631.

  • Weiss, C.H., Murphy-Graham, E. and Birkeland, S. (2005) An alternate route to policy influence: How evaluations affect D.A.R.E., American Journal of Evaluation, 26(1): 1230.

    • Search Google Scholar
    • Export Citation
  • Werner, A. (2004) A Guide to Implementation Research, Washington, DC: Urban Institute Press.

  • William D. Ford Federal Direct Loan Program: Final Regulations (2014) 79 Fed. Reg. 63317.

Rebecca S. Natow Hofstra University, USA

Search for other papers by Rebecca S. Natow in
Current site
Google Scholar
Close

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 989 541 67
Full Text Views 168 36 4
PDF Downloads 162 41 5

Altmetrics

Dimensions

You are not currently authorised to access the full text of this chapter or article.
Access options
To access the full chapter or article then please choose one of the options below.
Purchase
Pay to access content (PDF download and unlimited online access)
Other access options
Redeem Token
Institutional Login
Log in via Open Athens or Shibboleth. Please contact your librarian if you need any help.
Login with Institutional Access
Personal Login
Login to your BUP account with your individual credentials.
Login with BUP account

Institutional librarians can find more information about free trials here