Evidence & Policy
A journal of research, debate and practice

Research assessment in a National Health Service organisation: a process for learning and accountability

View author details View Less
  • 1 NHS Education for Scotland, UK
Restricted access
Get eTOC alerts
Rights and permissions Cite this article

This paper describes the only known research assessment exercise to be undertaken by a health board. There is a growing imperative to evidence the impact of national health services, but no formal mechanism exists to quality-rate research or assess its effect on practice. This study describes how one healthcare organisation adapted a research impact framework along with impact dimensions and impact indicators to better identify research outcomes and account for research funding.

We know that research findings rarely impact on practice in a linear manner, that their effects are difficult to measure, and that they need adaptation within practice and professional contexts before use. We found that while links to practice and professional settings within our health board were strong, researchers tended not to plan for specific outcomes or impacts at project conception. Hence, although influence on practice and policy did occur, this seemed to be the result of extempore processes. This is despite an increasing emphasis in the health service on accountability in an era of severe financial constraint. Most of our successful research outcomes arose when researchers engaged in active dissemination with different audiences including end users and opinion/senior leaders.

The findings of this research assessment are helping us to better consider dissemination and knowledge translation strategies at the outset of projects. Being able to evidence specific research outcomes to senior managers and board members is enabling us to continue our modest investment in health services research.

  • Aymerish, M., Carrion, C., Gallo, P., Garcia, M., Lopez-Bermejo, A., Quesada, M., Ramos, R. (2012) ‘Measuring the payback of research activities: a feasible ex-post evaluation methodology in epidemiology and public health’, Social Science and Medicine, 75(3), 505510. doi: 10.1016/j.socscimed.2012.03.044

    • Search Google Scholar
    • Export Citation
  • Banzi, R., Moja, L., Pistotti, V., Facchini, A., Liberati, A. (2011) ‘Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews’, Health Research Policy and Systems, 9, 26. doi: 10.1186/1478-4505-9-26

    • Search Google Scholar
    • Export Citation
  • Bennett, W., Bird, J., Burrows, S., Counter, P., Reddy, V. (2012) ‘Does academic output correlate with better mortality rates in NHS trusts in England?’, Public Health, 126, S40S43. doi: 10.1016/j.puhe.2011.09.010

    • Search Google Scholar
    • Export Citation
  • Buykx, P., Humphreys, J., Wakerman, J., Perkins, D., Lyle, D., McGrail, M., Kinsman, L. (2012) ‘Making evidence count: a framework to monitor the impact of health services research’, Australia Journal of Rural Health, 20(2), 5158. doi: 10.1111/j.1440-1584.2012.01256.x

    • Search Google Scholar
    • Export Citation
  • Crown (2016) Building on Success and Learning from Experience: an Independent Review of the Research Excellence Framework, www.gov.uk/government/publications/research-excellence-framework-review

    • Search Google Scholar
    • Export Citation
  • Diefenbach, T. (2009) ‘New public management in public sector organisations: the dark side of managerialist enlightenment’, Public Administration, 87(4), 892909. doi: 10.1111/j.1467-9299.2009.01766.x

    • Search Google Scholar
    • Export Citation
  • Exworthy, M. (2010) ‘The performance paradigm in the English NHS: potential, pitfalls, and prospects’, Eurohealth, 16(3), 1619.

  • Ferlie, E. (2017) ‘Exploring 30 years of UK public services management reform: the case of health care’, International Journal of Public Sector Management, 30(6–7), 615625. doi: 10.1108/IJPSM-06-2017-0178

    • Search Google Scholar
    • Export Citation
  • Greenhalgh, T., Raftery, J., Hanney, S., Glover, M. (2016) ‘Research impact: a narrative review’, BMC Medicine, 14, 78. doi: 10.1186/s12916-016-0620-8

    • Search Google Scholar
    • Export Citation
  • Hammersley, M. (2008) ‘Troubling criteria: a critical commentary on Furlong and Oancea’s framework for assessing educational research’, British Educational Research Journal, 34(6), 747762. doi: 10.1080/01411920802031468

    • Search Google Scholar
    • Export Citation
  • Hanney, S., Davies, A., Buxton, M. (1999) ‘Assessing benefits from health research projects: can we use questionnaires instead of case studies?’, Research Evaluation, 8(3), 189199. doi: 10.3152/147154499781777469

    • Search Google Scholar
    • Export Citation
  • Jonker, L., Fisher, S. (2018) ‘The correlation between National Health Service trusts’ clinical trial activity and both mortality rates and care quality commission ratings: a retrospective cross-sectional study’, Public Health, 157, 16. doi: 10.1016/j.puhe.2017.12.022

    • Search Google Scholar
    • Export Citation
  • Kalucy, E., Jackson-Bowers, E., McIntyre, E., Reed, R. (2009) ‘The feasibility of determining the impact or primary health care research projects using the Payback Framework’, Health Research Policy Systems, 7, 11. doi: 10.1186/1478-4505-7-11

    • Search Google Scholar
    • Export Citation
  • King’s Fund/Health Foundation (2017) Making the Case for Quality Improvement: Lessons for NHS Boards and Leaders, www.kingsfund.org.uk/publications/making-case-quality-improvement

    • Search Google Scholar
    • Export Citation
  • Kuruvilla, S., Mays, N., Pleasant, A., Walt, G. (2006) ‘Describing the impact of health research: a research impact framework’, BMC Health Services Research, 6, 134. doi: 10.1186/1472-6963-6-134

    • Search Google Scholar
    • Export Citation
  • Milat, A., Laws, R., King, L., Newson, R., Rychetnik, L., Rissel, C., Bauman, A., Redman, S., Bennie, J. (2013) ‘Policy and practice impacts of applied research: a case study analysis of the New South Wales Promotion Demonstration Research Grants Scheme 2000–1006’, Health Research Policy Systems, 11, 5. doi: 10.1186/1478-4505-11-5

    • Search Google Scholar
    • Export Citation
  • Milat, A., Bayman, A., Redman, S. (2015) ‘A narrative review of research impact assessment models and methods’, Health Research Policy and Systems, 13, 18. doi: 10.1186/s12961-015-0003-1

    • Search Google Scholar
    • Export Citation
  • Simonet, D. (2013) ‘The new public management theory in the British health care system: a critical review’, Administration & Society, 47(7), 802826. doi: 10.1177/0095399713485001

    • Search Google Scholar
    • Export Citation
  • Stocks, S.J., Alam, R., Bowie, P., Campbell, S., de Wet, C., Esmail, A. and Cheraghi-Sohi, S. (2017) ‘Never events in UK General Practice: A survey of the views of General Practitioners on their frequency and acceptability as a safety improvement approach’, Journal of Patient Safety, doi: 10.1097/PTS.0000000000000380

    • Search Google Scholar
    • Export Citation
  • Waddell, C., Cody, A., Shepherd, J., Lavis, J., Abelson, J., Bird-Grayson, T. (2007) ‘Balancing rigour and relevance: researchers’ contributions to children’s mental health policy in Canada’, Evidence & Policy, 3(2), 181195.

    • Search Google Scholar
    • Export Citation
  • Walter, I., Nutley, S., Davies, H. (2003) Research Impact: a Cross Sector Literature Review (ESRC Network for Evidence Based Policy and Practice).

    • Search Google Scholar
    • Export Citation

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 64 64 55
Full Text Views 14 14 11
PDF Downloads 17 17 12

Altmetrics

Dimensions