Research assessment in a National Health Service organisation: a process for learning and accountability
This paper describes the only known research assessment exercise to be undertaken by a health board. There is a growing imperative to evidence the impact of national health services, but no formal mechanism exists to quality-rate research or assess its effect on practice. This study describes how one healthcare organisation adapted a research impact framework along with impact dimensions and impact indicators to better identify research outcomes and account for research funding.
We know that research findings rarely impact on practice in a linear manner, that their effects are difficult to measure, and that they need adaptation within practice and professional contexts before use. We found that while links to practice and professional settings within our health board were strong, researchers tended not to plan for specific outcomes or impacts at project conception. Hence, although influence on practice and policy did occur, this seemed to be the result of extempore processes. This is despite an increasing emphasis in the health service on accountability in an era of severe financial constraint. Most of our successful research outcomes arose when researchers engaged in active dissemination with different audiences including end users and opinion/senior leaders.
The findings of this research assessment are helping us to better consider dissemination and knowledge translation strategies at the outset of projects. Being able to evidence specific research outcomes to senior managers and board members is enabling us to continue our modest investment in health services research.
-
Aymerish, M., Carrion, C., Gallo, P., Garcia, M., Lopez-Bermejo, A., Quesada, M., Ramos, R. (2012) ‘Measuring the payback of research activities: a feasible ex-post evaluation methodology in epidemiology and public health’, Social Science and Medicine, 75(3), 505–510. doi: 10.1016/j.socscimed.2012.03.044
-
Banzi, R., Moja, L., Pistotti, V., Facchini, A., Liberati, A. (2011) ‘Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews’, Health Research Policy and Systems, 9, 26. doi: 10.1186/1478-4505-9-26
-
Bennett, W., Bird, J., Burrows, S., Counter, P., Reddy, V. (2012) ‘Does academic output correlate with better mortality rates in NHS trusts in England?’, Public Health, 126, S40–S43. doi: 10.1016/j.puhe.2011.09.010
-
Buykx, P., Humphreys, J., Wakerman, J., Perkins, D., Lyle, D., McGrail, M., Kinsman, L. (2012) ‘Making evidence count: a framework to monitor the impact of health services research’, Australia Journal of Rural Health, 20(2), 51–58. doi: 10.1111/j.1440-1584.2012.01256.x
-
Crown (2016) Building on Success and Learning from Experience: an Independent Review of the Research Excellence Framework, www.gov.uk/government/publications/research-excellence-framework-review
-
Diefenbach, T. (2009) ‘New public management in public sector organisations: the dark side of managerialist enlightenment’, Public Administration, 87(4), 892–909. doi: 10.1111/j.1467-9299.2009.01766.x
-
Exworthy, M. (2010) ‘The performance paradigm in the English NHS: potential, pitfalls, and prospects’, Eurohealth, 16(3), 16–19.
-
Ferlie, E. (2017) ‘Exploring 30 years of UK public services management reform: the case of health care’, International Journal of Public Sector Management, 30(6–7), 615–625. doi: 10.1108/IJPSM-06-2017-0178
-
Greenhalgh, T., Raftery, J., Hanney, S., Glover, M. (2016) ‘Research impact: a narrative review’, BMC Medicine, 14, 78. doi: 10.1186/s12916-016-0620-8
-
Hammersley, M. (2008) ‘Troubling criteria: a critical commentary on Furlong and Oancea’s framework for assessing educational research’, British Educational Research Journal, 34(6), 747–762. doi: 10.1080/01411920802031468
-
Hanney, S., Davies, A., Buxton, M. (1999) ‘Assessing benefits from health research projects: can we use questionnaires instead of case studies?’, Research Evaluation, 8(3), 189–199. doi: 10.3152/147154499781777469
-
Jonker, L., Fisher, S. (2018) ‘The correlation between National Health Service trusts’ clinical trial activity and both mortality rates and care quality commission ratings: a retrospective cross-sectional study’, Public Health, 157, 1–6. doi: 10.1016/j.puhe.2017.12.022
-
Kalucy, E., Jackson-Bowers, E., McIntyre, E., Reed, R. (2009) ‘The feasibility of determining the impact or primary health care research projects using the Payback Framework’, Health Research Policy Systems, 7, 11. doi: 10.1186/1478-4505-7-11
-
King’s Fund/Health Foundation (2017) Making the Case for Quality Improvement: Lessons for NHS Boards and Leaders, www.kingsfund.org.uk/publications/making-case-quality-improvement
-
Kuruvilla, S., Mays, N., Pleasant, A., Walt, G. (2006) ‘Describing the impact of health research: a research impact framework’, BMC Health Services Research, 6, 134. doi: 10.1186/1472-6963-6-134
-
Milat, A., Laws, R., King, L., Newson, R., Rychetnik, L., Rissel, C., Bauman, A., Redman, S., Bennie, J. (2013) ‘Policy and practice impacts of applied research: a case study analysis of the New South Wales Promotion Demonstration Research Grants Scheme 2000–1006’, Health Research Policy Systems, 11, 5. doi: 10.1186/1478-4505-11-5
-
Milat, A., Bayman, A., Redman, S. (2015) ‘A narrative review of research impact assessment models and methods’, Health Research Policy and Systems, 13, 18. doi: 10.1186/s12961-015-0003-1
-
Simonet, D. (2013) ‘The new public management theory in the British health care system: a critical review’, Administration & Society, 47(7), 802–826. doi: 10.1177/0095399713485001
-
Stocks, S.J., Alam, R., Bowie, P., Campbell, S., de Wet, C., Esmail, A. and Cheraghi-Sohi, S. (2017) ‘Never events in UK General Practice: A survey of the views of General Practitioners on their frequency and acceptability as a safety improvement approach’, Journal of Patient Safety, doi: 10.1097/PTS.0000000000000380
-
Waddell, C., Cody, A., Shepherd, J., Lavis, J., Abelson, J., Bird-Grayson, T. (2007) ‘Balancing rigour and relevance: researchers’ contributions to children’s mental health policy in Canada’, Evidence & Policy, 3(2), 181–195.
-
Walter, I., Nutley, S., Davies, H. (2003) Research Impact: a Cross Sector Literature Review (ESRC Network for Evidence Based Policy and Practice).
| May 2022 onwards | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 8 | 8 | 8 |
| Full Text Views | 12 | 12 | 1 |
| PDF Downloads | 11 | 11 | 1 |
We are a leading social science publisher
committed to making a difference.
Bristol University Press and
Policy Press
bristoluniversitypress.co.uk