The approach to outcome and impact evaluation in this book works for the complex problems and issues that public services face. It has been used in settings such as health, social work and care, policy, international development, research, and children and adult services. There is more in common across these contrasting sectors and contexts than divides them in relation to outcome and impact evaluation. Whatever the setting, the key challenges remain: how to work well with complex contexts; how to understand and work well with outcomes and impacts; how to get the relevant data, evidence and feedback; and how to embed this in day-to-day work. This chapter explores those challenges and gives practical tips for overcoming them. It includes case studies of the approach in different organisations and settings.
Ahmed, I.K. and Dantata, B.S. (2016) ‘Problems and challenges of policy implementation for national development’, Research on Humanity and Social Sciences, 6(15): 60–65.
Archibald, T., Sharrock, G., Buckley, J. and Cook, N. (2016) ‘Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development’, Evaluation and Program Planning, 59: 119–127.
Australian Government (2019) ‘Delivering for Australians. A world-class Australian public service: The government’s APS reform agenda’, Department of the Prime Minister and Cabinet [online] 13 December. Available from: https://www.pmc.gov.au/resource-centre/government/delivering-for-australians [Accessed 4 October 2021].
Bicket, M., Christie, I., Gilbert, N., Hills, D., Penn, A. and Wilkinson, H. (2020) Magenta Book 2020: Supplementary Guide: Handling Complexity in Policy Evaluation, London: HM Treasury.
Black, L. (2021) ‘Funding Utopia’ [online]. Available from: https://leah-black.medium.com/funding-utopia-2-ten-fifteen-years-unrestricted-outcomes-community-wealth-and-more-b32760b32dd3 [Accessed 15 March 2022].
Blamey, A. and Mackenzie, M. (2007) ‘Theories of change and realistic evaluation: Peas in a pod or apples and oranges?’, Evaluation, 13(4): 439–455.
Bozarth, J.D. (2013) ‘Unconditional positive regard’, in M. Cooper, M. O’Hara, P.F. Schmid and A.C. Bohart (eds) The Handbook of Person-Centred Psychotherapy and Counselling, New York: Palgrave Macmillan/Springer Nature, pp 180–192.
Cameron, W.B. (1969) Informal Sociology: A casual introduction to sociological thinking, New York: Random House.
CECAN – Centre for Evaluating Complexity Across the Nexus (2020) ‘Supplementary Guide: Handling complexity in policy evaluation’, HM Treasury [online]. Available from: https://www.cecan.ac.uk/news/handling-complexity-in-policy-evaluation-magenta-book-2020-supplementary-guide/ [Accessed 4 October 2021].
Christie, C. (2011) ‘Christie Commission on the future delivery of public services’, Scottish Government [online] 29 June. Available from: https://www.gov.scot/publications/commission-future-delivery-public-services/ [Accessed 4 October 2021].
Cook, A. (2015) ‘Partnership working across UK public services: Evidence review’, Edinburgh: What Works Scotland [online]. Available from: http://whatworksscotland.ac.uk/wp-content/uploads/2015/12/WWS-Evidence-Review-Partnership-03-Dec-2015-.pdf [Accessed 7 October 2021].
Cook, A. (2017) ‘Outcome based approaches in public service reform’, Edinburgh: What Works Scotland [online]. Available from: http://whatworksscotland.ac.uk/wp-content/uploads/2017/04/OutcomeBasedApproachesinPublicServiceReform.pdf [Accessed 25 February 2022].
Cook, A. and Miller, E. (2012) Talking Points Personal Outcomes Approach: A practical guide for organisations, Edinburgh: Joint Improvement Team, Scottish Government.
Crawford, C., Edwards, A., Farquharson, C., Greaves, E., Trevelyan, G., Wallace, E. and White, C. (2016) Magic Breakfast: Evaluation report and executive summary, London: The Education Endowment Foundation.
Crewe, E. and Young, J. (2002) Bridging Research and Policy: Context, evidence and links, London: Overseas Development Institute.
Darnton, A. and Horne, J. (2013) ‘Influencing behaviours – moving beyond the individual: ISM user guide’, Scottish Government [online]. Available from: https://www.gov.scot/publications/influencing-behaviours-moving-beyond-individual-user-guide-ism-tool/ [Accessed 1 March 2022].
Davies, H.T. and Nutley, S.M. (2000) ‘Healthcare: Evidence to the fore’, in H.T. Davies, S.M. Nutley and P.C. Smith (eds) What Works: Evidence-based policy and practice in public services, Bristol: Policy Press, pp 43–67.
Gabbay, J. and le May, A. (2004) ‘Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care’, BMJ, 329: 1013 https://doi.org/10.1136/bmj.329.7473.1013
Garvin, D.A. (1993) ‘Building a learning organization’, Harvard Business Review, 71(4): 78–91 [online]. Available from: https://hbr.org/1993/07/building-a-learning-organization [Accessed 5 October 2021].
Glendinning, C., Clarke, S., Hare, P., Kitchetkova, I., Maddison, J. and Newbronner, L. (2006) Outcomes-focused Services for Older People, Bristol: Policy Press.
Government of Canada. ‘Theory-based approaches to evaluation: Concept and practices’ [online]. Available from: https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/theory-based-approaches-evaluation-concepts-practices.html [Accessed 5 October 2021].
Huff, D. and Geis, I. (1954) How to Lie with Statistics, New York: Norton.
Matter of Focus ‘Understand the unique context of your work’ [online]. Available from: https://www.matter-of-focus.com/understand-the-unique-context-of-your-work/ [Accessed 5 October 2021].
Matter of Focus ‘3 feedback tools to help you track your outcomes and impact’ [online]. Available from: https://www.matter-of-focus.com/3-feedback-tools/ [Accessed 7 October 2021].
Mayne, J. (2008) ‘Contribution analysis: An approach to exploring cause and effect’, ILAC Brief 16, pp 1–4. Available from: https://hdl.handle.net/10568/70124 [Accessed 7 October 2021].
Miller, E. and Barrie, K. (2016) ‘Learning from the Meaningful and Measurable Project: Strengthening links between identity, action and decision-making’, Healthcare Improvement Scotland [online]. Available from: https://ihub.scot/media/2126/po-learning.pdf [Accessed 25 February 2022].
Montague, S. (2012) ‘Theory-based approaches for practical evaluation’ [online]. Available from: https://www.pmn.net/wp-content/uploads/Theory-based-Approaches-for-Practical-Evaluation.pdf [Accessed 5 October 2021].
Morton, S. (2012) Exploring and Assessing Social Research Impact: A case study of a research partnership’s impacts on policy and practice, Edinburgh: University of Edinburgh.
Morton, S. (2015) ‘Creating research impact: The roles of research users in interactive research mobilisation’, Evidence & Policy: A Journal of Research, Debate and Practice, 11(1): 35–55.
Morton, S. and Casey, T. (2017) Changing National Policy on Violence Affecting Children: An impact assessment of UNICEF and partners’ multi-country study on the drivers of violence affecting children in Peru, Edinburgh: University of Edinburgh.
Morton, S., Grant, A., Cook, A., Berry, H., McMellon, C., Robbin, M. and Ipince, A. (2019) Children’s Experiences Online: Building global understanding and action, Innocenti [online]. Available from: https://www.unicef-irc.org/publications/1065-childrens-experiences-online-building-global-understanding-and-action.html [Accessed 7 October 2021].
National Performance Framework, Scottish Government website [online]. Available from: https://nationalperformance.gov.scot/ [Accessed 5 October 2021].
Needham, C. and Mangan, C. (2016) ‘The 21st century public servant: Working at three boundaries of public and private’, Public Money and Management, 36(4): 265–272.
Osborne, S. and Strokosch, K. (2013) ‘It takes two to tango? Understanding the co-production of public services by integrating the services management and public administration perspectives’, British Journal of Management, 24: 1–29.
Patton, M.Q. (2011) Developmental Evaluation: Applying complexity concepts to enhance innovation and use, New York: Guilford Press.
Public Health England (2019) ‘Health matters: Whole systems approach to obesity’, UK Government Publication [online] 25 July. Available from: https://www.gov.uk/government/publications/health-matters-whole-systems-approach-to-obesity/health-matters-whole-systems-approach-to-obesity [Accessed 4 October 2021].
Rogers, P.J. (2008) ‘Using programme theory to evaluate complicated and complex aspects of interventions’, Evaluation, 14(1): 29–48.
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H. and Jinks, C. (2018) ‘Saturation in qualitative research: Exploring its conceptualization and operationalization’, Quality & Quantity, 52(4): 1893–1907.
Schein, E.H. (1985) ‘Defining organizational culture’, Classics of Organization Theory, 3(1): 490–502.
Senge, P.M. (1990) The Fifth Discipline, New York: Random House Business.
Shove, E., Pantzar, M. and Watson, M. (2012) The Dynamics of Social Practice: Everyday life and how it changes, London: SAGE Publications Ltd.
Social Care Wales (2022) ‘Friend not foe: Supporting meaningful outcomes focused recording in social care in Wales’, Welsh Government [online]. Available from: https://socialcare.wales/cms_assets/file-uploads/Friend-not-Foe-English.pdf [Accessed 25 February 2022].
The Manchester Partnership (2021) ‘Public Service Reform’ [online]. Available from: https://www.manchester.gov.uk/manchesterpartnership/info/5/public_service_reform [Accessed 4 October 2021].
United Nations Sustainable Development Goals [online]. Available from: https://sdgs.un.org/ [Accessed 6 October 2021].
Weiss, C.H. (1998) Evaluation: Methods for studying programs and policies, Upper Saddle River, NJ: Prentice Hall.
Whitebloom, S. (2020) ‘“Care” can and should work but it is a postcode lottery for young care leavers’, Oxford News Blog, Oxford: University of Oxford website [online]. Available from: https://www.ox.ac.uk/news/arts-blog/care-can-and-should-work-it-postcode-lottery-young-care-leavers [Accessed 5 October 2021].
World Health Organization (2016) ‘INSPIRE: 7 strategies to end violence against children’ [online]. Available from: https://www.who.int/publications/i/item/inspire-seven-strategies-for-ending-violence-against-children [Accessed 7 October 2021].
|May 2022 onwards||Past Year||Past 30 Days|
|Full Text Views||0||0||0|