How Do You Know If You Are Making a Difference?
A Practical Handbook for Public Service Organisations
Authors: and

4: Owning your initiative’s outcomes and impacts

Authors: and

Being clear about the difference an initiative wants to make and the outcomes or the impacts of the work is the first step in knowing whether it is making a difference. Many organisations need to report to, or align with, multiple outcome frameworks and indicators. This can result in organisations feeling caught in a tangle of outcomes. Being able to navigate around this complex outcomes and impacts landscape is essential for anyone leading public services where they are expected to work towards, report on or be evaluated against outcomes or impacts. This is what we call ‘owning your outcomes or impacts’.

This chapter offers accessible ways to understand and operationalise outcomes and impacts and unpack some of the contradictions surrounding the use of outcomes and impacts within wider policy and funding systems.

  • Ahmed, I.K. and Dantata, B.S. (2016) ‘Problems and challenges of policy implementation for national development’, Research on Humanity and Social Sciences, 6(15): 6065.

    • Search Google Scholar
    • Export Citation
  • Archibald, T., Sharrock, G., Buckley, J. and Cook, N. (2016) ‘Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development’, Evaluation and Program Planning, 59: 119127.

    • Search Google Scholar
    • Export Citation
  • Australian Government (2019) ‘Delivering for Australians. A world-class Australian public service: The government’s APS reform agenda’, Department of the Prime Minister and Cabinet [online] 13 December. Available from: https://www.pmc.gov.au/resource-centre/government/delivering-for-australians [Accessed 4 October 2021].

  • Bicket, M., Christie, I., Gilbert, N., Hills, D., Penn, A. and Wilkinson, H. (2020) Magenta Book 2020: Supplementary Guide: Handling Complexity in Policy Evaluation, London: HM Treasury.

    • Search Google Scholar
    • Export Citation
  • Black, L. (2021) ‘Funding Utopia’ [online]. Available from: https://leah-black.medium.com/funding-utopia-2-ten-fifteen-years-unrestricted-outcomes-community-wealth-and-more-b32760b32dd3 [Accessed 15 March 2022].

  • Blamey, A. and Mackenzie, M. (2007) ‘Theories of change and realistic evaluation: Peas in a pod or apples and oranges?’, Evaluation, 13(4): 439455.

    • Search Google Scholar
    • Export Citation
  • Bozarth, J.D. (2013) ‘Unconditional positive regard’, in M. Cooper, M. O’Hara, P.F. Schmid and A.C. Bohart (eds) The Handbook of Person-Centred Psychotherapy and Counselling, New York: Palgrave Macmillan/Springer Nature, pp 180192.

    • Search Google Scholar
    • Export Citation
  • Cameron, W.B. (1969) Informal Sociology: A casual introduction to sociological thinking, New York: Random House.

  • CECAN – Centre for Evaluating Complexity Across the Nexus (2020) ‘Supplementary Guide: Handling complexity in policy evaluation’, HM Treasury [online]. Available from: https://www.cecan.ac.uk/news/handling-complexity-in-policy-evaluation-magenta-book-2020-supplementary-guide/ [Accessed 4 October 2021].

  • Christie, C. (2011) ‘Christie Commission on the future delivery of public services’, Scottish Government [online] 29 June. Available from: https://www.gov.scot/publications/commission-future-delivery-public-services/ [Accessed 4 October 2021].

  • Cook, A. (2015) ‘Partnership working across UK public services: Evidence review’, Edinburgh: What Works Scotland [online]. Available from: http://whatworksscotland.ac.uk/wp-content/uploads/2015/12/WWS-Evidence-Review-Partnership-03-Dec-2015-.pdf [Accessed 7 October 2021].

    • Search Google Scholar
    • Export Citation
  • Cook, A. (2017) ‘Outcome based approaches in public service reform’, Edinburgh: What Works Scotland [online]. Available from: http://whatworksscotland.ac.uk/wp-content/uploads/2017/04/OutcomeBasedApproachesinPublicServiceReform.pdf [Accessed 25 February 2022].

    • Search Google Scholar
    • Export Citation
  • Cook, A. and Miller, E. (2012) Talking Points Personal Outcomes Approach: A practical guide for organisations, Edinburgh: Joint Improvement Team, Scottish Government.

    • Search Google Scholar
    • Export Citation
  • Crawford, C., Edwards, A., Farquharson, C., Greaves, E., Trevelyan, G., Wallace, E. and White, C. (2016) Magic Breakfast: Evaluation report and executive summary, London: The Education Endowment Foundation.

    • Search Google Scholar
    • Export Citation
  • Crewe, E. and Young, J. (2002) Bridging Research and Policy: Context, evidence and links, London: Overseas Development Institute.

  • Darnton, A. and Horne, J. (2013) ‘Influencing behaviours – moving beyond the individual: ISM user guide’, Scottish Government [online]. Available from: https://www.gov.scot/publications/influencing-behaviours-moving-beyond-individual-user-guide-ism-tool/ [Accessed 1 March 2022].

  • Davies, H.T. and Nutley, S.M. (2000) ‘Healthcare: Evidence to the fore’, in H.T. Davies, S.M. Nutley and P.C. Smith (eds) What Works: Evidence-based policy and practice in public services, Bristol: Policy Press, pp 4367.

    • Search Google Scholar
    • Export Citation
  • Gabbay, J. and le May, A. (2004) ‘Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care’, BMJ, 329: 1013 https://doi.org/10.1136/bmj.329.7473.1013

    • Search Google Scholar
    • Export Citation
  • Garvin, D.A. (1993) ‘Building a learning organization’, Harvard Business Review, 71(4): 7891 [online]. Available from: https://hbr.org/1993/07/building-a-learning-organization [Accessed 5 October 2021].

    • Search Google Scholar
    • Export Citation
  • Glendinning, C., Clarke, S., Hare, P., Kitchetkova, I., Maddison, J. and Newbronner, L. (2006) Outcomes-focused Services for Older People, Bristol: Policy Press.

    • Search Google Scholar
    • Export Citation
  • Government of Canada. ‘Theory-based approaches to evaluation: Concept and practices’ [online]. Available from: https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/theory-based-approaches-evaluation-concepts-practices.html [Accessed 5 October 2021].

  • Huff, D. and Geis, I. (1954) How to Lie with Statistics, New York: Norton.

  • Matter of Focus ‘Understand the unique context of your work’ [online]. Available from: https://www.matter-of-focus.com/understand-the-unique-context-of-your-work/ [Accessed 5 October 2021].

  • Matter of Focus ‘3 feedback tools to help you track your outcomes and impact’ [online]. Available from: https://www.matter-of-focus.com/3-feedback-tools/ [Accessed 7 October 2021].

  • Mayne, J. (2008) ‘Contribution analysis: An approach to exploring cause and effect’, ILAC Brief 16, pp 14. Available from: https://hdl.handle.net/10568/70124 [Accessed 7 October 2021].

    • Search Google Scholar
    • Export Citation
  • Miller, E. and Barrie, K. (2016) ‘Learning from the Meaningful and Measurable Project: Strengthening links between identity, action and decision-making’, Healthcare Improvement Scotland [online]. Available from: https://ihub.scot/media/2126/po-learning.pdf [Accessed 25 February 2022].

  • Montague, S. (2012) ‘Theory-based approaches for practical evaluation’ [online]. Available from: https://www.pmn.net/wp-content/uploads/Theory-based-Approaches-for-Practical-Evaluation.pdf [Accessed 5 October 2021].

  • Morton, S. (2012) Exploring and Assessing Social Research Impact: A case study of a research partnership’s impacts on policy and practice, Edinburgh: University of Edinburgh.

    • Search Google Scholar
    • Export Citation
  • Morton, S. (2015) ‘Creating research impact: The roles of research users in interactive research mobilisation’, Evidence & Policy: A Journal of Research, Debate and Practice, 11(1): 3555.

    • Search Google Scholar
    • Export Citation
  • Morton, S. and Casey, T. (2017) Changing National Policy on Violence Affecting Children: An impact assessment of UNICEF and partners’ multi-country study on the drivers of violence affecting children in Peru, Edinburgh: University of Edinburgh.

    • Search Google Scholar
    • Export Citation
  • Morton, S., Grant, A., Cook, A., Berry, H., McMellon, C., Robbin, M. and Ipince, A. (2019) Children’s Experiences Online: Building global understanding and action, Innocenti [online]. Available from: https://www.unicef-irc.org/publications/1065-childrens-experiences-online-building-global-understanding-and-action.html [Accessed 7 October 2021].

    • Search Google Scholar
    • Export Citation
  • National Performance Framework, Scottish Government website [online]. Available from: https://nationalperformance.gov.scot/ [Accessed 5 October 2021].

  • Needham, C. and Mangan, C. (2016) ‘The 21st century public servant: Working at three boundaries of public and private’, Public Money and Management, 36(4): 265272.

    • Search Google Scholar
    • Export Citation
  • Osborne, S. and Strokosch, K. (2013) ‘It takes two to tango? Understanding the co-production of public services by integrating the services management and public administration perspectives’, British Journal of Management, 24: 129.

    • Search Google Scholar
    • Export Citation
  • Patton, M.Q. (2011) Developmental Evaluation: Applying complexity concepts to enhance innovation and use, New York: Guilford Press.

  • Public Health England (2019) ‘Health matters: Whole systems approach to obesity’, UK Government Publication [online] 25 July. Available from: https://www.gov.uk/government/publications/health-matters-whole-systems-approach-to-obesity/health-matters-whole-systems-approach-to-obesity [Accessed 4 October 2021].

  • Rogers, P.J. (2008) ‘Using programme theory to evaluate complicated and complex aspects of interventions’, Evaluation, 14(1): 2948.

    • Search Google Scholar
    • Export Citation
  • Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H. and Jinks, C. (2018) ‘Saturation in qualitative research: Exploring its conceptualization and operationalization’, Quality & Quantity, 52(4): 18931907.

    • Search Google Scholar
    • Export Citation
  • Schein, E.H. (1985) ‘Defining organizational culture’, Classics of Organization Theory, 3(1): 490502.

  • Senge, P.M. (1990) The Fifth Discipline, New York: Random House Business.

  • Shove, E., Pantzar, M. and Watson, M. (2012) The Dynamics of Social Practice: Everyday life and how it changes, London: SAGE Publications Ltd.

    • Search Google Scholar
    • Export Citation
  • Social Care Wales (2022) ‘Friend not foe: Supporting meaningful outcomes focused recording in social care in Wales’, Welsh Government [online]. Available from: https://socialcare.wales/cms_assets/file-uploads/Friend-not-Foe-English.pdf [Accessed 25 February 2022].

  • The Manchester Partnership (2021) ‘Public Service Reform’ [online]. Available from: https://www.manchester.gov.uk/manchesterpartnership/info/5/public_service_reform [Accessed 4 October 2021].

  • United Nations Sustainable Development Goals [online]. Available from: https://sdgs.un.org/ [Accessed 6 October 2021].

  • Weiss, C.H. (1998) Evaluation: Methods for studying programs and policies, Upper Saddle River, NJ: Prentice Hall.

  • Whitebloom, S. (2020) ‘“Care” can and should work but it is a postcode lottery for young care leavers’, Oxford News Blog, Oxford: University of Oxford website [online]. Available from: https://www.ox.ac.uk/news/arts-blog/care-can-and-should-work-it-postcode-lottery-young-care-leavers [Accessed 5 October 2021].

    • Search Google Scholar
    • Export Citation
  • World Health Organization (2016) ‘INSPIRE: 7 strategies to end violence against children’ [online]. Available from: https://www.who.int/publications/i/item/inspire-seven-strategies-for-ending-violence-against-children [Accessed 7 October 2021].

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 20 20 0
Full Text Views 0 0 0
PDF Downloads 0 0 0

Altmetrics