Search Results
An impact assessment of research into children's concerns about their families and relationships found many ways research had been used in different sectors by different actors. Specific impacts from the research were harder to identify. However, instances where there were clear impacts highlighted the ways research users had adapted research to fit the context for research use in order to create impact. Research users continued to draw on the research for many years after publication, creating further impact as new policy or practice agendas arose.
Why is it hard to know if you are making a difference in public services? What can you do about it?
Public services throughout the world face the challenge of tackling complex issues where multiple factors influence change. This book sets out practical and theoretically robust, tried and tested approaches to understanding and tracking change that any organisation can use to ensure it makes a difference to the people it cares about.
With case studies from health, community, research, international development and social care, this book shows that with the right tools and techniques, public services can track their contribution to social change and become more efficient and effective.
Barriers to using research in practice are well documented. This paper describes an innovative process developed by the Centre for Research on Families and Relationships to address these barriers. We supported people to define what they needed to know; how existing evidence could help; and how to use evidence in practice. An action-focused evidence review process was developed to synthesise and appraise varied relevant literatures, and communicate this in meaningful, timely, relevant and action-focused ways. Both making evidence accessible and facilitating processes for deliberating evidence were essential in supporting evidence users to understand the extent and usefulness of evidence and identify implications for policy, practice and services.
This chapter explores why it is so hard to know if projects, programmes, partnerships and organisations are making a difference, and what is needed to go on the journey to be able to make evidenced claims that really demonstrate impact. It introduces why we have written this book, and what will be covered in the rest of the chapters, including an overview of our approach.
Public services sit in complex webs of programmes, policies and practices and are not in the control of any one organisation. Change happens through a web of interlinked and overlapping approaches and services, which makes it difficult to understand. Engaging with this complexity is essential for knowing whether any initiative is making a difference, but can be challenging. This chapter explores what needs to be taken into account to work within complex systems when seeking to understand and track outcomes and impact.
This chapter explores what data, evidence and feedback practices are helpful or unhelpful for knowing whether initiatives are having their intended impact. It considers some of the cultures and practices that can get in the way of having the required feedback for understanding change.
It sets out the kinds of data and evidence that are best suited to understanding complex change. Drawing on experience of working in different settings, this chapter explores the concept of ‘data cultures’ and how these data cultures can help or hinder initiatives in getting the data they need. Examples highlight ways of applying this thinking to any specific work, especially making the case for using mixed data that include qualitative data. The concept of a data improvement journey is introduced, along with practical ways of thinking about the data and evidence needs in any specific setting.
Being clear about the difference an initiative wants to make and the outcomes or the impacts of the work is the first step in knowing whether it is making a difference. Many organisations need to report to, or align with, multiple outcome frameworks and indicators. This can result in organisations feeling caught in a tangle of outcomes. Being able to navigate around this complex outcomes and impacts landscape is essential for anyone leading public services where they are expected to work towards, report on or be evaluated against outcomes or impacts. This is what we call ‘owning your outcomes or impacts’.
This chapter offers accessible ways to understand and operationalise outcomes and impacts and unpack some of the contradictions surrounding the use of outcomes and impacts within wider policy and funding systems.
Whether supporting people to move on in their lives, implementing new policy initiatives, providing training or spreading good practice, the evidence shows that the most effective approaches are those that are tailored to the specific context. Rather than ignoring it, trying to work well with the challenges and opportunities of any initiative’s context can help improve delivery and really make a difference to people and communities. It’s also essential for shaping how the initiative will make a difference, and for tracking that difference. This chapter discusses why context matters and sets out some practical ways of working effectively with the complex context in which anyone seeking to make a difference finds themselves.
A meaningful connection between activities and outcomes is often vague or unexplored – but this magic ‘how’ change happens is essential for driving programmes and for understanding their impact. Outcome or impact maps are the core of the approach in this book, as the building block for setting out, understanding, learning about and evidencing change. This is one type of ‘theory of change’ approach which is common for understanding change in complex systems. We believe that process-driven theories of change are most effective for public services, and in this chapter we set out our version of this – outcome or impact mapping – and illustrate how they can be used at different levels.
Picking up on the issues from Chapter 3 about getting the data, evidence and feedback needed to understand complex change, this chapter presents some practical ways to improve and use data. It includes how to undertake a data audit, filling gaps in data, and practical data collection techniques. It provides some examples of how these have been applied in different public services settings. It aims to help organisations get all of the data and information they need to evaluate their progress towards outcomes and demonstrate their impact.
