Using knowledge brokering to produce community-generated evidence

View author details View Less
  • 1 University of Sheffield, , UK
  • | 2 University of Alberta, , Canada
  • | 3 Manor & Castle Development Trust, , UK
  • | 4 SOAR (Southey and Owlerton Area Regeneration), , UK
  • | 5 University of Sheffield, , UK
Open access
Get eTOC alerts
Rights and permissions Cite this article

Background:

Devolution and integration of health and social care have placed increasing pressure on local statutory services, with a corresponding shift of health and social care to community organisations. The voluntary and charitable sector (VCS) is expected to make the case for increased funding by providing evidence of value and impact.

Aims and objectives:

This paper explores the challenges of compiling evidence on health outcomes which do not reflect the holistic nature of VCS support. We document how knowledge brokering can be used to enable the VCS to generate evidence.

Key conclusions:

Knowledge brokering (KB) may be an effective approach for developing community-generated evidence. Brokering is also needed to change perspectives on what counts as good evidence

Abstract

Background:

Devolution and integration of health and social care have placed increasing pressure on local statutory services, with a corresponding shift of health and social care to community organisations. The voluntary and charitable sector (VCS) is expected to make the case for increased funding by providing evidence of value and impact.

Aims and objectives:

This paper explores the challenges of compiling evidence on health outcomes which do not reflect the holistic nature of VCS support. We document how knowledge brokering can be used to enable the VCS to generate evidence.

Key conclusions:

Knowledge brokering (KB) may be an effective approach for developing community-generated evidence. Brokering is also needed to change perspectives on what counts as good evidence

Key messages

  • Health outcome measures are not seen to be appropriate by the voluntary sector for social prescribing services.

  • A new evidence base is needed that reflects the social determinants of health.

  • Knowledge brokering may be an effective approach for developing community-generated evidence.

  • Brokering is also needed to change perspectives on what counts as good evidence.

Introduction

This paper describes how the process of knowledge brokering (KB) has been used over the past six years to generate more relevant forms of evidence documenting the value of voluntary and charitable sector (VCS) services.

We start by describing the policy context, outlining how austerity has led to cuts in public services and a drive for community-based solutions delivered by the VCS. We then describe the ‘evidence tension’ that was subsequently created when traditional methods for evaluating health outcomes were challenged by community organisations, and reflect on the role of knowledge brokering in realising a community evidence base.

Over the past nine years, there has been an increasing trend in UK Government policy for local government in England to devolve public services to local levels, prompted by the global financial crisis and policies for economic austerity (Lowndes and Pratchett, 2012). This was branded as the ‘Big Society’, which aimed to foster innovative approaches for dealing with reduced resources at the local authority level by engaging local citizens, volunteers and community organisations in service provision. This was described as a ‘huge cultural change… where people don’t always turn to officials, local authorities or central government for answers to the problems they face’ (Cameron, 2010).

With the increased responsibility, however, came major funding cuts to local authorities, placing communities in the position of having to do more with considerably less resource, especially in health and social care. The funding gap for publicly funded social care is currently estimated at £1.3 billion and the Local Government Association predicts a shortfall of £3.6 billion by 2024/25 (LGA, 2018). At the same time, there has been a steady increase in proportions of people with multiple and complex health conditions but also individuals with non-medical needs such as housing issues and problems accessing welfare benefits. This increasing demand has created pressure to develop new models of care where patients are linked to a wider range of resources in the VCS (Baird, 2018). For example NHS England (2019) have established social prescribing services where link workers support patients from primary care to access support from the VCS (NHS England, 2019). Because these initiatives are led and funded for the most part by the health sector, the services are conceptualised as episodes of care and the measures of success are those traditionally used in health services, which are such as length of treatment or completion rates (throughput) and achievement of mental and/or physical wellbeing (Comptroller and Auditor General, 2018). This can be incompatible with the holistic support delivered by VCS, where service users access different services on an ongoing basis, making it challenging to conceptualise the support as discrete episodes of care. Furthermore, services delivered by the VCS are generally aimed at supporting people to become capable of addressing the non-medical, social determinants of health, which encompass the conditions under which people live and the corresponding economic, political and social systems which influence health. Services include offering welfare advice, advocacy, social support, local environment and sustainable development, and community sector development (Henderson et al, 2018). This has led to a national debate questioning the appropriateness of using outcome measures that are traditionally used in primary and acute healthcare sectors (Gamsu et al, 2019). Despite this, commissioners funding VCS services continue to mandate the use of health outcomes as a condition of receiving funding (Foster et al, 2020). Additionally, the VCS infrastructure is not funded to collect this evidence, creating a conundrum where longer term funding cannot be obtained because there is no resource to create an evidence base (Foster et al, 2020). Given these challenges, knowledge brokerage was used to create a consensus on how to evidence the impact of the VCS, with the focus of this paper reporting on a case study of the brokering process and experience.

Methods

Knowledge mobilisation was used in one city, over a six-year period, to explore the feasibility of co-creating an evidence base that reflected what the VCS does to promote health and wellbeing, in the broadest sense of the term.

The knowledge-brokering process was instigated by one individual (JH), with the aim of identifying existing brokers and fostering connections to facilitate the creation of a shared knowledge base. Knowledge brokering was defined as a transdisciplinary, inclusive, iterative approach to enlisting VCS organisations, academic researchers and commissioners in the process of creating information (Schuttenberg and Guth, 2015). In the first stages of the project, we aimed to create a knowledge system framework, across four VCS organisations that support people in their local neighbourhoods to develop capabilities to deal with issues of employment, finance, housing, education, wellbeing and health (referred to as community organisations). By their own admission, these organisations stated that they had little history of working together because existing funding models created competition. We therefore used a participatory approach to brokering, which aims to maximise the participation of the organisations in the process of deciding the relevant outcomes for their services and the appropriate tools and methods for measurement (ICPHR, 2013). Further, a participatory approach gives equal value to local and tacit knowledge, promoting a collaborative process, which in turn leads to local and collective ownership. We felt that this was key to the co-production of an evidence base. As this knowledge base was created, we adopted a transactional model of knowledge brokering, where links with commissioners were forged in order to get them to consider community-generated knowledge alongside evidence of health outcomes (Ward et al, 2009).

The brokering approach was long-term, using informal engagement and participatory networking (Murdock et al, 2013) to facilitate interactions across community organisations and with commissioners, in order to lead to a transformational understanding of effective community support (Blackstock et al, 2007; Pohl, 2008; Lang et al, 2012). Brokering success was defined as shifting the focus from evidence generated using traditional methods for collecting health outcomes, to production of alternative forms of evidence more appropriate for community organisations (Roth, 2003). We used an adapted version of Ward’s framework for knowledge mobilisation (Ward, 2017) to describe the process of generating new forms of knowledge (Box 1).

Box 1 Approach to knowledge generation

WHO will be included: community service providers, professional knowledge producers (researchers, evaluators) and commissioners ot services.

WHAT type of knowledge will be drawn upon: professional judgement of valid approaches to measurement will be combined with organisations’ experience and judgements of relevant tools and methods.

HOW will knowledge be generated: by making connections between community organisations, facilitating interactive learning and co-production of methods for generating evidence, and brokering relationships between organisations and commissioners.

WHY will knowledge be generated: to produce useful and robust evidence that can be used to inform commissioners of appropriate and relevant methods for evaluating community outcomes.

There were six knowledge mobilisation questions that were evaluated during the course of the project (Box 2).

Box 2 Knowledge mobilisation questions

  1. Can we identify relevant tools and methods to capture client outcomes?

  2. Can we get consensus on client-valued outcomes?

  3. Can community organisations find the capacity to pilot tools and assess their feasibility?

  4. Can organisations agree with commissioners a uniform approach to measuring outcomes?

  5. Can organisations find the resources to conduct ongoing monitoring and evaluation (M&E)?

  6. Can their evidence be used to shift funding criteria from a health outcomes focus to a more holistic conceptualisation of promoting capability and wellbeing?

A case-study approach (Yin, 2009) was used in construct a preliminary logic model of how various KB roles might work (Table 1).

Table 1:

Logic model for knowledge brokering

KB roleShort-term effectsOutcomes
  • Dissemination

    • Share commonly used tools and methods

  • Exchange

    • Promote exchange of experiences of measuring health outcomes in community settings

    • Encourage critique of tools and methods, asking people to draw upon their experiential and tacit knowledge of assessing client progress

  • Linkage

    • The KB identifies other people who act as natural brokers, establishes relationships with them

Participants
  • realise that they have similar challenges to measurement

  • feel their experiences are validated • are motivated to adapt/find alternative ways to evaluate

  • begin to communicate with each other in the exchange forums

  • informal brokers actively participate in discussions

Increased interest in further participation
  • People from different organisations realise they share common evaluation issues

  • Facilitation of knowledge creation

    • Provide technical support and training to pilot existing measurement tools

  • Linkage agent and translator

    • Bring academics and VCS together to analyse data from pilots

  • Exchange

    • Share findings across organisations

Participants
  • have the skills and confidence to conduct pilots

  • act as brokers within their own organisations

  • make informed decisions about relevance and appropriateness of the adapted tools

  • realise that the types of support offered are similar across different neighbourhoods and groups

Organisations agree that
  • a single tool is not appropriate across all activities and services

  • tools and methods are fit for purpose, and have academic support

  • Intermediary and message bearer

    • Meetings to discuss issues with measuring wellbeing; what community organisations do and how they promote wellbeing

    • Informal feedback to commissioners and academics regarding concerns with tools and outcomes

  • Linkage agent

    • Link community organisations and commissioners up to share experiences of pilot and review relevance of required measures

  • University sponsors the forum and presents research on measuring wellbeing

  • Commissioners support the process of developing and adapting tools and identifying meaningful outcomes

Facilitating diffusion of knowledge
  • Organising meetings and shared learning forums where evidence is used to develop client valued outcomes; a theory is co-produced for how and why community support works

  • Consensus across organisations on client-valued outcomes

  • Agreement with commissioners on a minimum core data set

  • Organisations given more autonomy in choice of measurement tools to reflect client-valued outcomes

Community organisations were given some project-specific funding to develop evidence. For example, funding from the Big Lottery contains requirements for funding, as do other VCS funders. The development of the knowledge framework, and ongoing technical assistance was funded by a National Institute of Health Research (NIHR) Fellowship in Knowledge Mobilisation. This also contained funding earmarked for organisations to pilot their evaluation methods. The Fellowship was sponsored by two of the community organisations, public health commissioning and the university to promote transactional brokering.

A number of sources of data were used within the case study, which included: documentation (attendance, levels of participation, numbers of meetings, meeting notes; decisions taken); group and individual conversations and critical reflection, and unsolicited feedback where participants directly attributed an output or impact to the KB process. We also documented increase in requests for academic input, increased networking leading to consensus, changes in monitoring and evaluation (M&E) processes, increase in collaborative cross-organisation and cross-sectoral funding proposals, and development of partnerships (Table 2). The information was co-produced and verified by participating organisations during the course of the initiative.

Table 2:

Relationship between KB activities, outputs, and outcomes

QuestionKB activitiesKB outputsKB outcomes
1. Identifying relevant evaluation tools and methods
  • Participatory critical review (3 meetings) interspersed with individual visits to community organisations

  • Training in participatory evaluation (2 courses)

  • Ongoing technical support with small-scale evaluations

  • Participation by 14 community organisations in critical review of tools (1)

  • Attendance by 38 people from 25 community organisations

  • Acknowledgement that community services were too diverse for one mode of measurement

  • Commissioners agree to a range of tools in the Toolkit

  • Invitations for university/ community co-production of 7 funding proposals

2. Developing client-valued outcomes
  • Community workers trained to conduct interviews (2, 3, 4)

  • Interviews with clients (2, 3, 4)

  • Most Significant Change stories triangulated with interviews (3, 4)

  • Collaborative review of existing case studies triangulated with interviews and stories (1, 2, 3, 4)

  • A list of client-valued outcomes is co-produced

  • Co-production of a description of how social prescribing is offered in different communities

  • Different community organisations agree to take collective action to increase understanding of what they offer

  • Commissioners agree not to stipulate a single social prescribing model

  • Community organisations produce their first collaborative funding application

3. Capacity to pilot evaluation tools
  • Tools developed by workers

  • Pilots conducted by organisations (4)

  • Cross-sector Engaged Learning events to describe process of creating a community evidence base (4)

  • Needs for technical assistance, equipment, training identified

  • Academic department provides ad hoc support on an ongoing basis

  • Organisations develop closer relations with academic researchers

  • Five projects funded, leading to more partnership working

4. Agreement on outcome measurement
  • Forum convened to present findings to commissioners and academics (4)

  • KB invited to facilitate workshops where organisations share their approaches to social prescribing

  • Consensus that single approach to measurement would not reflect diversity of services and populations

  • Feedback from commissioners re increased insight into what organisations actually provide

  • Decisions on what tools are appropriate for what circumstances and which activities/ services

  • Organisations host and co-produce 4 additional projects with researchers (independent of the original KB)

  • Emerging partnerships across organisations

  • One organisation agrees to host a NIHR Fellowship on developing People Reported Outcomes Measures (Foster, 2020; Foster et al, 2019)

  • Co-produced applications for funding

  • Emerging partnerships across organisations

  • Commissioners take a flexible approach where they proscribe required monitoring data, but organisations can stipulate the tools that will be used for evaluation

5. Resources for M&E
  • Organisations use the piloting experiences to estimate required resources

  • Support for M&E is consistently requested in funding applications

  • Successfully resourced in 3 further applications

  • The Board of one organisation agrees to fund maintenance of an evidence database

6. Co-produced evidence base with a shift in focus to capability and wellbeing
  • VCS obtains funding from primary care to establish partnership working

  • VCS is given access to patient records to input client outcomes from community support services

  • People with expertise in primary care record systems meet with VCS

  • VCS is enabled to negotiate the input of client outcomes into patient records

  • Capacity to co-produce evaluations of the value of community support to primary care

* Projects: (1) Community Evaluation Toolkit; (2) People Keeping Well in the Community; (3) TImebuilders; (4) NIHR Fellowship in Knowledge Mobilisation (4 community anchor organisations)

Findings

What happened as a direct result of brokering is summarised in Table 2. We discuss how the process maximised participation, fostered knowledge exchange, promoted collaboration and facilitated collective production of an evidence base.

Identifying tools

Organisations actively participated in discussions to review potential outcome measures. This was partly because the discussions took place in routine funding meetings held between 14 local community organisations and public health commissioners. The broker (JH) used a participatory approach, which emphasised from the beginning that tools needed to be appropriate and relevant to community settings. This enabled participants to constructively criticise traditional tools. All perspectives were valued and documented. Initially, discussions were held with individual community organisations, because by their own admission VCS organisations have a history of having to compete with one another for funding. The roles of messenger and intermediary were extensively used in this phase. Conversations outside meetings, with individuals, were used to illustrate that everyone had the same concerns about the relevance and appropriateness of specific outcome measures to community organisations, regardless of the very different neighbourhoods and groups that they served. Visits to each organisation to review their existing tools and methods also revealed similar challenges: different activities and diverse needs for support within organisations had different aims and outcomes, meaning that it would be inappropriate for the same outcome tool to be used by all of the involved organisations. These explorations served to create relationships between the broker and individual organisations. The broker noted commonality of issues during each individual visit, which created interest in organisations to form a community of practice, where they met together because of having similar conceptions of what works. Through meeting together, a ‘knowledge space’ was created, for example, a forum where all types of knowledge are exchanged and equally valued (Ranga and Etzkowitz, 2015). The meetings brought people together to develop an emerging consensus about what needed to be done in terms of deciding how to evaluate success.

Across organisations, the broker’s role was to explore whether different organisations base their work on bodies of knowledge that are developed by different professions (Brown and Duguid, 1998). Comparing and contrasting what was valued indicated that organisations serving different communities had very similar conceptions of what works. This means that although the organisations operated independently of one another, they in effect formed a community of practice that drew upon the same knowledge base to provide and evaluate services. What emerged from the meetings was a realisation that the evidence valued by healthcare was very different from the knowledge valued by community organisations, and there was a need to build understanding of the different types of knowledge systems (Walter et al, 2007). Organisations were concerned that measures are designed to be used by researchers and clinicians in research studies, rather than being designed for use with clients to reflect on progress. Community workers felt that the tools use academic language which is difficult to understand and does not align with the usual conversations that they have with service users. The VSC also felt that the tools aim to extract and measure the progress that the health system deems important, whereas workers aim to get their clients to offer reflection on individual journeys using their own markers of the distance travelled. Further, the VCS is often asked to measure improvements over a predefined, fairly short period of time, whereas the organisations felt their work has longer-term impact, which is not captured by current tools. The consensus on issues enabled the VCS group to argue against a single city-wide tool or standardised approach to evaluation, when it was proposed by commissioners.

As a result of the frequent contact and communication, interest amongst the community organisations in receiving training to develop evaluation skills was high. The training gave some of the participants confidence to design evaluations. These sessions also served as a bridge to academia, with the broker linking needs for technical assistance with university experts. In the initial academic/VCS meetings, the broker acted as a translator, being alert for the jargon used by the different sectors, and modelling how people needed to be alert to – and question – things that they didn’t understand. Attendance at further training to teach interview and data collection skills was also high. People said they attended because they believed that they would be able to use the skills right away to document the value of their services. Organisations became active partners in collecting and analysing data to produce an agreed list of client outcomes.

Developing client-valued outcomes

Organisations agreed that although health outcomes are required by many funders, the outcomes valued by clients may not necessarily focus on health. Identifying client-valued outcomes was very much a team effort, with each organisation contributing staff time to review existing data and collect new data (Table 2). Different methods were used to triangulate the findings. The outcomes were verified via a very different exercise, which concerned the need to produce a description of what organisations do, and how clients benefit for special prescribing. The city was considering making it a requirement to provide one of their services – social prescribing - in a standardised way. Social prescribing is a nationally supported programme that enables health professionals to refer people to community link workers, who in turn connect people with local community services that support them in addressing social determinants of health. A knowledge space was created where organisations used visual scribing to produce an overarching description of what worked. Constructing drawings presented a different way of interacting for members of the group, opening up a communicative space that helped to articulate the service (Habermas, 2015). The description was then translated into a co-produced document outlining how and why the service worked. The document was used with commissioners, to highlight that although there was a common process for providing social prescribing services, delivery needed to be responsive to the need for a diverse range of support in different neighbourhoods. Co-production of the description was key to catalysing further development of the community evidence base (Paavola and Hakkarainen (2005). To further the knowledge, a forum was created between the community organisations, commissioners and academics to describe how social prescribing is offered and how it may be evaluated.

What emerged from this process was communicative action – a process whereby people deliberately aimed for agreement about the value of what they provide, and achieved an unforced consensus about how to move forward (Kemmis and McTaggart, 2005). The consensus on valued outcomes, across different types of clients and sites, made a convincing case for commissioners to adjust their expectations of the type of outcomes data that needed to be collected. Agreement on outcome measurement, however, was threatened by a city council proposal to adopt a uniform approach to social prescribing. Because organisations had insight into their common issues, they invited the broker to facilitate theory-building sessions, with the aim of producing a description of what worked in which circumstances with social prescribing. As a result of this experience, the organisations acted collectively to bargain for their theory of social prescribing, which was used to co-produce a city-wide funding proposal. This was the first city-wide funding proposal of its kind, and participants believed that it would not have happened without the brokering.

As time went on, informal brokers in each organisation took control of decisions about how to generate evidence. It was decided that tools needed to be fit for purpose, for example, relevant for measuring the different types of services provided in each organisation; and approaches to collecting data needed to align with their current ways of working with clients (for example, using existing case studies). Some organisations are taking the lead on developing wellbeing tools for evaluation, while others have focused more on developing and validating client-valued outcomes (Figure 1).

In the illustration, there is a square with curved edges and double-layered sides. The square is divided into three sections by two lines. The third section houses a rectangle with curved edges. On the left side of the square there are three circular nodes. Starting from the top, they are strategic provision, performance management, and M and E training. M and E training is connected to a circular node labeled monitoring to inform commissioning via channel formed by a pair of parallel lines. The lines that divided the square into three parts, there are four circular nodes. The top and bottom nodes on the left line are labeled manage demand and cost, respectively and the other two are not labeled. The top and bottom nodes on the right line are labeled plan capacity and utilization. , respectively and the other two are not labeled. The first three nodes on both lines are connected. Between the plan capacity and the first unlabeled circular node on the right line are two tiny rightward horizontal projections. The one on the top is labeled gap analysis and the one on the bottom is labeled needs addressed. Between the two unlabeled circular nodes on the right line are two tiny rightward horizontal projections. The one on the top is labeled provided and the one on the bottom is labeled taken up. Between the second unlabeled circular node and utilization on the right line is a tiny rightward horizontal projection labeled referred. Next to utilization is a circular node labeled client experiences. The bottom right corner of the curved rectangle is close to client experiences. The left side of the rectangle has four tiny rightward projections. Starting from the top they are client valued outcomes, validate with interviews develop social prescribing theory, and review case studies. A circular node above client valued outcomes is connected to the circular node on the projection labeled needs addressed. A line from client experiences connects all those tiny horizontal projections. A tiny vertical downward projection is on the top side of the rectangle and is labeled reporting. On the right of reporting is a line, which is a part of a smaller rectangle inside the first rectangle. This line has a tiny leftward horizontal projection and above and below that projection is are two circular nodes. The one on the top is labeled monitor effectiveness and the one at the bottom is labeled health and wellbeing outcomes. The top and right side of the second and first rectangle are closely spaced while their respective bottom and left sides are spaced wide apart. On the left side of the second rectangle are tiny rightward horizontal projections. Starting from the top they are labeled analyze, refine, pilot, and co-develop. A horizontal line from bottom side of the second rectangle and the right side of the first rectangle moves rightward and pierces through and the square. This line is labeled developmental evaluation. A circular node on the bottom right of the square is labeled health outcomes.
Figure 1:

Generating community evidence: stages of development

Citation: Evidence & Policy 2022; 10.1332/174426421X16190024737973

Mutual inquiry continues, with each organisation exploring solutions for its particular setting, and sharing these with the others. In these ways, the initial knowledge brokering, done by one individual, has shifted to become a knowledge brokerage comprised of many people. This was achieved by identifying informal brokers within each organisation and developing relationships through productive interactions.

Collective action and control are positive indicators that community organisations are actively doing research alongside and with each other, rather than the broker having proprietary status over the research process (Edelstein, 2016). There are a number of activities that have been triggered by the initial brokering, which are now being independently conducted by the brokerage that was created during the project. The knowledge brokerage is in turn producing ‘behind the scenes’ activities and unintended consequences that ripple out from an initiative (Hansen Kollock et al, 2012). Our knowledge brokerage has produced a number of ripple effects (Box 3).

Box 3 Ripple effects from the knowledge brokerage

  • Community-based research placements for students (Andreeva, 2017; Lunn, 2018; Harris et al, 2018);

  • Community organisations teaching at a university;

  • Supervision of a further Fellowship and four postgraduate students in VCS organisations;

  • Co-produced academic/community funding applications;

  • Funding applications that include knowledge mobilisation as a key role;

  • Increased demand for university consultancy on community and local authority projects;

  • Wider public and patient engagement networks.

Over the past three years, relationships have expanded in several ways. People situated outside the initiative have asked to be involved in learning and training sessions. For example workers, academics and commissioners actively participated in learning Most Significant Change technique – an approach that we used to get the client’s perspective on meaningful outcomes (Dart and Davies, 2003). Some of these participants – who were not part of the original project – volunteered to conduct some of the interviews. Some events, when appropriate, have been attended by a wider group of academics and commissioners, further raising awareness of the initiative across a broader base. As organisations move into partnerships with Primary Care Networks, they are using Transformation Challenge Funding to co-develop evaluations in the neighbourhoods where they work, which cover over half of the city (Figure 2). They have all developed wellbeing measures. Two organisations have completed evaluations; a third will be piloted in autumn 2020, while the fourth organisation needs to adapt evaluation to the change in services triggered by the pandemic.

A shaded and a dashed line run along the boundary of the city, and they depict utilization data collected and cost data to be collected, respectively. A large circle shaded in light colors is on the southeast of the city, and it depicts consensus on what works in community-based support. A small circle shaded in dark colors is on the southeast of the center of the large circle. A vertical rectangle with curved edges is almost half inside the circle and half out of it, and it depicts case studies reviewed; interviews confirm valued outcomes. This rectangle is on the north of the circle. Similar rectangles are on the northeast, northwest, and southeast of the circle. Inside the rectangles on the north and northeast are smaller rectangles that depict wellbeing measures implemented. Inside the rectangle on the southeast is a rectangle with dashed boundaries and it depicts wellbeing measures in development. A similar rectangle but much larger in size is on the west of the circle and it partially intersects the case studies reviewed; interviews confirm valued outcomes rectangle on the northeast of the circle. The wellbeing measures implemented rectangle and wellbeing measures in development are connected.
Figure 2:

Community generated evidence: mapping city-wide progress

Citation: Evidence & Policy 2022; 10.1332/174426421X16190024737973

The role for the initial knowledge broker (JH) is now becoming one of monitoring momentum, providing technical assistance and linking to resources on an as-needed basis.

Discussion and conclusions

In their recent analysis of team brokering, Wye et al (2020) note that we need more comprehensive explanations for how knowledge brokers construct positive interactions and mediate across different institutions and levels of authority. It has been suggested that evaluations could be based on constructing a preliminary theory (Ward, 2017), also recommended in case-study method, which emphasises the importance of creating a preliminary logic model to measure progress (Yin, 2003). Ongoing discussions with our participating organisations confirmed that the elements in the preliminary logic model proved to be important. Several additional elements, however, emerged which include:

  • Knowledge, skills and credibility of the initial broker, and the evolving brokerage;

  • Using national and local ‘windows of opportunity’, where policies and political interests increase the chances that actions will achieve the desired outcomes;

  • Facilitating incremental and productive working;

  • Understanding of organisational capacity and ‘pace’.

Knowledge and skills of the brokers

The initial broker is a former public health commissioner and mental health service provider, who is experienced in conducting participatory evaluation. These multiple roles made it easier to establish credibility across workers, managers and commissioners. While skills in communicating, mediating and networking are key, these are often assessed in terms of individual brokers (Lomas, 2007). Identifying other brokers was a process of finding motivated people, then identifying what skills they could bring to the collaborative effort. Our project indicates that having multiple brokers who possess these skills is key in reaching a consensus about appropriate outcomes for community services.

Using national and local ‘windows of opportunity’

The broker became involved in national networks which were challenged to produce meaningful evaluations; these activities enabled her to show local organisations that their concerns were echoed by others, and that their work was groundbreaking. Second, knowledge brokers can act as information bearers, presenting the wider context of national debates on the relative value of different kinds of knowledge. These debates can promote critical discourse locally about what knowledge is valued.

Facilitating incremental and productive working

Reviews of knowledge brokering state the linkage and exchange process takes considerable time (Ward, 2017). We believe that there are several solutions. The NIHR Fellowship funding was instrumental in the first instance, but it was used to set up productive individual interactions, which convinced people that it would be worthwhile to dedicate further time. An incremental approach was initially taken, where community organisations, academics and commissioners were brought together when needed to progress the work. These periodic and focused interactions made small but productive demands on people’s time. Attendance was consistently high because people trusted that dedicating time would lead to solutions. Regular interactions created opportunities for exchange and served to develop working relationships. As relationships became established, brokers in different places led on various tasks, sharing the load.

Being sensitive to organisational capacity and pace

Knowledge brokers need to understand the organisational context well enough to know how to mobilise at an ‘organisational pace’. The speed at which organisations can participate is influenced by many local factors, and resources within each one may dictate who is actively participating at any given point in time. The knowledge spaces become a critical arena for ensuring that those who are time-poor can continue to gain useful knowledge, even when they are not able to be in the role of knowledge producers. Following the principles of participatory working meant that people were included even when they were unable to join in.

While a linkage and exchange model is commonly used in KB, there are still few reports on its ability to influence decision making (Ward, 2017), for example hierarchies, where decisions about the knowledge produced are made by commissioners. Further, the literature on evaluating partnerships across the health and voluntary sectors is sparse, as noted in recent reviews of primary care partnerships for social prescribing (Husk et al, 2020). The need for appropriate evidence to guide policy decisions has been supported by a number of academics (Petticrew and Roberts, 2003; Nutley et al, 2013; Parkhurst and Abeysinghe, 2016), and there has been ongoing consultation to define community measures of wellbeing (Brown et al, 2015). The value of client-valued outcomes is an ongoing process of negotiation. In the next stage of developing community-generated evidence, VCS organisations will need to negotiate the incorporation of their evaluation and monitoring systems into funding specifications, and may need further training to enable this. Commissioners continue to be expected at national level to produce evidence using indicators that are increasingly agreed to have problems of validity and relevance. Sustaining momentum will be enabled and constrained by the wider debates about relevant measures for community programmes. Agreement on robust evidence for community services is therefore highly dependent on being able to broker across the boundaries between health, social care and voluntary sectors, both at local and national levels.

Funding and Acknowledgements

This work was supported by the National Institute of Health Research under Grant number KMRF-2014-03-003. We would like to acknowledge the ongoing work of the community organisations and commissioners who are creating community-generated evidence. Thank you to Christopher Stokes, who provided technical assistance with the graphics.

Research Ethics Statement

The authors of this paper have declared that research ethics approval was not required since the paper does not present or draw directly on data/findings from empirical research.

Contributor Statement

JH was funded by a National Institute for Health Research (NIHR)

Knowledge Mobilisation Fellowship for this research project. She conceptualised the study with JS, DM and GW. JH wrote the first and subsequent drafts of the manuscript, with comments from JS, AF, DM and GW.

Conflict of interest

The authors declare that there is no conflict of interest.

References

  • Andreeva, O. (2017) Improving the Outcome Measure Tool Used for Individual Client Work Within a Third Sector Organization, MSc Thesis, Sheffield: University of Sheffield.

    • Search Google Scholar
    • Export Citation
  • Baird, B. (2018) Innovative Models of General Practice, London: The Kings Fund, https://www.kingsfund.org.uk/projects/innovative-models-care-delivery-general-practice.

    • Search Google Scholar
    • Export Citation
  • Blackstock, K.L., Kelly, G.J. and Horsey, B.L. (2007) Developing and applying a framework to evaluate participatory research for sustainability, Ecological Economics, 60(4): 72642. doi: 10.1016/j.ecolecon.2006.05.014

    • Search Google Scholar
    • Export Citation
  • Brown, J.S. and Duguid, P. (1998) Organizing knowledge, California Management Review, 40(3): 90111. doi: 10.2307/41165945

  • Brown, H., Abdallah, R. and Townsley, R. (2015) Understanding local needs for wellbeing data, what works wellbeing, https://whatworkswellbeing.org/product/understanding-local-needs-for-wellbeing-data/.

    • Search Google Scholar
    • Export Citation
  • Comptroller and Auditor General (2018) Developing new care models through the NHS vanguards, HC1129 Session 2017–2019, 29 June 2018, https://www.nao.org.uk/wp-content/uploads/2018/06/Developing-new-care-models-through-NHS-Vanguards.pdf.

    • Search Google Scholar
    • Export Citation
  • Dart, J. and Davies, R. (2003) A dialogical, Story-based evaluation tool: the most significant change technique, American Journal of Evaluation, 24(2): 13755. doi: 10.1177/109821400302400202

    • Search Google Scholar
    • Export Citation
  • Edelstein, H. (2016) Collaborative research partnerships for knowledge mobilisation, Evidence & Policy, 12(2): 199216.

  • Foster, A. (2020) Facilitators and Barriers to Implementing Patient Reported Outcome Measures (PROMs) in Wellbeing Activities Delivered by Third Sector Organisations, PhD thesis, Sheffield: University of Sheffield.

    • Search Google Scholar
    • Export Citation
  • Foster, A., O’Cathain, A. and Harris, J. (2020) How do third sector organisations or charities providing health and wellbeing services in England implement patient reported outcome measures (PROMs)? A qualitative interview study, BMJ Open, 10(10): e03911, doi: doi: 10.1136.

    • Search Google Scholar
    • Export Citation
  • Gamsu, M., Southby, K., South, J., Newton, R., Di Martino, S. and Bagnall, AM. (2019) Debating what Works for Community Wellbeing: Findings from Two National Public Hearings on Community Wellbeing, Health Inequalities and Local Systems, London: What Works Centre for Wellbeing, https://whatworkswellbeing.org/product/public-debate-wellbeing/.

    • Search Google Scholar
    • Export Citation
  • Habermas, J. (2015) Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy, Chichester: John Wiley & Sons.

  • Hansen Kollock, D.A., Flage, L., Chazdon, S., Paine, N. and Higgins, L. (2012) Ripple effect mapping: a ‘radiant’ way to capture program impacts, Journal of Extension, 50(5): 5TOT6, http://www.joe.org/joe/2012october/tt6.php.

    • Search Google Scholar
    • Export Citation
  • Harris, J., Foster, A. and Weston, G. (2018) Collaborative measurement development, Sheffield: University of Sheffield. https://doi.org/10.15131/shef.data.5981722.v1.

    • Search Google Scholar
    • Export Citation
  • Henderson, J., Revell, P. and Escobar, O. (2018) Transforming communities? Exploring the roles of community anchor institutions in public service reform, local democracy, community resilience, and social change, What Works Scotland, http://whatworksscotland.ac.uk/wp-content/uploads/2018/05/WWSExploringTheRolesOfCommunityAnchorOrganisationsInPublicServiceReform.pdf.

    • Search Google Scholar
    • Export Citation
  • Husk, K., Blockley, K., Lovell, R., Bethel, A., Lang, I., Byng, R. and Garside, R. (2020) What approaches to social prescribing work, for whom, and in what circumstances? A realist review, Health & Social Care in the Community, 28(2): 30924.

    • Search Google Scholar
    • Export Citation
  • ICPHR (International Collaboration for Participatory Health Research) (2013) Position Paper 1: What is Participatory Health Research?, Berlin: International Collaboration for Participatory Health Research.

    • Search Google Scholar
    • Export Citation
  • Kemmis, S. and McTaggart, R. (2005) Participatory action research: communicative action and the public sphere, in N.K. Denzin and Y.S. Lincoln (eds) The Sage Handbook of Qualitative Research, New York: Sage, pp 271330.

    • Search Google Scholar
    • Export Citation
  • Lang, D.J., Wiek, A., Bergmann, M., Stauffacher, M., Martens, P., Moll, P., Swilling, M. and Thomas, C.J. (2012) Transdisciplinary research in sustainability science: practice, principles, and challenges, Sustainability Science, 7(1): 2543. doi: 10.1007/s11625-011-0149-x

    • Search Google Scholar
    • Export Citation
  • LGA (Local Government Association) Green Paper (2018) The lives we want to lead: The LGA green paper for adult social care and wellbeing, https://futureofadultsocialcare.co.uk/wp-content/uploads/2018/11/29.13-Green-paper-full_web.pdf.

    • Search Google Scholar
    • Export Citation
  • Lomas, J. (2007) The In-between world of knowledge brokering, BMJ, 334(7585): 12932. doi: 10.1136/bmj.39038.593380.AE

  • Lowndes, V. and Pratchett, L. (2012) Local governance under the coalition government: austerity, localism and the ‘Big Society’, Local Government Studies, 38(1): 2140. doi: 10.1080/03003930.2011.642949

    • Search Google Scholar
    • Export Citation
  • Lunn, I. (2018) Is the SOAR Wellbeing Questionnaire Usable and Acceptable Amongst Clients and Staff?, Sheffield: University of Sheffield.

    • Search Google Scholar
    • Export Citation
  • Murdock, A., Shariff, R. and Wilding, K. (2013) Knowledge exchange between academia and the third sector, Evidence & Policy, 9(3): 41930.

    • Search Google Scholar
    • Export Citation
  • NHS England (2019) A Five-year framework for GP contract reform to implement the NHS long term plan, https://www.england.nhs.uk/publication/gp-contract-five-year-framework/.

    • Search Google Scholar
    • Export Citation
  • Nutley, S., Powell, A. and Davies, H. (2013) What Counts as Good Evidence?, University of St Andrews, http://www.alliance4usefulevidence.org/assets/What-Counts-as-Good-Evidence-WEB.pdf.

    • Search Google Scholar
    • Export Citation
  • Paavola, S. and Hakkarainen, K. (2005) The knowledge creation metaphor: an emergent epistemological approach to learning, Science & Education, 14(6): 53557.

    • Search Google Scholar
    • Export Citation
  • Parkhurst, J.O. and Abeysinghe, S. (2016) What constitutes ‘good’ evidence for public health and social policymaking? From hierarchies to appropriateness, Social Epistemology, 30(5–6): 66579. doi: 10.1080/02691728.2016.1172365

    • Search Google Scholar
    • Export Citation
  • Petticrew, M. and Roberts, H. (2003) Evidence, hierarchies, and typologies: horses for courses, Journal of Epidemiology & Community Health, 57(7): 52729.

    • Search Google Scholar
    • Export Citation
  • Pohl, C. (2008) From science to policy through transdisciplinary research, Environmental Science and Policy, 11(1): 4653. doi: 10.1016/j.envsci.2007.06.001

    • Search Google Scholar
    • Export Citation
  • Ranga, M. and Etzkowitz, H. (2015) Triple helix systems: an analytical framework for innovation policy and practice in the knowledge society, in J. Mitra and J. Edmondson (eds) Entrepreneurship and Knowledge Exchange, Abingdon-on-Thames: Routledge, pp 11758.

    • Search Google Scholar
    • Export Citation
  • Roth, J. (2003) Enabling knowledge creation: learning from an R&D organisation, Journal of Knowledge Management, 7(1): 32. doi: 10.1108/13673270310463608

    • Search Google Scholar
    • Export Citation
  • Schuttenberg, H.Z. and Guth, H.K. (2015) Seeking our shared wisdom: a framework for understanding knowledge coproduction and coproductive capacities, Ecology and Society, 20(1): 15. doi: 10.5751/ES-07038-200115

    • Search Google Scholar
    • Export Citation
  • Walter, A.I., Helgenberger, S., Wiek, A. and Scholz, R.W. (2007) Measuring societal effects of transdisciplinary research projects: design and application of an evaluation method, Evaluation and Program Planning, 30(4): 32538. doi: 10.1016/j.evalprogplan.2007.08.002

    • Search Google Scholar
    • Export Citation
  • Ward, V. (2017) Why, whose, what and how? A framework for knowledge mobilisers, Evidence & Policy, 13(3): 47797.

  • Ward, V., House, A. and Hamer, S. (2009) Knowledge brokering: the missing link in the evidence to action chain? Evidence & Policy, 5(3): 26779.

    • Search Google Scholar
    • Export Citation
  • Wye, L., Cramer, H., Beckett, K., Farr, M., Le May, A., Carey, J., Robinson, R., Anthwal, R., Rooney, J. and Baxter, H. (2020) Collective knowledge brokering: the model and impact of an embedded team, Evidence & Policy, 16(3): 42952, https://doi.org/10.1332/174426419X15468577044957.

    • Search Google Scholar
    • Export Citation
  • Yin, R.K. (2009) Case Study Research: Design and Method, London: Sage.

  • Andreeva, O. (2017) Improving the Outcome Measure Tool Used for Individual Client Work Within a Third Sector Organization, MSc Thesis, Sheffield: University of Sheffield.

    • Search Google Scholar
    • Export Citation
  • Baird, B. (2018) Innovative Models of General Practice, London: The Kings Fund, https://www.kingsfund.org.uk/projects/innovative-models-care-delivery-general-practice.

    • Search Google Scholar
    • Export Citation
  • Blackstock, K.L., Kelly, G.J. and Horsey, B.L. (2007) Developing and applying a framework to evaluate participatory research for sustainability, Ecological Economics, 60(4): 72642. doi: 10.1016/j.ecolecon.2006.05.014

    • Search Google Scholar
    • Export Citation
  • Brown, J.S. and Duguid, P. (1998) Organizing knowledge, California Management Review, 40(3): 90111. doi: 10.2307/41165945

  • Brown, H., Abdallah, R. and Townsley, R. (2015) Understanding local needs for wellbeing data, what works wellbeing, https://whatworkswellbeing.org/product/understanding-local-needs-for-wellbeing-data/.

    • Search Google Scholar
    • Export Citation
  • Comptroller and Auditor General (2018) Developing new care models through the NHS vanguards, HC1129 Session 2017–2019, 29 June 2018, https://www.nao.org.uk/wp-content/uploads/2018/06/Developing-new-care-models-through-NHS-Vanguards.pdf.

    • Search Google Scholar
    • Export Citation
  • Dart, J. and Davies, R. (2003) A dialogical, Story-based evaluation tool: the most significant change technique, American Journal of Evaluation, 24(2): 13755. doi: 10.1177/109821400302400202

    • Search Google Scholar
    • Export Citation
  • Edelstein, H. (2016) Collaborative research partnerships for knowledge mobilisation, Evidence & Policy, 12(2): 199216.

  • Foster, A. (2020) Facilitators and Barriers to Implementing Patient Reported Outcome Measures (PROMs) in Wellbeing Activities Delivered by Third Sector Organisations, PhD thesis, Sheffield: University of Sheffield.

    • Search Google Scholar
    • Export Citation
  • Foster, A., O’Cathain, A. and Harris, J. (2020) How do third sector organisations or charities providing health and wellbeing services in England implement patient reported outcome measures (PROMs)? A qualitative interview study, BMJ Open, 10(10): e03911, doi: doi: 10.1136.

    • Search Google Scholar
    • Export Citation
  • Gamsu, M., Southby, K., South, J., Newton, R., Di Martino, S. and Bagnall, AM. (2019) Debating what Works for Community Wellbeing: Findings from Two National Public Hearings on Community Wellbeing, Health Inequalities and Local Systems, London: What Works Centre for Wellbeing, https://whatworkswellbeing.org/product/public-debate-wellbeing/.

    • Search Google Scholar
    • Export Citation
  • Habermas, J. (2015) Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy, Chichester: John Wiley & Sons.

  • Hansen Kollock, D.A., Flage, L., Chazdon, S., Paine, N. and Higgins, L. (2012) Ripple effect mapping: a ‘radiant’ way to capture program impacts, Journal of Extension, 50(5): 5TOT6, http://www.joe.org/joe/2012october/tt6.php.

    • Search Google Scholar
    • Export Citation
  • Harris, J., Foster, A. and Weston, G. (2018) Collaborative measurement development, Sheffield: University of Sheffield. https://doi.org/10.15131/shef.data.5981722.v1.

    • Search Google Scholar
    • Export Citation
  • Henderson, J., Revell, P. and Escobar, O. (2018) Transforming communities? Exploring the roles of community anchor institutions in public service reform, local democracy, community resilience, and social change, What Works Scotland, http://whatworksscotland.ac.uk/wp-content/uploads/2018/05/WWSExploringTheRolesOfCommunityAnchorOrganisationsInPublicServiceReform.pdf.

    • Search Google Scholar
    • Export Citation
  • Husk, K., Blockley, K., Lovell, R., Bethel, A., Lang, I., Byng, R. and Garside, R. (2020) What approaches to social prescribing work, for whom, and in what circumstances? A realist review, Health & Social Care in the Community, 28(2): 30924.

    • Search Google Scholar
    • Export Citation
  • ICPHR (International Collaboration for Participatory Health Research) (2013) Position Paper 1: What is Participatory Health Research?, Berlin: International Collaboration for Participatory Health Research.

    • Search Google Scholar
    • Export Citation
  • Kemmis, S. and McTaggart, R. (2005) Participatory action research: communicative action and the public sphere, in N.K. Denzin and Y.S. Lincoln (eds) The Sage Handbook of Qualitative Research, New York: Sage, pp 271330.

    • Search Google Scholar
    • Export Citation
  • Lang, D.J., Wiek, A., Bergmann, M., Stauffacher, M., Martens, P., Moll, P., Swilling, M. and Thomas, C.J. (2012) Transdisciplinary research in sustainability science: practice, principles, and challenges, Sustainability Science, 7(1): 2543. doi: 10.1007/s11625-011-0149-x

    • Search Google Scholar
    • Export Citation
  • LGA (Local Government Association) Green Paper (2018) The lives we want to lead: The LGA green paper for adult social care and wellbeing, https://futureofadultsocialcare.co.uk/wp-content/uploads/2018/11/29.13-Green-paper-full_web.pdf.

    • Search Google Scholar
    • Export Citation
  • Lomas, J. (2007) The In-between world of knowledge brokering, BMJ, 334(7585): 12932. doi: 10.1136/bmj.39038.593380.AE

  • Lowndes, V. and Pratchett, L. (2012) Local governance under the coalition government: austerity, localism and the ‘Big Society’, Local Government Studies, 38(1): 2140. doi: 10.1080/03003930.2011.642949

    • Search Google Scholar
    • Export Citation
  • Lunn, I. (2018) Is the SOAR Wellbeing Questionnaire Usable and Acceptable Amongst Clients and Staff?, Sheffield: University of Sheffield.

    • Search Google Scholar
    • Export Citation
  • Murdock, A., Shariff, R. and Wilding, K. (2013) Knowledge exchange between academia and the third sector, Evidence & Policy, 9(3): 41930.

    • Search Google Scholar
    • Export Citation
  • NHS England (2019) A Five-year framework for GP contract reform to implement the NHS long term plan, https://www.england.nhs.uk/publication/gp-contract-five-year-framework/.

    • Search Google Scholar
    • Export Citation
  • Nutley, S., Powell, A. and Davies, H. (2013) What Counts as Good Evidence?, University of St Andrews, http://www.alliance4usefulevidence.org/assets/What-Counts-as-Good-Evidence-WEB.pdf.

    • Search Google Scholar
    • Export Citation
  • Paavola, S. and Hakkarainen, K. (2005) The knowledge creation metaphor: an emergent epistemological approach to learning, Science & Education, 14(6): 53557.

    • Search Google Scholar
    • Export Citation
  • Parkhurst, J.O. and Abeysinghe, S. (2016) What constitutes ‘good’ evidence for public health and social policymaking? From hierarchies to appropriateness, Social Epistemology, 30(5–6): 66579. doi: 10.1080/02691728.2016.1172365

    • Search Google Scholar
    • Export Citation
  • Petticrew, M. and Roberts, H. (2003) Evidence, hierarchies, and typologies: horses for courses, Journal of Epidemiology & Community Health, 57(7): 52729.

    • Search Google Scholar
    • Export Citation
  • Pohl, C. (2008) From science to policy through transdisciplinary research, Environmental Science and Policy, 11(1): 4653. doi: 10.1016/j.envsci.2007.06.001

    • Search Google Scholar
    • Export Citation
  • Ranga, M. and Etzkowitz, H. (2015) Triple helix systems: an analytical framework for innovation policy and practice in the knowledge society, in J. Mitra and J. Edmondson (eds) Entrepreneurship and Knowledge Exchange, Abingdon-on-Thames: Routledge, pp 11758.

    • Search Google Scholar
    • Export Citation
  • Roth, J. (2003) Enabling knowledge creation: learning from an R&D organisation, Journal of Knowledge Management, 7(1): 32. doi: 10.1108/13673270310463608

    • Search Google Scholar
    • Export Citation
  • Schuttenberg, H.Z. and Guth, H.K. (2015) Seeking our shared wisdom: a framework for understanding knowledge coproduction and coproductive capacities, Ecology and Society, 20(1): 15. doi: 10.5751/ES-07038-200115

    • Search Google Scholar
    • Export Citation
  • Walter, A.I., Helgenberger, S., Wiek, A. and Scholz, R.W. (2007) Measuring societal effects of transdisciplinary research projects: design and application of an evaluation method, Evaluation and Program Planning, 30(4): 32538. doi: 10.1016/j.evalprogplan.2007.08.002

    • Search Google Scholar
    • Export Citation
  • Ward, V. (2017) Why, whose, what and how? A framework for knowledge mobilisers, Evidence & Policy, 13(3): 47797.

  • Ward, V., House, A. and Hamer, S. (2009) Knowledge brokering: the missing link in the evidence to action chain? Evidence & Policy, 5(3): 26779.

    • Search Google Scholar
    • Export Citation
  • Wye, L., Cramer, H., Beckett, K., Farr, M., Le May, A., Carey, J., Robinson, R., Anthwal, R., Rooney, J. and Baxter, H. (2020) Collective knowledge brokering: the model and impact of an embedded team, Evidence & Policy, 16(3): 42952, https://doi.org/10.1332/174426419X15468577044957.

    • Search Google Scholar
    • Export Citation
  • Yin, R.K. (2009) Case Study Research: Design and Method, London: Sage.

  • 1 University of Sheffield, , UK
  • | 2 University of Alberta, , Canada
  • | 3 Manor & Castle Development Trust, , UK
  • | 4 SOAR (Southey and Owlerton Area Regeneration), , UK
  • | 5 University of Sheffield, , UK

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 34 34 21
PDF Downloads 18 18 13

Altmetrics

Dimensions