Abstract
Background:
There has been a rapid increase in the number of, and demand for, organisations offering behavioural science advice to government over the last ten years. Yet we know little of the state of science and the experiences of these evidence providers.
Aims and objectives:
To identify current practice in this emerging field and the factors that impact on the production of high-quality and policy-relevant research.
Methods:
A qualitative study using one-to-one interviews with representatives from a purposeful sample of 15 units in the vanguard of international behavioural science research in policy. The data were analysed thematically.
Findings:
Relationships with policymakers were important in the inception of units, research conduct, implementation and dissemination of findings. Knowledge exchange facilitated a shared understanding of policy issues/context, and of behavioural science. Sufficient funding was crucial to maintain critical capacity in the units’ workforces, build a research portfolio beneficial to policymakers and the units, and to ensure full and transparent dissemination.
Discussion and conclusion:
Findings highlight the positive impact of strong evidence-provider/user relationships and the importance of governments’ commitment to co-produced research programmes to address policy problems and transparency in the dissemination of methods and findings. From the findings we have created a framework, ‘STEPS’ (Sharing, Transparency, Engagement, Partnership, Strong relationships), of five recommendations for units working with policymakers. These findings will be of value to all researchers conducting research on behalf of government.
Background
Changing behaviour is at the heart of much government policy (Kelly, 2016). This may be at an individual level, for example, Brazil’s Bolsa Família initiative where low-income families receive cash transfers if they ensure their children attend school and are vaccinated (Neves et al, 2022) or at organisational/population level, as in the UK’s sugar tax targeting the consumption of sugary carbonated soft drinks (Gov.uk, 2016). These are examples of the application of traditional economic incentives and taxes, parts of a range of policy instruments available to governments to change behaviours. Others include mandates and bans, campaigns/messages and nudges – sometimes used in combination.
Behavioural science and the evidence it produces, has the potential to make a major contribution to the development and evaluation of policies because of the focus on understanding and changing behaviour (Behavioural Insights Team, 2012; Congdon and Shankar, 2015; OECD, 2017). Recognition of this potential has led to a demand for, and commitment to, behavioural science across the globe (McManus et al, 2008; Lourenço et al, 2016; Afif et al, 2019; Manning et al, 2020; UN Innovation Network, 2021; Baggio et al, 2021) and supply of behavioural science evidence (UK Science and Technology Select Committee, 2011; Afif et al, 2019). The last ten years have witnessed a rapid growth in the demand and provision of behavioural science advice to government to address societal problems, ranging from encouraging citizens to pay their taxes on time, to increasing organ donor registration (OECD, 2017; Afif et al, 2019). In 2018, the Organisation for Economic Co-operation and Development (OECD) map listed 202 institutions/units applying behavioural science to public policy across the world, and the number continues to grow across all sectors (OECD, nd). However, the existence and establishment of such units has not solved the problem of getting behavioural and social science evidence into practice and there remains a translational gap (Sniehotta and Chater, 2018). This paper explores empirically some of the issues that arise even when there are institutional arrangements in place to facilitate evidence exchange.
The World Health Organisation (WHO) devoted their 2021 bulletin to behavioural science and its importance to inform policy and achieve better population health and well-being globally (Altieri et al, 2021). However, they also drew attention to the underutilisation of behavioural science in health policy and gave reasons why this might be the case. These include: intervention details and data not being made available which prevents standardisation across contexts, replication, and secondary analyses (Carrasco et al, 2021); policymakers’ reservations about adopting a behavioural science approach because of uncertainties surrounding the impact, sustainability, and replicability of interventions (Ghebreyesus, 2021); the extent of data collection; and evaluation (Altieri et al, 2021).
With the increase in institutions/units globally offering behavioural science advice to government, this raises questions as to whether they have encountered these problems and, if so, how they were addressed. Although the World Bank (Afif et al, 2019) and European Commission (Lourenço et al, 2016) have published profiles and overviews of institutions and units in 10 and 30 countries respectively, we know little about their experiences. Others have reflected on the success factors of individual Behavioural Science Units (BSUs) (Behavioural Insights Team, 2015; Ball et al, 2017; John, 2017) and described the behavioural science landscape in national (Feitsma, 2019) and local government (Moffat et al, 2022). While behavioural science in government is an emerging field there is little published literature comparing the experiences of units across the globe, and how they have gained traction with policymakers. The application of behavioural science to health policy is arguably more complex and learning about these BSUs could assist future coworking and scientific development activities. Also, the findings will be of relevance to the WHO’s emphasis on the importance of behavioural science for better health when addressing global health issues (WHO, 2022). This study aimed to identify and understand the critical factors in producing high-quality and policy-relevant research evidence and make recommendations to inform others working in this field. These findings may be valuable to researchers from all disciplines, not just behavioural science, conducting research on behalf of, or for government.
The first section of this paper presents BSU factors key to producing high-quality policy-relevant research, and the second section describes the processes for policymaker and evidence-provider partnerships. Finally, we discuss the critical factors to produce impactful research and present our STEPS framework (Sharing, Transparency, Engagement, Partnership, Strong relationships) of five recommendations to achieve these factors (Figure 1).
STEPS to translational behavioural science, policy and practice between evidence users and Behavioural Science Units
Citation: Evidence & Policy 20, 2; 10.1332/17442648Y2023D000000001
Methods
Study design and data collection
This was a qualitative study employing one-to-one semi-structured interviews, all conducted by JL through a virtual platform (Zoom) and recorded with permission. An interview topic guide was developed by our team (see https://osf.io/rc6d2) including our Patient and Public Involvement and Engagement Strategy Group.
As the remit of our Policy Research Unit in Behavioural Science is in health and social care, units providing behavioural science evidence in this policy area internationally were of interest. A list of units was generated from reports (Lourenço et al, 2016; Afif et al, 2019), a database collated by another organisation, personal contacts, internet searches and snowball sampling. This was a purposive sample and recruitment was driven by the decision to include BSUs from a range of settings rather than global representativeness.
From the BSU list 28 were contacted between April 2020 and March 2021. Three responded to say their unit did not meet the project inclusion criteria and one was unable to participate due to work pressures. Between May 2020 and March 2021 JL conducted interviews with one team member from each of 15 BSUs. Most interviewees were directors or senior members of the team, and a number are prominent in the field of behavioural science. The units had been actively operating for between two and 12 years and between them had 103 years of experience of providing evidence and behavioural science advice to policymakers. Six were government (G) units, three university (A), two research institutes (RI), three commercial (C), and one a charity (Ch). Units were located in North America, the UK, Australia, Europe, Scandinavia, the Middle East, and Africa. Only one unit focused exclusively on health and social care policy and the rest had a broader policy remit, for example, the environment and education. Eleven received core funding from their government, but the levels varied. There were different models of income generation between the units.
Analysis
Interview sound-files were transcribed verbatim and uploaded to NVivo 14. Data were analysed thematically (Braun and Clarke, 2006) by generating initial codes inductively and the development of a coding frame (Table 1). The final themes (see https://osf.io/pshgx) were discussed and refined further by JL, MPK and FFS.
Steps in data analysis
| Step 1: Familiarisation with data – read, re-read and listen to interview/focus groups |
| Step 2: Generate initial codes – systematically record interesting features across the data |
| Step 3: Identify themes – sort coded extracts into overarching themes. Develop subthemes where appropriate |
| Step 4: Review of themes – combine, refine, redefine or separate themes. Devise framework |
| Step 5: Defining and naming themes – refine themes and sub-themes. Add working definitions of each theme |
Findings
This section is structured to reflect the five key overarching themes and subthemes as outlined in Figure 2. The term ‘policymaker’ is used to encompass civil/public servants, politicians, and government officials who work with the BSUs.
Themes and subthemes elicited from data analysis
Citation: Evidence & Policy 20, 2; 10.1332/17442648Y2023D000000001
BSU factors key to producing high-quality policy-relevant research
Theme 1: Creating critical capacity
To identify, assess or produce the required evidence to inform policy questions there is a need to create capacity either within or outside of the team. BSU teams ranged from two to 30 members and varied in disciplinary, methodological and professional expertise, and structure. Government BSUs described the majority in their teams as behavioural scientists, with a few also having members with a policy background. In non-government BSUs most members were described as psychologists and all but one had behavioural economists. Non-government BSUs were more likely to have team members from a wider range of disciplines than their government counterparts, for example, sociology, political science, health systems, education, epidemiology, anthropology and service design.
Level of funding affected how government BSUs managed their capacity to deliver projects. In units with a small core team of four or less, capacity was created through the employment of external consultants and academic researchers or funding a fellowship scheme. Another way of addressing shortfalls in capacity was to develop and build this in government employees and partner with other organisations for technical assistance.
‘We do have an academic advisory board, it’s not terribly well used…. The more useful thing is we’ll (have) specific academics on specific research topics…. Sometimes we compensate them for their time and sometimes they’re just happy to be involved if they get to write it up and that works for us as well’. (C08)
Theme 2: Generating and managing a project portfolio
BSU criteria for accepting projects
‘I didn’t want them all to just be about saving money. I wanted to demonstrate it could be used for saving time, money and lives…. I didn’t want to just tweak some emails and then just have six simple email trials. I wanted to demonstrate it can be used in low, medium, high complexity areas’. (G13)
The third consideration is whether the unit has the capacity to engage in a particular project. Government BSUs are not obliged to take on every project, but one interviewee said, “we tread carefully” (G11), and to maintain good relations they signpost other routes for support. Intake management tools, primarily forms that policymakers complete, are used, that can be formally reviewed or discussed directly with the policymaker. This provides policymakers with a clear explanation of why the unit cannot take on the project/idea.
‘We’re not going to try to make that programme work better because the problem is the programme itself, not just is there a possibility to make some changes that could increase things marginally but also is this a thing that we should even prop up in the first place?’ (Ch01)
Funding project work
Projects were funded in several ways: through government on an individual project basis – commissioned calls or negotiations between the BSU and policymakers – or core funding; grant-funding structures; or with revenue from other BSU activities. In governmental BSUs, project costs tended to be subsumed within the units’ core funding. In the experience of one non-government unit, conducting the short-term low-budget government projects was “stressful”, and there was no guarantee they would win the grant (RI07). Functioning on consecutive single projects in the longer term could make it difficult to maintain core staff, and for those in a university setting to manage overheads. A longer-established unit obtained funding for research programmes which was a more secure option: “We agree a research programme with that government department to tackle a particular policy problem… [for] perhaps two- or three-years’ worth of research” (RI09).
Views were mixed on government-commissioned calls. One BSU found pitching project ideas to policymakers “takes a lot of time, and a fraction of these efforts are successful” (RI07), though they had themselves secured funding through a government-commissioned call and incorporated behavioural science. Another asserted that commissioned calls produce the worst kind of projects as applicants “promise everything”. They went on to describe their approach when they were interested in a particular topic: “If there is a project that we really want to do and we spot it before it’s announced then we call them up and say we’ll do it for (a nominal amount)… and then we do the project because we want to do the project” (A&C14).
Several non-governmental BSUs conducted projects for industry. BSUs considered the consequences of their involvement in specific industry projects and the unit’s reputation: “I can’t exactly be running a project with McDonalds and doing a policy for obesity strategy with the (government)” (C08). Although around 20% of their income was from industry, one BSU stated “We won’t just bang our name on anything to generate some income” (A10).
Fostering a shared understanding of behavioural science
‘So you need to make sure you hit it from several angles, which means the span of attention is often reached. If someone senior has given you two minutes or so and you’re already still trying to find an entry point, you might have missed it, it’s a longer narrative’. (G02)
Another BSU had developed a diagram of the approach they take to address a policy problem for use in discussions with policymakers, outlining all of the possible stages. They commented it is “hard to make these things representative of what you’re doing but simple enough to explain to people who aren’t from research backgrounds” (A10).
‘We’ve done our best to infiltrate the public sector here by making available a variety of capacity-building resources that are going to help people come to the table and those initial conversations with better questions and better ideas.’ (Ch01)
‘We would come in, partner with a department, help them carry out an initiative in a behaviourally-informed way but once we left that work didn’t continue. The thought was by embedding experts in departments, perhaps that would help build the capacity and integrate behavioural science more systematically’. (G11)
‘It backfired because after… seven courses of training they thought they could have their own unit and they weren’t aware of the fact that they needed… real expertise of someone who knows behavioural science, someone who knows how to run experiments etc. etc.’. (G03)
Theme 3: Transparency – permission and capacity
Publishing full details of research conduct and findings is imperative for transparency and evidence accumulation. It ensures quality through peer review and adds to the body of knowledge in the specific content area. Several BSUs publicised projects and findings without peer review on their unit’s website. All but four BSUs claimed they try to publish findings in peer-reviewed journals where possible. Units differed in how much value they placed on academic publications and other dissemination routes were more important. For academic units, peer-reviewed publications were an important performance metric, and for others they provided a level of validation which instilled a sense of trust and credibility in potential policymakers: “We do it not because it helps us gain or keep our academic standing or tenure but because it is unlocking doors to make sure that we’re increasing impact” (Ch01). One reflected that peer-reviewed publications are of less importance to a government unit, “I feel okay about that because we’re government, we don’t in any way have publication as currency within our work” (G05). In one research institute and one commercial BSU their project budgets did not cover the time for team members to produce peer-reviewed publications and this was left to academic partners. The commercial BSU declared there was no incentive for their team to produce and publish papers, due to the lengthy process, and blogs were their main dissemination route. In contrast another BSU was producing timely outputs through pre-registrations and pre-prints, ensuring transparency of methods and findings.
Permission for full public dissemination varied. Two of the six government BSUs said there were no restrictions on dissemination. Although several of the units had complete freedom to disseminate, most constructed a formal agreement with the policymaker before the project commenced. For one BSU freedom to disseminate was a non-negotiable condition, and this had resulted in some policymakers being reluctant to work with them.
There were concerns that BSUs advocating full dissemination could be a sticking point in early negotiations with policymakers, and disagreements over publication could impact negatively on relationships. Several units described the process of securing agreement to share research results as one of continued negotiation. Initially there may have been an ethos of full dissemination but some policymakers had “different levels of comfort in sharing with different audiences” (G13). Another said they try to publish academically but the process of agreeing the final content can be difficult and “comes down to the nuances of what they’re comfortable with us saying or not saying” (G11). One commercial BSU had no concerns about restrictions on dissemination and stressed that ensuring the findings reach the policymakers is more important.
‘We did (research) for a government agency several years ago, which is still under embargo. I’ve got some versions of it we made public but that report, just repeated requests we aren’t able to share it. So we do keep a lot under wraps in that traditional model of providing reports to government’. (A10)
Processes for policymaker and evidence-provider partnerships
Theme 4: Relationships with policymakers
Establishing and maintaining relationships
Relationships with policymakers are key for BSUs to help understand governmental policy priorities, to facilitate the application of behavioural science to policy problems and the uptake of evidence. For one BSU, engagement with government was attributable to the unit being one of the earliest established and “ahead of the curve” in providing advice. It had taken time to establish strong links with policymakers and the interviewee added, “We’ve got enough of history now that we’re quite close to a lot of the agencies” (A10). Another BSU had a team of consultants who acted as a liaison between the unit and government/industry. However, they had experienced difficulties in convincing ministries to agree to projects. They were planning to provide training to government staff to build bridges between the unit and the government and establish partnership working on projects: “We plan to do training of government officials. It is not training in the sense that we want them to be competent, to do the project on their own but more in the sense to establish some connection… do the projects together with us” (RI07).
Maintaining, and developing new, relationships with policymakers ensured continued engagement and opportunities to conduct policy-related research. This could be hampered by frequent staff turnover and the ability to reach beyond a few select policymakers to promote and raise awareness of the BSU. Even at a policy-team level, buy-in from senior policymakers was not always sufficient. If others within a policy team, delegated to work on the project, were not supportive it would fail. Experience of encouraging policymakers to engage with the unit varied. One had not had to scout for work or go out and sell their services to potential clients. In contrast, another government BSU had spent a few years promoting the unit through presentations until they found they were ineffective in generating requests for projects: “In fact, they were coming from those gems of people that sporadically work in government that have a knack for behavioural science and an interest in psychology, something that makes them a little bit more motivated than the next person” (G05).
Reputation, built through the conduct of projects, was also important. Continuation of work with the same policymakers meant they brought more relevant ideas to the BSU.
Choosing policy partners
Either through choice or necessity, some BSUs worked with semi-public organisations (described as a public service maintained by a private organisation) or the municipalities (described as lower-level government entities) in their country, rather than with central government. Reasons given were these were “less bureaucratic” than ministries (RI07), a lack of trust in ministerial policymakers, and difficulties engaging at that higher level particularly in relation to health policy: “I know that the government is interested… environmental policy has been very big and strong… the health sector has a lot more to develop, they are not really there yet to understand that (behavioural science) could be implemented and applied in so many different societal issues” (C12).
Others had made a conscious decision to work with particular policymakers where they believed the relationship would be more conducive to the research. One was attempting to work on larger-scale projects with policymakers who have an interest in the findings, irrespective of what they show, and have a commitment to make changes. This was echoed by another who talked about the difference it can make when projects, partners and funders are aligned: “I think part of the importance of selecting projects and partners is also selecting the right funders because if you’re working with the right folks then they can hear whatever it is you’ve got to say” (Ch01).
‘They have waited for those designers to… change to another job and then they have implemented it. So given this experience in seven years I have just found out that these systems are made up of people and they basically go to work because they have other interests than serving the citizen… and then sometimes you meet that one person in your network a wise [policymaker] did a lot of great work with us and got it implemented then he moved to [another ministry] and suddenly our activities moved with him there’. (A&C14)
Understanding policy priorities and generating research questions
Research questions/projects came directly from policymakers and/or the BSU pitching project ideas to government departments. Some government BSUs learned of policy priorities through internal processes, such as ministerial mandate letters, and one commented that the “announcement of the budget sometimes naturally creates synergies where it becomes clear we’re going to be looped in on certain initiatives” (G11). To gain insight into government priorities and increase the potential to generate projects, one university-based BSU ran an embedded researcher scheme. Others in government described a combination of top-down and bottom-up approaches. Top-down is where policymakers request the BSU involvement to address a particular policy issue. The bottom-up approach was where the BSU explored the policy agenda/priorities for “potential areas for experimentation” (G02) or “go through a particular set of interventions to see what potentially we could use to look at through a behavioural lens” (G04).
‘The process took us two months to pick the right topics and now after three years, I’m not sure I would pick the same topics so it’s not an easy thing to do… we based a lot on low hanging fruits, but we saw the limit of that approach which is that it’s super basic to change something in a letter and want to do things that are much more like social or complex’. (G03)
Decisions on which projects to propose for policymakers’ approval were based on what would be most impactful and “the [policymaker] would be more inclined to buy into” (G04). Outside of government one BSU was trying to think more “large-scale about missions rather than individual projects” (A10). This was echoed by another who had government agreement for a “research and development programme of work” to tackle a particular policy problem (RI09). Relationships again emerged as important; those with long-established links to policymakers could be “a bit more proactive and say this is a real problem or we’ve found this great result in area X, we think you could apply a pretty similar thing into area Y” (C08).
Theme 5: Knowledge exchange
The journey from the inception of BSUs to the reporting of research findings to government involves a two-way process of knowledge exchange where evidence providers share with policymakers, and policymakers share information about government mechanisms and priorities. Interactions with policymakers that focused on project development were critical for helping BSUs develop their understanding about the workings of government: “It’s not until you’ve… worked with the [policymakers] for a little while, you get a sense of oh okay that’s how things work around here” (G13).
Interactions with government stakeholders and beyond were sometimes limited and BSUs were not always privy to important discussions: “Different stakeholders… all have different questions… how do you sync those conversations when you’re not necessarily in all of them? That’s a piece of the challenge” (Ch01). Pressures on policymakers to find a quick solution meant the process of engagement was not always as extensive as BSUs would prefer. With less urgent projects one BSU’s criterion for accepting projects was that policymakers are “engaged in the process until the end of the process… [they] stay until the evaluation” (G03). Interactions were not always one sided, and sustained dialogue over time was important for agreeing the parameters of a project and for BSUs to negotiate what is of interest to their own unit: “There is some kind of shaping the project together, or if you want to do this then we can do it but then we also want to do this, so it’s very much on a personal relation basis” (A&C14).
Need for a shared understanding
‘They don’t have the ability to make the recommendation…. We want people to go for cervical cancer screening…. The only thing that [policymaker] is able to do is change the invitation letter, so it’s not to say we think [that] is a number one intervention that would impact this behaviour but it’s the thing that the people who are paying us have control over, so that’s the recommendation that is useful to them and is sensible to them’. (C08)
Although some units were satisfied to recommend simple-to-implement interventions, others found this disappointing, particularly when there was the opportunity to do something with potentially greater impact. Some BSUs were reluctant to push back when policymakers disagreed with their proposed design or intervention, which led to compromises.
‘It’s a risky business because we can also at the end see that our design and our ideas are not giving what they want. Normally people leave, you give the ideas and just leave’. (C12)
‘People expect you to have a result that’s conclusive and you’re able to… make a decision immediately. With behavioural science… it’s about testing, it’s about piloting, it’s about understanding what works and what doesn’t work and sometimes politicians get a bit annoyed because you’re not able to give them that final answer, and that’s been a struggle for us’. (G04)
When faced with a policymaker who disputes the findings one interviewee had “analysed the data ten times in different ways” and found the same result which they had to accept (G05). Another stated they “wait until the personnel change before re-approaching the same organisation” (RI09).
Discussion
There are three critical factors (Figure 3) to facilitate producing research that impacts on policy, based on the five key themes identified in this study. As demonstrated in Figure 3 and the following, many of these are interrelated and interdependent. The majority are applicable to any team conducting research on behalf of the government.
Key factors and processes for producing high-quality and policy-relevant research
Citation: Evidence & Policy 20, 2; 10.1332/17442648Y2023D000000001
Understanding the policy arena and priorities
Essentially the function of units is to provide, and to some degree specify and define, the evidence required by policymakers, from primary and/or secondary sources. To propose research that is beneficial, units require an understanding both of the policy arena – what is feasible in terms of the research and potential interventions (Fox and Sitkin, 2015) – and government’s policy priorities. Knowledge exchange – a two-way process where evidence providers and users share skills and experience, and research evidence – and ensuring a shared understanding of behavioural science and the policy arena was central. Although knowledge should be exchanged at each unit/policymaker encounter, some units had a liaison person bridging the unit and government, or researchers embedded in government departments. Knowledge of policy priorities led to a shared understanding between units and policymakers on what was feasible in terms of research design and proposed intervention. Without this units could waste time proposing projects that are of little interest to policymakers.
Adequate funding
Continuity of funding was critical to the team, building a translational research portfolio and transparency. To produce robust research, a skills mix is required that demands a certain team size and/or the ability to flexibly extend through collaboration or subcontracting the expertise of others. There is a need for investment in behavioural science (McBride et al, 2021) and BSUs require adequate funding to retain a skilled core workforce and pay for other ad hoc expertise. Secure funding will enable BSUs to be more agile to work at speed, such as during crises, and to build a strategic portfolio. Various models of evidence and policy engagement have emerged, but those which rely on the voluntary unfunded time of behavioural science experts can only ever be temporary and only sustainable if adequately funded. One example is the Template for Rapid Iterative Consensus of Experts (TRICE) that enables the use of behavioural science at speed (Chater et al, 2021).
It was difficult for units to function on a project-by-project basis without a stable source of revenue or financial support, and in the research reported here, this limited the opportunity to generate a research portfolio. Single projects carry the risk of being written off if policies or governments change, and result in losses in costs and outputs for the unit. It is important for all stakeholders to understand the financial situations of all parties and the metrics of assessment, such as publications and impact in academia, and respect and voter loyalty for government. Outside of government, programmes of work agreed in partnership with policymakers brought greater security. Regarding publications, adequate funding was again pertinent to cover either the time of BSU core researchers or academic partners to produce papers. The importance of reporting research methods in peer-reviewed journals, as much for policymakers as for the scientific community, has been well-documented (Whitty, 2015). Transparency in reporting is also key to promote public trust in governments (Ghio et al, 2021). Full and effective dissemination also serves to promote the BSU and potentially generate more project work. Despite this a relatively small number of interviewees expressed a strong commitment to, and belief in, academic publications.
Strong relationships
Relationships were important for the units involved in this study more broadly, from initial support for a BSU, agreeing the research design, negotiating dissemination of research findings, to implementation. Those who had developed successful relationships had achieved this through word of mouth from policy partners championing their work. However, there were also issues of reach, questioning how BSUs can promote themselves and develop new relationships considering the large pool, and fluidity, of policymakers. Relationships took time to develop, and experience had led some BSUs to choose particular policy partners. Although power relations were not explicitly discussed, most units were dependent on their government’s financial support and buy-in for behavioural science, and the establishment and continuation of strong relationships must be considered in this context. Research on interventions to facilitate the use of research in policymaking has grown over the years, though reviews report methodologically weak studies and little evidence of effectiveness (Campbell and Moore, 2018), and variable evidence (Langer et al, 2016; Haynes et al, 2018). Good relationships with policymakers have been shown to facilitate the uptake of research evidence (Oliver et al, 2014; Byrne-Davis et al, 2022).
There were tensions between policymakers and BSUs in the acceptability of the research methods and the need for rigour and transparency. These challenges could have been attributable to policymakers’ skills in behavioural science and/or to BSUs not always appreciating what was feasible for government in research conduct and implementation of findings. Working in partnership with policymakers and co-producing projects ensured they were appropriate for a behavioural science approach and acceptable to government. Others have also demonstrated that genuine collaboration, namely ‘when both parties have the power and ability to shape critical decisions and have input in the processes’ (Haynes et al, 2018: 14), impacts positively on the ownership of, and investment in, research. Longer-term collaborative working may also improve uptake of research evidence by policymakers (Haynes et al, 2018; Oliver et al, 2022) and enable researchers and policymakers to co-produce knowledge of value to both (Matthews et al, 2018; Western, 2019).
The current study found that without strong partnerships policymakers could stall projects when, for example, they were uncomfortable with the design. Interviewees were clearly passionate about behavioural science, and from their accounts some in government units felt stifled when they were not able to conduct the projects they believed were needed.
Policymakers were not always satisfied with the findings, and there is a need for a culture change where policymakers accept that an intervention may not always work, as a great deal can be learnt from failures. Researchers adhere to the principle of equipoise – where they have genuine uncertainty as to the study findings – which may not be shared by policymakers who have other considerations such as the political direction of elected leaders.
Freedom to share findings publicly was a matter of negotiation between the unit and policymaker. This suggests policymakers did not always appreciate the importance for some BSUs to fully disseminate the findings, or the culture within certain government departments was not one of sharing.
Behavioural policy research is complex, and the research skills of policymakers have been found to be an enabler to evidence uptake (Oliver et al, 2014; van de Goor et al, 2017; Marquez et al, 2018). The current study demonstrated that policymakers’ understanding of behavioural science impacted on all stages of the research process. Communicating effectively with policymakers, and also providing basic training in behavioural science, were key to building strong relationships and developing a shared understanding.
From this empirical research we have created a framework of five key recommendations with the mnemonic ‘STEPS’ (Sharing, Transparency, Engagement, Partnership, Strong relationships), for units working with policymakers to overcome these issues (Figure 1): Sharing knowledge to understand the policy and research landscapes; Transparency through full dissemination of research methods and evidence; Engagement of policymakers throughout a research project; Partnerships between government and researchers to generate policy-relevant research; and Strong relationships to unite evidence providers and users and pursue credible and policy-relevant research.
Critical reflection
Behavioural science applied to health and social care policy is in its infancy. Some BSUs reported precarious funding and insecure status. Collaborating with policymakers is described as a series of compromises on topics, timelines, methods and, critically, the ability to have work peer reviewed and disseminated through academic publications and open science practices. Thus, the application of behavioural science is highly selective, and its reporting even more so. This will inevitably limit the rate at which research produced by BSUs will be accumulated into a reliable evidence base. It is in the interest of the public that policymakers have access to high-quality evidence related to health and social care policy, and to achieve this, the application of rigorous and transparent behavioural science methods needs to be normalised in the policymaking context.
Limitations
One limitation of this paper is that we do not know with any certainty how many established BSUs with a health focus there are internationally. There may be members of units whose experience differs from those included in our study. However, based on the published country profiles (Lourenço et al, 2016; Afif et al, 2019), we believe we have captured most of the successful units currently in operation whose remit encompasses health policy.
Conclusions
The key messages from this work are the benefits to BSUs and policymakers of developing a shared understanding and strong working relationships, for units to receive adequate funding to produce robust high-quality research and be able to openly share their methods and findings. Governments who have an interest in the application of behavioural science to policy should work with BSUs to establish robust programmes of work to address policy issues, and fund the units accordingly so they can recruit and retain a strong core team. Crucial to the field of behavioural science is peer-reviewed publications. Policy based on research that is not open to scrutiny will always be questioned and selective publication will lead to suspicions of a biased evidence base. Greater transparency would ensure that policy is based on robust evidence and may increase public trust in government. Although academic peer-reviewed publications may not represent currency for certain BSUs it remains a measure of quality, and there is a need to change the culture to one where its importance for building and maintaining population trust, alongside value to the scientific and policy community, is recognised.
Future research should explore opportunities to develop a knowledge-exchange network with the potential to exchange methods/develop novel ones together, learn from each other’s approaches to address health/social care issues, identify synergies, reduce duplication in projects addressing worldwide problems, share research findings and identify experts.
Disclaimer
This paper is independent research funded by the National Institute for Health and Social Care Research Policy Research Unit in Behavioural Science (project reference PR-PRU-
1217–20501). The views expressed in this publication are those of the authors and not necessarily those of the National Health Service, the National Institute for Health and Social Care Research, the Department of Health and Social Care or its arm’s-length bodies, or other Government Departments.
Funding
This work was supported by the National Institute for Health and Social Care Research Policy Research Unit in Behavioural Science (Project reference PR-PRU1217-20501). The views expressed in this publication are those of the authors and not necessarily those of the National Health Service, the National Institute for Health and Social Care Research, the Department of Health and Social Care or its arm’s-length bodies, or other Government Departments.
Acknowledgements
We would like to acknowledge the contribution of the Behavioural Science Units who participated in this study.
Research ethics statement
Ethical approval was granted from Newcastle University Ethics Committee (Ref: 1964/2020).
Contributor statement
JL and FFS conceptualised and designed the study. JL conducted the data analysis and interpretation with contributions from FFS, MPK, PC and AMC. JL conceived the paper and wrote the first and subsequent drafts of the manuscript; all authors contributed to further drafts and editing. All authors read and approved the final manuscript.
Conflict of interest
The authors declare that there is no conflict of interest.
References
Afif, Z., Islan, W.W., Calvo-Gonzalez, O. and Dalton, A. (2019) Behavioral science around the world: Profiles of 10 countries, World Bank. https://documents1.worldbank.org/curated/en/710771543609067500/pdf/132610-REVISED-00-COUNTRY-PROFILES-dig.pdf.
Altieri, E., Grove, J., Davies, O.L., Habersaat, K.B., Okeibunor, J., Samhouri, D. and Bezbaruah, S. (2021) Harnessing the power of behavioural science to improve health, Bulletin of the World Health Organization, 99(11): 754. doi:
Baggio, M., Ciriolo, E., Marandola, G. and van Bavel, R. (2021) The evolution of behaviourally informed policymaking in the EU, Journal of European Public Policy, 28(5): 658–76. doi:
Ball, S., Hiscox, M. and Oliver, T. (2017) Starting a behavioural insights team: three lessons from the Behavioural Economics Team of the Australian Government, Journal of Behavioral Economics for Policy, 1(Special Issue): 21–26, https://sabeconomics.org/journal/RePEc/beh/JBEPv1/articles/JBEP-1-S-5.pdf.
Behavioural Insights Team (2012) Behavioural Insights Team Annual update 2011–2012. Government of the United Kingdom, Cabinet Office. Retrieved 29 March 2022 from, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/83719/Behavioural-Insights-Team-Annual-Update-2011-12_0.pdf.
Behavioural Insights Team (2015) The global spread of behavioural insights: conditions for success of a central unit, https://www.bi.team/blogs/the-global-spread-of-behavioural-insights-conditions-for-success.
Braun, V. and Clarke, V. (2006) Using thematic analysis in psychology, Qualitative Research in Psychology, 3(2): 77–101. doi:
Byrne-Davis, L.M., Turner, R.R., Amatya, S., Ashton, C., Bull, E.R., Chater, A.M., Lewis, L.J.M., Shorter, G.W., Whittaker, E. and Hart, J.K. (2022) Using behavioural science in public health settings during the COVID-19 pandemic: the experience of public health practitioners and behavioural scientists, Acta Psychologica, 224: 103527. doi:
Campbell, D.M. and Moore, G. (2018) Increasing the use of research in population health policies and programs: a rapid review, Public Health Research & Practice, 28(3). doi:
Carrasco, M.A., Mickler, A.K., Young, R., Atkins, K., Rosen, J.G. and Obregon, R. (2021) Behavioural and social science research opportunities, Bulletin of the World Health Organization, 99(11): 834. doi:
Chater, A.M. et al. (2021) Template for rapid iterative consensus of experts (TRICE), International Journal of Environmental Research and Public Health, 18(19): 10255, doi:
Congdon, W.J. and Shankar, M. (2015) The White House social and behavioral sciences team: lessons learned from year one, Behavioral Science & Policy, 1(2): 77–86.
Feitsma, J. (2019) Brokering behaviour change: the work of behavioural insights experts in government, Policy & Politics, 47(1): 37–56. doi:
Fox, C.R. and Sitkin, S.B. (2015) Bridging the divide between behavioral science & policy, Behavioral Science & Policy, 1(1): 1–12, https://behavioralpolicy.org/wp-content/uploads/2017/05/BSP_vol1is1_Fox.pdf.
Ghebreyesus, T.A. (2021) Using behavioural science for better health, Bulletin of the World Health Organization, 99(11): 755. doi:
Ghio, D. et al. (2021) What influences people’s responses to public health messages for managing risks and preventing infectious diseases? A rapid systematic review of the evidence and recommendations, BMJ Open, 11(11): e048750. doi:
Gov.uk (2016) Policy paper, soft drinks industry levy, 5 December. https://www.gov.uk/government/publications/soft-drinks-industry-levy/soft-drinks-industry-levy.
Haynes, A., Rowbotham, S.J., Redman, S., Brennan, S., Williamson, A. and Moore, G. (2018) What can we learn from interventions that aim to increase policymakers’ capacity to use research? A realist scoping review, Health Research Policy and Systems, 16: 31. doi:
John, P. (2017) Behavioural science, randomized evaluations and the transformation of public policy: the case of the UK government, in J. Pykett, R. Jones and M. Whitehead (eds) Psychological Governance and Public Policy: Governing the Mind, Brain and Behavior, Abingdon: Routledge.
Kelly, M.P. (2016) The politics of behaviour change, in F. Spotswood (ed) Beyond Behaviour Change: Key Issues, Interdisciplinary Approaches and Future Directions, Bristol: Policy Press, pp 11–26.
Langer, L., Tripney, J. and Gough, D. (2016) The Science of Using Science: Researching the Use of Research Evidence in Decision-Making, London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London.
Lourenço, J.S., Ciriolo, E., Almeida, S.R. and Dessart, F.J. (2016) Behavioural insights applied to policy: country overviews, European Commission, https://publications.jrc.ec.europa.eu/repository/handle/JRC100547.
Manning, L., Dalton, A.G., Afif, Z., Vakos, R. and Naru, F. (2020) Behavioral Science Around the World, Vol II: Profiles of 17 International Organizations (English), Washington, DC: World Bank Group.
Marquez, C. et al. (2018) Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policymakers? A mixed-methods study, Implementation Science, 13: 84. doi:
Matthews, P., Rutherfoord, R., Connelly, S., Richardson, L., Durose, C. and Vanderhoven, D. (2018) Everyday stories of impact: interpreting knowledge exchange in the contemporary university, Evidence and Policy, 14(4): 665–82. doi:
McBride, E. et al. (2021) Behavioural science investment needed to mitigate the long-term health impacts of Covid-19, The British Psychological Society, https://www.bps.org.uk/news-and-policy/behavioural-science-investment-long-term-health-covid-19.
McManus, J., Constable, M., Bunten, A. and Chadborn, T. (2008) Improving People’s Health: Applying Behavioural and Social Sciences to Improve Population Health and Wellbeing in England, London: Public Health England.
Moffat, A., Cook, E.J. and Chater, A.M. (2022) Examining the influences on the use of behavioural science within UK local authority public health: qualitative thematic analysis and deductive mapping to the COM-B model and Theoretical Domains Framework, Frontiers in Public Health, 10: 1016076. doi:
Neves, J.A., Vasconcelos, F.D.A.G.D., Machado, M.L., Recine, E., Garcia, G.S. and Medeiros, M.A.T.D. (2022) The Brazilian cash transfer program (Bolsa Família): a tool for reducing inequalities and achieving social rights in Brazil, Global Public Health, 17(1): 26–42. doi:
OECD (Organisation for Economic Cooperation and Development) (2017) Behavioural insights and public policy: lessons from around the world, https://www.oecd.org/gov/regulatory-policy/behavioural-insights-and-public-policy-9789264270480-en.htm.
OECD (Organisation for Economic Cooperation and Development) (nd) Behavioural insights, https://www.oecd.org/gov/regulatory-policy/behavioural-insights.htm.
Oliver, K., Hopkins, A., Boaz, A., Guillot-Wright, S. and Cairney, P. (2022) What works to promote research-policy engagement?, Evidence & Policy, 18(4): 691–713. doi:
Oliver, K., Innvar, S., Lorenc, T., Woodman, J. and Thomas, J. (2014) A systematic review of barriers to and facilitators of the use of evidence by policymakers, BMC Health Services Research, 14: 1–12. doi:
Sniehotta, F.F. and Chater, A.M. (2018) Developing and applying translational behavioural science to improve population health and reduce inequalities, Health Psychology and Public Health, 2(2): 5–6, https://global-uploads.webflow.com/63236652b830869ac6463e34/6384c0998ab063cdfe4ebea3_HPPHN-Publication-Summer-2018-Volume-2-Issue-2.pdf.
UK Science and Technology Select Committee (2011) Second report: behaviour change, https://publications.parliament.uk/pa/ld201012/ldselect/ldsctech/179/179.pdf.
UN Innovation Network (2021) United Nations Behavioural Science Report, https://www.uninnovation.network/assets/BeSci/UN_Behavioural_Science_Report_2021.pdf.
Van de Goor, I. et al (2017) Determinants of evidence use in public health policy making: results from a study across six EU countries, Health Policy, 121(3): 273–81. doi:
Western, M. (2019) How to increase the relevance and use of social and behavioral science: lessons for policymakers, researchers and others, Justice Evaluation Journal, 2(1): 18–34. doi:
Whitty, C.J. (2015) What makes an academic paper useful for health policy? BMC Medicine, 13: 301. doi:
WHO (World Health Organisation) (2022) Behavioural sciences for better health initiative, Report by the Director-General, https://cdn.who.int/media/docs/default-source/documents/behavioural-science-a75_25.pdf?sfvrsn=c6319537_3&download=true.
