What do advocates want from policy research? Evidence from elite surveys

View author details View Less
  • 1 University of Missouri, , USA
  • | 2 George Washington University, , USA
Full Access
Get eTOC alerts
Rights and permissions Cite this article

Background:

Policy advocates play a key role linking the separate worlds of research and policymaking – often serving as research brokers who increase the use of research and promoting more informed decision making. Yet this group is often overlooked in studies of research utilisation.

Methods:

We undertook two surveys of state-level advocates in the United States in order to better understand the views of these ‘research brokers’ on the utility of research and the characteristics of research most needed in the policymaking process.

Findings:

The advocates we surveyed report that research plays an important, if limited, role in shaping the policy outcomes in their state. They value objective and unbiased research, as evidenced by the credibility of the source, and relevance to their state context. At the same time, advocates were not particularly interested in novel research on unfamiliar outcomes in other policy domains, instead preferring studies that stick to the familiar framing of the issue dominant in the policy community in which they work. Advocates use research findings primarily as justification for their policy positions.

Discussion and conclusion:

Perceived impartiality and objectivity are a major asset of academic researchers seeking to influence the policy process. Advocates value this credibility and other sources of information that they can use to justify their policy positions. At the same time, their preference for familiar rather than novel findings may limit the degree to which policy advocates can serve as intermediary for such results, hampering the ability of research to reframe policy debate.

Abstract

Background:

Policy advocates play a key role linking the separate worlds of research and policymaking – often serving as research brokers who increase the use of research and promoting more informed decision making. Yet this group is often overlooked in studies of research utilisation.

Methods:

We undertook two surveys of state-level advocates in the United States in order to better understand the views of these ‘research brokers’ on the utility of research and the characteristics of research most needed in the policymaking process.

Findings:

The advocates we surveyed report that research plays an important, if limited, role in shaping the policy outcomes in their state. They value objective and unbiased research, as evidenced by the credibility of the source, and relevance to their state context. At the same time, advocates were not particularly interested in novel research on unfamiliar outcomes in other policy domains, instead preferring studies that stick to the familiar framing of the issue dominant in the policy community in which they work. Advocates use research findings primarily as justification for their policy positions.

Discussion and conclusion:

Perceived impartiality and objectivity are a major asset of academic researchers seeking to influence the policy process. Advocates value this credibility and other sources of information that they can use to justify their policy positions. At the same time, their preference for familiar rather than novel findings may limit the degree to which policy advocates can serve as intermediary for such results, hampering the ability of research to reframe policy debate.

Key messages

  • Advocates value policy research from objective, unbiased sources like universities.

  • Liberal and conservative advocates value research that justifies their (distinct) policy positions.

  • Research on novel problems or indicators is less likely to be used by advocates.

  • Advocates can serve as effective research brokers, but not equally for all forms of research.

A great deal of attention has been paid to the separate worlds of research and policy – each with distinct norms, incentives, timelines, priorities, and modes of dissemination. Peterson describes these worlds as ‘largely reverse images of one another – veritable foreign lands with distinctive languages’ (Peterson, 2018: 346). There is no question that these differences limit the utility of policy research to actually inform policy decisions – even for those whom Bogenschneider and Corbett (2010) identify as ‘research-minded policymakers’ and ‘policy-minded researchers’. Typically, those best positioned to bridge this gap are neither the researcher nor the policymaker – but intermediaries who serve as research brokers (Rigby, 2005; Ward et al, 2009; Tseng, 2012). Research brokers translate and package complex research findings into ideas and formats that can be used by policymakers, and facilitate meetings and establish relationships between researchers and policymakers (Tseng, 2012). They play an essential role in the process of research utilisation.

Research brokers operate within foundations, think tanks, evaluation firms, government trade organisations, and policy-oriented organisations. Within these independent entities, intermediaries are well-positioned to play a liaison role – facilitating information dissemination (Neal et al, 2015). It is to these individuals and organisations that policymakers often turn for information when forming policy (Jabbar et al, 2014; Dodson et al, 2015; Patashnik and Peck, 2017), meaning they can serve as conduits for communicating new research findings to policymakers (Rigby and Morgan, 2018). As a result, these research brokers have the potential to play a critical role in using research to shape the policy agenda, identify promising solutions, and ultimately influence policy decisions (Rigby, 2005; Tseng, 2012).

Over the last few decades, we have seen a proliferation of organisations serving this intermediary research broker role within existing and newly-created organisations (Radin, 2013; Gen and Wright, 2018). An example is the advocacy coalition formed to defend the Affordable Care Act from repeal. In its first four months (January to April 2017), this coalition disseminated 122 briefs, reports, or data estimates for use in advocacy. Although drawing on a wide range of information, the vast majority of the pieces (85%) were written within the same week they were published – and therefore allowed for tailoring of the information to the current policy debate in ways most published research cannot be (Rigby and Morgan, 2018).

Within this extensive network of intermediaries, it is rare to find an organisation focused on research dissemination as a goal unto itself. Instead, organisations typically hold and advocate for their own policy priorities – often viewing research dissemination as a tool to address other goals. Contandriopoulos et al (2010: 464–5) suggest that this may be the most viable ‘equilibrium’ for research dissemination, yet others express concerns about the objectivity of research disseminated by research brokers in intermediary organisations (Knight and Lyall, 2013; Smith et al, 2013; Doberstein, 2017), asking whether potential research brokers may act as research gatekeepers – sorting and winnowing out research findings that they deem irrelevant for their policy priorities (Rigby, 2005; Ward et al, 2009).

Due to the central and potentially conflicted role played by research brokers in the research-to-policy translation process, we must better understand this group of policy actors. Yet most of the literature on research dissemination focuses on the needs, views, and priorities of knowledge users (policymakers, bureaucrats, and staff) followed by knowledge producers inside and outside of academia (for an overview, see Bogenschneider and Corbett, 2010). Less attention has been paid to the views of research brokers, particularly those disseminating research in pursuit of their own policy objectives. In this paper we focus on the potential for research brokering among a key set of actors in the policymaking process: advocates. We use this term broadly to refer to lobbyists and others who try to influence policymakers on behalf of some client, organisation, or issue. These advocates undertake a range of strategies, which may or may not include brokering research results.

To better understand the potential and implications of advocates operating as research brokers, we ask: how useful do advocates find research in the policymaking process? What characteristics of research do they value? Do their views differ by ideological orientation or policy context? And ultimately, do advocates operate as honest research brokers, research distorters, or research gatekeepers? Better understanding these issues can help both researchers and policymakers understand the advantages and limitations of relying on advocates to play this role, as well as provide guidance for strategically engaging advocates in research dissemination efforts.

We conduct a pair of elite surveys asking advocates how they define good research, and how they assess research as useful (or not) in the policymaking process. We focus on advocates working at organisations involved in economic policy at the state level in each of the American states (for example, unions, business groups, think tanks, and advocacy groups focused on low-income families). This strategy allowed us to generate a large enough sampling frame of both liberal and conservative advocates, while providing general comparability in terms of the nature of the research available to inform these policy debates. We find these advocates to be somewhat optimistic about the utility of research, to value objective and credible sources of information, and to perceive an importance in having research to support their positions when communicating with policymakers. At the same time, advocates describe a limited role of research in the policy process, and they expressed a clear preference for new research that sticks to the same indicators traditionally associated with the issue, rather than more novel research suggesting new ways to consider the policies they are working on. Together these findings provide insight into the situations in which advocates may best serve as research brokers, as well as those when they may instead serve as research gatekeepers who block the dissemination of more novel research findings into the policymaking process.

Background

Advocates as research brokers

There are a number of reasons why policymakers seek evidence from advocates. Policymakers are interested in identifying new policy problems, and in crafting solutions that will work. Information about policy problems and solutions helps to reduce uncertainty, which should be attractive for risk-averse, re-election-oriented politicians. Yet while the supply of information that policymakers have access to is quite large, much of it is ignored. Policymakers complain that research information is often irrelevant, out of date, or presented in a format that is not useful to them (Sorian and Baugh, 2002; Dagenais et al, 2015).

Advocates are well-positioned to step into this research-policy gap as brokers. Bridging this gap requires substantial work, including tailoring information to fit users’ needs, facilitating contacts between individuals, building the capacity of end users to understand information, assessing the needs of users, and adjusting activities to fit those needs (Dagenais et al, 2015: 11). Face-to-face verbal conversations and interpersonal channels of communication, which require both communication skills and time, are also an important means by which information is conveyed to policymakers (Dodson et al, 2015). Indeed, there is an extensive literature demonstrating that advocates are valuable sources of information for decision makers (Hall and Deardorff, 2006; Contrandriopoulos et al, 2010; Jabbar et al, 2014; Dodson et al, 2015). Others note that for academics, collaboration with advocates may be a promising channel for conveying relevant research findings to policymakers (Rigby and Morgan, 2018).

Are advocates ‘honest’ research brokers?

The most cynical view of advocates suggests that they use any research that meets their policy goals. In fact, Weiss (1979) describes a political model of research use in which findings are used as ‘ammunition’ for one side or the other, rather than for true decision making or enlightenment functions as in her other models of use. Although this type of distortion does occur, we note that the sophistication of policy research has risen over the last few decades (Radin, 2013), likely increasing scrutiny on efforts to use research this way. Further, policymakers are well aware that everything advocates do is in pursuit of the interests of their organisations or clients (Doberstein, 2017). This limits the trust that policymakers put in advocates, and trust of the source is crucial to the credibility of information (Sorian and Baugh, 2002; Lavis et al, 2003; Smith et al, 2013).

This reality suggests a few ways in which advocates may engage with research. In one scenario, an advocate may devalue research and downplay the potential for research to shape the policymaking process, opting out of research dissemination in favour of other advocacy strategies. In another, an advocate may adopt Weiss’ research-as-ammunition strategy and pay little attention to the source, rigour, objectivity, or comprehensiveness of the information they share – but eagerly work to disseminate biased information. We argue that a third type of response is more likely in today’s political and policy context: one in which an advocate may view research as influential in the policy process and actively engage in dissemination of ostensibly high-quality and unbiased research from outside of the political fray – but only when that information aligns with their policy priorities.

Any of these three responses may change the supply of information in the policymaking process, with advocates serving to block the dissemination of some or all information, as well as calling attention to other forms of information. In this way, we may think of them as research distorters. However, policy dissemination takes place in a complex environment with multiple advocates engaged in policy debates. Policymakers understand that advocates are focused on particular policy objectives and organisational priorities and may still be able to use this information to weigh the evidence presented to them by each advocate, providing a more pluralistic information environment to guide policy choices (Doberstein, 2017).

Advocates’ view of research: some expectations

We start from the premise that at least some factors that make research more attractive to policymakers should also tend to appeal to advocates using research to sway policymakers. This suggests that advocates should value research that is rigorous and credible. Of course, since assessing the quality of the research itself is difficult, advocates (like policymakers) are likely to place a premium on the perceived credibility and impartiality of the source (Sorian and Baugh 2002; Lavis et al, 2003). Advocates should be on the lookout for research employing data specific to their decision-making context – such as regional or local data for policymakers at those levels (Fielding and Frieden, 2004, Laugesen and Isett, 2013, van de Goor et al, 2017). The status of an issue on the policy agenda should also play a role in the way advocates view studies, since policy systems are characterized by ‘disproportionate information processing’ (Jones and Baumgartner, 2005: 5), in which policymakers only demand information once an issue is on the agenda. If advocates are attuned to what policymakers want (a comparative advantage of advocates relative to researchers), they should be most interested in studies that address issues currently on the agenda.

Advocates and policymakers are not necessarily the same on all dimensions, however. Because they are much more specialised, advocates are likely to belong to narrower intellectual networks than policymakers (Heclo, 1978). This may affect their receptivity to novel or unfamiliar pieces of information. Policymakers sometimes engage in what Baumgartner and Jones (2015) call ‘entropic search’ for information, considering indicators and issues that are not traditionally associated with a given policy area – often seeking new ways to claim credit for new or important ideas. This opens the door to ‘attribute intrusion’ (Jones and Baumgartner, 2005: 55), or a reframing of issues and problems.

Evidence suggests that advocates are less likely to engage in this reframing than policymakers. Baumgartner et al (2009) describe an ‘information-induced equilibrium’ among policy advocates working on both sides of 99 issues on the federal lobbying agenda in the US. They noted how advocates who disagreed on policy positions still tended to share the same understanding and information. As a result, they were unlikely to be swayed by new information, particularly efforts to reframe an issue. An advocate who has worked for years or decades on a particular issue may view new interpretations, frames, or measures as less credible or important than those with which they are familiar. Thus, advocates may serve as a barrier to novel information reaching the attention of policymakers who could find it useful.

In summary, we expect the advocates we survey to recognise some value of research in the policymaking process, particularly when that research is provided by credible sources, relevant to the current agenda, specific to their particular context, and focused on familiar (rather than novel) variables or indicators. Under these circumstances, advocates may serve as research brokers. Yet, given their motivations, we expect advocates to see research as a helpful tool for justifying their arguments to policymakers rather than as a means of learning and changing their policy positions. In this way advocates may play the role of research gatekeeper, rather than research broker.

Methods

Minimum wage policy as case study

We examine advocates’ preferences for policy research using the case of minimum wage policy in US states – a high-profile issue often included on legislative agendas. This issue also exhibits a high degree of polarisation – with Democrats favouring increases while Republicans oppose them – a common and important constraint on the knowledge transfer process (Contandriopoulos et al, 2010). Key to our interest in novel versus familiar information, minimum wage policy is also the subject of a growing body of literature in the field of population health, with many studies suggesting that laws mandating a higher minimum wage are associated with better health (for example, Aron et al, 2015; Rigby and Hatch, 2016). This creates the potential for attribute intrusion, through which policymakers considering minimum wage might come to contemplate health indicators (for example, mortality) alongside more familiar indicators of economic prosperity like growth and employment. This case allows us to better understand the role of advocates in bridging the research-to-policy gap by gaining clarity on what advocates value in terms of research and how they may be using information in their advocacy work.

We survey state-level policy advocates from a stratified sample of liberal and conservative policy organisations likely to be active in minimum wage debates in their respective states; this allows for comparison of respondents’ views by the ideological orientation of their organisation. While surveys may sacrifice nuance relative to open-ended interviews or participant observation, they allow us to gather easily comparable responses from large numbers of advocates, and to implement a randomised experiment designed to measure the appeal of novel versus familiar information (see below).

Survey experiment: novel versus familiar information

Recall that one of our expectations about advocates as potential research brokers is that they tend to favour familiar and established issue frames and information, rather than novel frames or outcomes that have the potential to change the political playing field on a given issue. To test whether this is the case for minimum wage policy, we run a unique survey experiment. In the survey, we asked state-level economic policy advocates to rate the likely political influence of a hypothetical academic study on the impacts of a minimum wage increase. This is an indirect measure of advocates’ views of research and its role in the policymaking process; this approach was selected in order to reduce social desirability bias in questions that ask directly about the value one puts on research or information. Roughly half of the respondents were randomised into a condition in which the hypothetical studies dealt with health impacts,1 while the other answered a parallel set of questions about studies on economic impacts, a more familiar set of outcomes for this policy area. We then allowed respondents to provide open-ended remarks on the types of policy research that would have the most impact.

The sample consists of 146 state-level policy advocates, 130 of whom answered all three of our experimental questions. In order to identify a sampling frame of state-level advocates that would be generally comparable across states, we identified seven national networks of state-level organisations active on this issue. Four of these were networks of liberal advocacy organisations: the Annie E. Casey Foundation Kids Count Network, The State Priorities Partnership, and affiliates of two different labour unions (the American Federation of Labor-Congress of Industrial Organizations and the Service Employees International Union). The other three were large networks of conservative advocacy organisations: the State Policy Network, state Chambers of Commerce, and affiliates of the National Federation of Independent Businesses. Although most of these organisations are non-profit and non-partisan officially, their engagement in either these liberal or conservative networks allows us to classify them ideologically and infer their likely position on minimum wage policy.

For each organisation, trained research assistants sought contact information for a defined set of relevant staff members: executives (executive directors, chief executive officers, presidents) and policy or government relations staff, particularly those focused on economic, business, or labour issues. This approach yielded a sampling frame of 439 individuals with available email addresses (54% identified through the liberal networks and 46% identified through the conservative networks). The online survey was fielded in early April 2016, with recruitment emails sent via Qualtrics. We sent follow-up emails on a weekly basis, and research assistants made follow-up phone calls. We closed the survey on June 8 with a final response rate of 31.3%.

While not a fully representative sample of the economic policy advocacy community, our final sample of 130 respondents from 122 organisations2 offers geographic representativeness, with respondents from 48 states and the District of Columbia. This gives us useful variation in the state political climate surrounding the minimum wage. The sample skews liberal, with 84 representatives of liberal groups compared to only 46 from conservative groups. This raises the concern that liberal groups responded because they are especially receptive to or interested in research, yet we found no evidence that responses or experimental effects differed based on whether the respondent was from a liberal or a conservative group, with some slight exceptions we report below. We must also acknowledge that the advocates who agreed to complete our surveys are those more likely to be interested in and open to research in general. Of course, it is also these advocates who are most likely to engage in the role of research broker within their policy community, and so it is their views we are most interested in understanding. We discuss the representativeness of the sample more below and in Appendix 3.

The experimental questions ask respondents to rate the likely influence of hypothetical academic studies on the state policy debate surrounding the minimum wage. We asked about the likely impact of studies showing both positive and negative impacts, and also asked respondents to rate the overall influence on the minimum wage policy debate of studies on the outcomes in question. The text of the first question is as follows, with the experimental manipulation in bold:

Imagine that researchers at your state university released a new and well-publicised study on the effects of a higher minimum wage for people in your state.

The study used a rigorous research design and produced strong evidence that an increase in the minimum wage would worsen [health outcomes (for example, increase chronic disease and reduce life expectancy) / economic outcomes (for example, increase poverty and reduce economic growth)] over the next ten years.

What would be the most likely effect of this study on the political debate? Please select a response on this 5-point scale. (‘1 – Make no difference’ to ‘5 – Make it much harder for advocates seeking to increase the minimum wage’.)

The next question then asked respondents to imagine that the same study had found a positive impact on the outcome in question, and to rate the likely effect on a scale ranging from ‘1 – Make no difference’ to ‘5 – Make it much easier for advocates seeking to increase the minimum wage’. The final experimental question then asked respondents for a global assessment of how influential studies on the effects of minimum wage policy on health (economic) outcomes are in state minimum wage debates, with a four-point response scale.3 Finally, we asked respondents to tell us in their own words what types of research studies they believe have an impact on the policy debate about the minimum wage. While respondents were not forced to answer this question, 101 chose to do so.

To capture the relevant political and policy context of the state, we also asked respondents to estimate the ‘current level of support in your state – among policymakers and the public – for efforts to increase the minimum wage’, with five response options running from ‘strong support’ to ‘strong opposition’. This question is intended to tap whether or not a minimum wage increase is on the policy agenda.

In addition, we expected that a number of other state-level, organisational-level, and individual-level factors may influence respondents’ views on policy research. While we kept our survey brief to maximise participation, we were able to collect data from other sources. In addition to the dichotomous variable dividing liberal and conservative organisations, we coded indicators for state branches of national organisations (as opposed to completely independent state-level organisations), and for business, labour, and children’s advocacy organisations. We also collected data on whether or not each listed individual was an executive (that is, president or chief executive officer) within their organisation. We then tasked trained research assistants with gathering information on individual and organisational campaign contributions from the National Institute on Money in Politics’ FollowTheMoney.org website, as well as the number of listings of the organisation as a state lobbying client.4 Table A3.1 in Appendix 3 summarises the differences between our respondents and other individuals in the sampling frame on the available variables. It appears that executives and representatives from business, labour, and national organisations were significantly less likely to respond relative to other individuals. In terms of campaign donations and lobbying, the picture is mixed: respondents tended to be heavier individual campaign contributors than non-respondents, while non-respondents tended to represent larger organisational contributors.

At the state level, we include measures of state citizen ideology as measured by Fording (2018), legislative professionalism as measured by Squire (2007), and an ordinal variable measuring partisan control of state government (Democratic, split, or Republican).

Survey: ranking factors that make studies influential

Informed by the results of the first survey, we implemented a second survey in the summer of 2017 – this time focused on more directly comparing the factors these advocates identify as most important for a study’s influence.

We fielded this survey to an overlapping but not identical sample of state policy advocates. Beginning with the previous study’s sampling frame but excluding people who unsubscribed from our 2016 emails, research assistants again consulted publicly available contact information to update the list. The end result was a list of 333 individuals with available email addresses, 127 of whom had taken the previous survey. These 333 advocates remained stratified by ideology, with 186 (56%) from liberal organisations and 147 (44%) from conservative organisations.

We followed similar email recruiting procedures to the first study, with the initial email send on July 11, 2017, and follow-up emails sent over the subsequent three weeks. A total of 79 respondents participated in the survey, for a response rate of 23.5%. Of those 79 responses, 56 (71%) were from respondents who also completed the previous survey. We once again achieved good geographic coverage, with responses from 74 organisations in 41 states, though the sample again skewed liberal, with 49 respondents from liberal groups to 27 from conservative groups.5 Table A3.2 compares the sample with the frame on this and other variables. In this case, children’s advocacy organisations were significantly overrepresented, while the underrepresentation of business, labour, and national organisations was less pronounced than in the 2016 study. Again, the overall picture is mixed for campaign contributions and lobbying.

The second survey begins with a question about what types of studies the respondent believes ‘have the biggest impact on the policy debate over the minimum wage in your state’. Respondents were presented with ten items in random order that they could drag and drop to rank order in descending importance from 1 to 10. These items, gleaned from the open-ended responses to the previous study, other research, and our theoretical expectations, were as follows:

  • Rigorous study design

  • Credible, trusted, impartial source

  • Contains novel or unexpected findings

  • Updates previous studies using the most recent data

  • Provides state-specific estimates and findings

  • Demonstrates effects on trusted economic outcomes (for example, unemployment)

  • Demonstrates effects on outcomes not typically considered in economic policy

  • Confirms the position of politicians in power

  • Challenges the position of politicians in power

  • Draws a lot of media attention

The next question asked respondents to rank in order five categories of organisations that produce research in terms of their impact on policy debates: universities, think tanks or policy research organisations, advocacy organisations or interest groups, government agencies, and legislative committees or offices. The final question asked respondents how important it is for them to have ‘strong research to back up [their] arguments’ in communication with policymakers, with a four-point response scale.6

Findings

Survey experiment findings

Before considering the effects of our treatments, we note that the simple descriptive results suggest that advocates are not completely cynical about the potential of research to influence policy. For the questions about hypothetical studies showing negative and positive impacts of raising the minimum wage, the median and (roughly) mean response was 3 on the 5-point scale, indicating that the respondent believed the study would ‘Make it somewhat harder (easier) for advocates seeking to increase the minimum wage’. For the general question about the influence of studies, the median and rough mean response was 2 on the 4-point scale, correspondent to ‘They have a small influence’. To be sure, these responses do not suggest that policy research dominates the policy process, but they do indicate that research has a meaningful role to play in the minds of these policy advocates.

Turning to our experimental findings, Figure 1 shows the mean response by experimental condition for the three questions of interest, with 95% confidence intervals. The results indicate support for the expectation that lobbyists would attribute greater effect or influence to the studies of economic outcomes. For both of the questions on hypothetical studies, however, these differences are modest (less than one fourth of a standard deviation) and fall short of statistical significance (p=.16 for the negative impact study and .21 for the positive impact study). For the question about overall influence, the difference is both more pronounced (more than one half of a standard deviation) and statistically significant (p<.001).

Figure 1:
Figure 1:

Mean responses to questions about the effect or influence of studies by experimental condition, with 95% confidence intervals

Citation: Evidence & Policy 17, 3; 10.1332/174426420X16000978959673

The experimental findings are consistent with our expectation that advocates will favour familiar, trusted measures and issue frames over novel ones. There are other possible interpretations of course – the differences may be due to the wording of these particular treatments, or may have something to do with health outcomes specifically rather than familiarity versus novelty more generally. This ambiguity is part of the motivation for the second study, discussed below.

Table 1 displays the results of OLS regressions incorporating the other variables. We found support for the expectation that research may be more influential when the issue is already moving forward, consonant with the agenda-setting literature (particularly Jones and Baumgartner, 2005). Specifically, respondents’ estimates of the current level of support for a minimum wage increase in the state was associated with how influential they expected a new study to be on the odds of policy change. We only found this effect for the question about the influence of a study finding a positive impact and for the question asking about the influence of research overall. There was no corresponding effect for the negative impact study question – respondents did not think negative impact studies would be more consequential in states with an anti-increase political climate. These findings suggest that new research may be most influential when it supports the direction policy is already moving, and less influential in stopping policy momentum or raising issues that have little existing political support.

Table 1:

Models of estimated influence of studies showing health or economic impacts of minimum wage increases

NegativePositiveInfluence
Health impact condition−0.24 (0.22)−0.22 (0.17)−0.47*** (0.12)
Conservative organisation−0.09 (0.24)0.22 (0.19)0.01 (0.14)
Estimated current support for MW increase0.01 (0.10)0.15 (0.08)0.19*** (0.06)
State Citizen Ideology, 2016−0.01 (0.01)0.01 (0.01)0.00 (0.01)
Legislative professionalism (Squire)−0.08 (0.92)0.94 (0.72)0.48 (0.53)
Party control of state government (ordinal)0.40 (0.23)−0.00 (0.18)−0.19 (0.13)
Executive (president, CEO, and so on)−0.07 (0.26)0.03 (0.20)−0.11 (0.15)
Branch of national organisation−0.15 (0.23)−0.31 (0.18)−0.01 (0.13)
Organisational contribution $−0.00 (0.00)0.00 (0.00)−0.00 (0.00)
Individual contribution $−0.00 (0.00)−0.00 (0.00)−0.00 (0.00)
Organisation lobbying listings0.00 (0.00)0.00 (0.00)−0.00 (0.00)
Constant3.16** (1.06)1.73* (0.83)2.41*** (0.61)
Observations128128127

Standard errors in parentheses

* p<0.05, ** p<0.01, *** p<0.001

We found null effects for all other state-, organisation-, and individual-level variables. Legislative professionalism has no discernible impact on responses. Neither did party control of state government or state citizen ideology, which suggests that the respondents’ estimates of the current level of support for an increase captured the most relevant part of the state political climate. Individuals’ assessments did not differ on the basis of holding an executive title or having contributed more to state political campaigns. Organisational characteristics (branch of a national organisation, total state campaign contributions, number of listings as a lobbying client) also appear to be unrelated to these answers.7 The results displayed in Table 1 are virtually unchanged when we incorporate sampling weights to account for the lack of representativeness of the sample relative to the frame (see Table A3.3).

In the final section of this survey, we asked respondents to tell us in their own words what types of research studies they believe have an impact on the policy debate about the minimum wage. This question allowed us to consider advocates’ attitudes toward research’s role in the policy process in their own words, as exemplified by comments like these:

Studies of employment effects have a modest impact, though tend to be accepted and rejected [in] large part based on whether they agree with preset opinion but this area does have a very modest effect with a few. Most other studies tend to be influential only with those who already agree. (Respondent from conservative organisation in economic treatment)

Research is supportive for advocates, but legislators don’t make decisions based primarily on research. It’s often about power and horse trading, not what makes good policy. (Respondent from liberal organisation in health treatment)

Academic studies have a place in the debate and the more rigorous the better. To believe, however, that facts identified in rigorous analysis significantly influence the outcome of policy debates is a bit naïve. My experience is that both sides of a policy debate must have some grounding in facts in order to persuade folks. Facts however are often in the eyes of the beholder and are bent to be consistent with the preconceived views of that beholder. (Respondent from liberal organisation in economic treatment)

These comments suggest a real, but circumscribed, role for research in the policy process on a hot-button issue like the minimum wage.

Table 2 displays the frequency with which respondents mentioned certain factors as important to a study’s influence. These include factors related to the qualities of studies: their empirical rigour (mentioned by 17%), their clarity (20%), and the objectivity of their sources (21%). This supports our expectation that advocates care about the content and credibility of research – or at least view more credible research as more likely to influence the policy debate. Other content themes that emerged were potential impact on small businesses (a common critique of minimum wage increases) and the potential of an increase to reduce government spending on social welfare programmes (a common argument made for increases).

Table 2:

Factors respondents identify as important to minimum wage studies’ influence

FactorsNumber mentioning% mentioning
Examines economic impact7069.3%
Shows positive impact5453.5%
Source of study2120.8%
Clarity of study2019.8%
Shows negative impact1918.8%
Rigour of study1716.8%
Impact on small business1615.8%
Government spending1211.9%
Examines health impact87.9%

Many policy elites also suggested that studies that come down on a specific side of the minimum wage debate (that is, show a positive or negative impact) are particularly effective. Fifty-four percent of respondents mentioned the value of research showing a positive effect, while 19% mentioned research showing a negative effect. Research demonstrating positive effects of the minimum wage was mentioned more often by those representing left-leaning organisations (62%, compared to 39% for right-leaning advocates). In contrast, it was primarily those from conservative organisations that highlighted the value of studies showing negative effects (36%, compared to 9% from liberal organisations). This is consistent with the notion that advocates are focused on research that serves their interests and illustrates how they view these studies to be more impactful in the policymaking process.

As expected, we saw some differences between those previously asked about health versus economic studies. Table 2 shows the clear importance of economic outcomes (mentioned by 69%) as compared to health outcomes (mentioned by 8%) – likely reflecting some of the resistance to new findings and new policy frames found in earlier analyses. In addition, 80% of those who received the health frame voluntarily raised the impact of research that looked at economic outcomes (compared to 58% of those in the economic treatment).

Ranking survey findings

For a more systematic examination of how advocates weight the relative importance of different characteristics of research studies, we turn to the second survey. A summary of respondents’ rankings of the study characteristics is displayed in a box plot in Figure 2. A few themes emerge from an initial examination of these results. First, the respondents place the greatest importance on a credible and trusted source and state-specific findings. There was no statistically significant difference between their rankings of these two items in a difference-of-means test, but a clear difference between these items and others. Notably, having an objective and credible source was rated as more important than having a rigorous study design. This is consistent with the idea that, since they are not experts themselves, advocates and the policymakers they seek to influence have few tools for assessing the rigour of a research study aside from the credibility of its source. Respondents again showed a preference for studies examining economic outcomes over those examining other policy domains. Following these two factors was the item describing studies that update previous findings; and then, rated even lower, was the item describing research with novel or unexpected findings. This lends some support to our interpretation of the experimental results in the first study: advocates express a fairly clear preference for familiar over novel frames or measures.8

Figure 2:
Figure 2:

Respondents’ rankings of study characteristics by importance for impacting the policy debate on the minimum wage (1=most important, 10=least important)

Citation: Evidence & Policy 17, 3; 10.1332/174426420X16000978959673

The second survey also allowed us to assess which sources were considered the most credible. Figure 3 displays box plots of respondents’ rankings. Universities and think tanks are clearly preferred over the other alternatives, including bureaucratic agencies. Interest groups and legislative committees or offices were rated as the least impactful, though their mean rankings were not statistically different from those of agencies.9 These results speak to the importance of perceived objectivity and impartiality – even though agencies, interest groups, and legislative committees may be closer to the policy process and better adapted to its culture, advocates see the research they produce as less impactful than that put out by more ‘academic’ organisations. Of course, think tanks occupy a somewhat ambiguous space in this story. Many think tanks are openly ideological, so it is possible that their appeal is not based on perceived impartiality. More work is needed to determine why advocates value think tank research specifically.10

Figure 3:
Figure 3:

Respondents’ rankings of organisations producing research by impact on policy debates (1=most impactful research, 5=least impactful research)

Citation: Evidence & Policy 17, 3; 10.1332/174426420X16000978959673

Responses to the final question, about the importance of research to the advocates’ jobs, further establishes the value placed on research and evidence: over 60% of respondents answered ‘very important’, while a further 28% responded ‘important’. Only a single respondent out of 79 rated research as ‘not important at all’.

All of these results were remarkably similar among the ‘liberal’ and ‘conservative’ subgroups in the sample. This suggests that lobbyists and advocates across the spectrum have similar perspectives on what matters in a research study; it also enhances our confidence that they are able to give honest and objective answers, and helps to mitigate concerns about our sample’s liberal tilt.

Discussion and conclusions

While extensive prior work on dissemination has found an important role for advocates in getting research into the hands of policymakers, we know comparatively little about the views of these individuals on research or its utilisation in the policymaking process. Our results offer some insights in this direction.

The credibility and objectivity of the research, signified most readily by its source, appears to be of major importance to policy advocates. Since advocates themselves are typically understood to be pushing a particular point of view that serves their group’s interests, it is valuable to them to point to research that comes from an unbiased source, such as a university. For all the rhetoric about academia as an out-of-touch ‘ivory tower’, advocates still express a demand for the sort of knowledge that can only be provided by organisations with enough distance from the process to be credibly objective. This demand for objectivity is likely heightened for polarised issues like the minimum wage.

Still, to conclude that advocates value quality research in and of itself would be naïve. The demand for academic research is likely limited to research aligned with the advocates’ policy agenda. Our results also suggest that outward signals of credibility and objectivity (a trusted and impartial source) are substantially more important than the actual rigour of the study, an unsurprising result given that those in the policy world do not always have the training to assess the methodological soundness of cutting-edge research. It is possible, then, that shoddy or biased research that is packaged to seem credible and impartial (perhaps by laundering it through a seemingly-objective policy research shop) could have as much influence through these channels as actual high-quality scholarship.

Our findings also suggest that advocates consider their particular state context and policy agenda when evaluating research. They rated research as more influential when it showed positive effects of a higher minimum wage and their state government was supportive of an increase, indicating that they are mindful of the current policy agenda (Jones and Baumgartner, 2005). Advocates in our study also placed a premium on state-level research. This is consistent with existing work that finds regional or local research to be under-supplied from the standpoint of policymakers (Fielding and Frieden, 2004; Laugesen and Isett, 2013; van de Goor et al, 2017). This factor ranked ahead of all others – far ahead of methodological rigour. For researchers wishing to make an impact, this suggests an uncomfortable trade-off: to have impact on local or regional policymakers, it may be preferable to focus research on the local or regional level, even if national data is of higher quality or more appropriate to a researcher’s questions. A study focused on a particular jurisdiction might be of less interest to academic journal editors and reviewers, but of greater interest to policymakers and the advocates who talk to them. The vacuum of geographically specific research from academics may also create opportunities for less-than-scrupulous actors to step in with biased research of their own.

Both our experimental and observational findings establish that advocates prefer research on familiar rather than novel indicators, at least in this case. Studies showing that the minimum wage can impact population health could be of interest to policymakers, who are sometimes open to considering new frames or attributes in relation to familiar issues (Jones and Baumgartner, 2005; Baumgartner and Jones, 2015), but the resistance of key intermediaries to picking up these findings is a barrier to putting such research into practice.

Conceptualising advocates as research brokers suggests that many advocates would be eager to partner with academics or other researchers viewed as objective and credible in order to strengthen their policy justifications. This provides an important avenue for academics to get their research incorporated into the policy discourse, since most academics lack the training and insider knowledge of advocates. Of course, this partnership would only be mutually beneficial when the research findings align with the advocates’ goals. Since this is understood by policymakers, working closely with advocates is something that can compromise the objective, unbiased, and independent reputation of academic researchers and institutions, so this is a delicate balance to sustain over time. At the same time, advocates are likely to have blind spots in terms of research that they fail to pick up and incorporate in the policy debate; this is particularly likely for research that attempts to introduce new dimensions, interrelationships, or outcomes into a long-standing policy debate characterised by advocates with well-honed positions and strong research justifications. Researchers with novel findings may be better off trying to connect with policymakers directly rather than relying on intermediaries.

Of course, some caveats apply. Our samples achieved good geographic coverage, but were not fully representative of the population of state-level economic policy advocates. While the similarity of the weighted analyses in Appendix 3 to the main results suggests that the findings may generalise well, the samples may be unrepresentative in unmeasured ways, particularly when it comes to interest in research. While recruiting a perfectly representative sample was not possible, we suspect that if we had, the responses may have reflected less interest in research than those of our sample. Nevertheless, we believe these surveys offer insights about how a substantial portion of the state-level economic policy advocacy community views research in the policy process.

The advantages and limitations advocates bring to the role of research broker are important to recognise and factor into any efforts to use advocacy organisations to help disseminate information needed in the policymaking process. It also points to the need to consider the policy goals and organisational priorities of intermediary organisations in general – who should be conceptualised as potential research gatekeepers as well as potential research brokers.

Notes

1

Randomisation was implemented using the features of the Qualtrics survey platform.

2

Fifteen respondents came from organisations with more than one respondent in the survey. This creates the potential for spillover effects of randomisation and other forms of bias. To guard against this possibility, we re-ran all of the analyses presented here on a modified sample including only the first respondent from each organisation to take the survey (N=120). The results are virtually identical to those reported in the manuscript.

3

See survey instrument in Appendix 1.

4

For both individuals and organisations, research assistants were trained to only code the number and amount of campaign contributions within the state for which the individual or organisation is listed in the sampling frame. This focuses the measures on state-level political activity, and greatly reduces the problem of misattributing contributions made by other individuals with the same name. Only contributions listed in 2017 or earlier were counted.

5

Nine respondents came from organisations with more than one respondent. The results reported here do not noticeably differ when we re-run the analyses on a modified sample including only the first respondent from each organisation (N=74).

6

The survey instrument can be viewed in Appendix 2.

7

Substituting the number of individual and organisational campaign contributions for the dollar amounts does not change these conclusions. Nor does replacing the number of listings as a lobbying client with a dichotomous indicator.

8

Figure A3.1 replicates Figure 2 using sampling weights (see Appendix 3 for details), producing nearly identical results.

9

Again, incorporating sampling weights does not change these conclusions; see Figure A3.2.

10

While Doberstein (2017) finds a significant preference for universities over think tanks among bureaucrats in British Columbia, we did not find any such difference.

Funding

There were no external funders for this project.

Acknowledgments

The authors would like to thank Colleen Barry, Douglas Dion, Sarah Gollust, Megan Hatch, Julia Lynch, Tracy Osborne, Julianna Pacheco, Eric Patashnik, three anonymous reviewers, and the editor for their helpful comments and suggestions. The authors would also like to thank Peter Chu, Cody Drolc, Adam Hoelting, Hang Qi, Rachel Schwartz, and Ruoyan Sun for research assistance.

Contributors statement

JH and ER conceived and planned this project together. They collaborated on designing and fielding the surveys. JH conducted the analyses. Both equally shared in the writing of the manuscript.

Research ethics statement

The project was declared exempt from full IRB review by the University of Michigan Health Sciences and Behavioral Sciences IRB on October 7, 2015.

Supplemental files

For all appendices and replication files, please visit https://sites.google.com/view/haselswerdtj/research

Conflict of interest

The authors declare that there is no conflict of interest.

References

  • Aron, L., Dubay, L., Zimmerman, E., Simon, S., Chapman, D. and Woofe, S. (2015) Can Income-Related Policies Improve Population Health?, Center on Society and Health and Urban Institute, Washington, DC, https://www.urban.org/research/publication/can-income-related-policies-improve-population-health

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R. and Jones, B.D. (2015) The Politics of Information: Problem Definition and The Course of Public Policy In America, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R., Jeffrey, M.B., Marie, H., Leech, B.L. and Kimball, D.C. (2009) Lobbying and Policy Change: Who Wins, Who Loses, and Why?, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Bogenschneider, K., and Corbett, T.J. (2010) Evidenced-Based Policymaking: Insights from Policy-Minded Researchers and Research-Minded Policymakers, Abingdon: Routledge/Taylor & Francis Group.

    • Search Google Scholar
    • Export Citation
  • Contandriopoulos, D., Marc, L., Jean‐Louis, D. and Émile, T. (2010) Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature, Milbank Quarterly, 88(4): 44483.

    • Search Google Scholar
    • Export Citation
  • Dagenais, C., Marie-Claire, L. and Mélodie, B.-L. (2015) Knowledge brokering in public health: a critical analysis of the results of a qualitative evaluation, Evaluation and Program Planning, 53: 1017.

    • Search Google Scholar
    • Export Citation
  • Doberstein, C. (2017) Whom do bureaucrats believe? A randomized controlled experiment testing perceptions of credibility of policy research, Policy Studies Journal, 45(2): 384405.

    • Search Google Scholar
    • Export Citation
  • Dodson, E., Geary, N.A. and Brownson, R.C. (2015) State legislators’ sources and use of information: bridging the gap between research and policy, Health Education Research, 30(6): 84048.

    • Search Google Scholar
    • Export Citation
  • Fielding, J.E. and Frieden, T.R. (2004) Local knowledge to enable local action, American Journal of Preventive Medicine, 27(2): 1834.

    • Search Google Scholar
    • Export Citation
  • Fording, R.C. (2018) State ideology data, https://rcfording.wordpress.com/state-ideology-data/

  • Gen, S. and Wright, A.C. (2018) Strategies of policy advocacy organizations and their theoretical affinities: evidence from Q-Methodology, Policy Studies Journal, 46(2): 298325.

    • Search Google Scholar
    • Export Citation
  • Hall, R.L. and Deardorff, A.V. (2006) Lobbying as legislative subsidy, American Political Science Review, 100(1): 6984.

  • Heclo, H. (1978) Issue networks and the executive establishment, in A. King (ed) The New American Political System, Washington, DC: American Enterprise Institute, pp 87124.

    • Search Google Scholar
    • Export Citation
  • Jabbar, H., La Londe, P.G., DeBray, E., Scott, J. and Lubienski, C. (2014) How policymakers define evidence: the politics of research use in New Orleans, Policy Futures in Education, 12(8): 101327.

    • Search Google Scholar
    • Export Citation
  • Jones, B.D. and Baumgartner, F.R. (2005) The Politics of Attention: How Government Prioritizes Problems, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Knight, C. and Lyall, C. (2013) Knowledge brokers: the role of intermediaries in producing research impact, Evidence & Policy, 9(3): 30916.

    • Search Google Scholar
    • Export Citation
  • Laugesen, M.J. and Isett, K.R. (2013) Evidence use in New York city public health policymaking, Frontiers in Public Health Services and Systems Research, 2(7): 37.

    • Search Google Scholar
    • Export Citation
  • Lavis, J.N., Robertson, D., Woodside, J.M., McLeod, C.B. and Abelson, J. (2003) How can research organizations more effectively transfer research knowledge to decision makers?, Milbank Quarterly, 81(2): 22148.

    • Search Google Scholar
    • Export Citation
  • Neal, J.W., Neal, Z.P., Kornbluh, M., Mills, K.J. and Lawlor, J.A. (2015) Brokering the research-practice gap: a typology, American Journal of Community Psychology, 56 (3–4): 42235.

    • Search Google Scholar
    • Export Citation
  • Patashnik, E.M. and Peck, J. (2017) Can congress do policy analysis: the politics of problem solving on Capitol Hill, in L.S. Friedman (ed) Does Policy Analysis Matter? Exploring its Effectiveness in Theory and Practice, Berkeley, CA: University of California Press, pp 85153.

    • Search Google Scholar
    • Export Citation
  • Peterson, M.A. (2018) In the shadow of politics: the pathways of research evidence to health policy making, Journal of Health Politics, Policy, and Law, 43(3): 34176.

    • Search Google Scholar
    • Export Citation
  • Radin, B. (2013) Beyond Machievelli: Policy Analysis at Midlife, 2nd edn, Washington, DC: Georgetown University Press.

  • Rigby, E. (2005) Linking research and policy on Capitol Hill: insights from research brokers, Evidence & Policy, 1(2): 195213.

  • Rigby, E. and Hatch, M. (2016) Incorporating economic policy into a health-in-all-policies agenda, Health Affairs, 35(11): 204452.

  • Rigby, E. and Morgan, K. (2018) Academic research and legislative advocacy: information use in the campaign against repeal of the ACA, Journal of Health Politics, Policy, and Law, 43(3): 51135.

    • Search Google Scholar
    • Export Citation
  • Smith, K.E., Kay, L. and Torres, J. (2013) Think tanks as research mediators? Case studies from public health, Evidence & Policy, 9(3): 37190.

    • Search Google Scholar
    • Export Citation
  • Sorian, R. and Baugh, T. (2002) Power of information: closing the gap between research and policy, Health Affairs, 21(2): 26473.

  • Squire, P. (2007) Measuring state legislative professionalism: the Squire Index revisited, State Politics & Policy Quarterly, 7(2): 21127.

    • Search Google Scholar
    • Export Citation
  • Tseng, V. (2012) The uses of research in policy and practice, Social Policy Report, Society for Research on Child Development, 26(2): 123.

    • Search Google Scholar
    • Export Citation
  • van de Goor, I., Hämäläinen, R-M., Syed, A., Lau, C.J., Sandu, P., Spitters, H., Karlsson, L.E., Dulf, D., Valente, A., Castellani, T., Arof, A.R. and on behalf of the REPOPA consortium (2017) Determinants of evidence use in public health policy making: results from a study across six EU countries, Health Policy, 121(3): 27381.

    • Search Google Scholar
    • Export Citation
  • Ward, V., House, A. and Hamer, S. (2009) Knowledge brokering: the missing link in the evidence to action chain?, Evidence & Policy, 5(3): 26779.

    • Search Google Scholar
    • Export Citation
  • Weiss, C.H. (1979) The many meanings of research utilization, Public Administration Review, 39(5): 42631.

  • View in gallery

    Mean responses to questions about the effect or influence of studies by experimental condition, with 95% confidence intervals

  • View in gallery

    Respondents’ rankings of study characteristics by importance for impacting the policy debate on the minimum wage (1=most important, 10=least important)

  • View in gallery

    Respondents’ rankings of organisations producing research by impact on policy debates (1=most impactful research, 5=least impactful research)

  • Aron, L., Dubay, L., Zimmerman, E., Simon, S., Chapman, D. and Woofe, S. (2015) Can Income-Related Policies Improve Population Health?, Center on Society and Health and Urban Institute, Washington, DC, https://www.urban.org/research/publication/can-income-related-policies-improve-population-health

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R. and Jones, B.D. (2015) The Politics of Information: Problem Definition and The Course of Public Policy In America, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Baumgartner, F.R., Jeffrey, M.B., Marie, H., Leech, B.L. and Kimball, D.C. (2009) Lobbying and Policy Change: Who Wins, Who Loses, and Why?, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Bogenschneider, K., and Corbett, T.J. (2010) Evidenced-Based Policymaking: Insights from Policy-Minded Researchers and Research-Minded Policymakers, Abingdon: Routledge/Taylor & Francis Group.

    • Search Google Scholar
    • Export Citation
  • Contandriopoulos, D., Marc, L., Jean‐Louis, D. and Émile, T. (2010) Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature, Milbank Quarterly, 88(4): 44483.

    • Search Google Scholar
    • Export Citation
  • Dagenais, C., Marie-Claire, L. and Mélodie, B.-L. (2015) Knowledge brokering in public health: a critical analysis of the results of a qualitative evaluation, Evaluation and Program Planning, 53: 1017.

    • Search Google Scholar
    • Export Citation
  • Doberstein, C. (2017) Whom do bureaucrats believe? A randomized controlled experiment testing perceptions of credibility of policy research, Policy Studies Journal, 45(2): 384405.

    • Search Google Scholar
    • Export Citation
  • Dodson, E., Geary, N.A. and Brownson, R.C. (2015) State legislators’ sources and use of information: bridging the gap between research and policy, Health Education Research, 30(6): 84048.

    • Search Google Scholar
    • Export Citation
  • Fielding, J.E. and Frieden, T.R. (2004) Local knowledge to enable local action, American Journal of Preventive Medicine, 27(2): 1834.

    • Search Google Scholar
    • Export Citation
  • Fording, R.C. (2018) State ideology data, https://rcfording.wordpress.com/state-ideology-data/

  • Gen, S. and Wright, A.C. (2018) Strategies of policy advocacy organizations and their theoretical affinities: evidence from Q-Methodology, Policy Studies Journal, 46(2): 298325.

    • Search Google Scholar
    • Export Citation
  • Hall, R.L. and Deardorff, A.V. (2006) Lobbying as legislative subsidy, American Political Science Review, 100(1): 6984.

  • Heclo, H. (1978) Issue networks and the executive establishment, in A. King (ed) The New American Political System, Washington, DC: American Enterprise Institute, pp 87124.

    • Search Google Scholar
    • Export Citation
  • Jabbar, H., La Londe, P.G., DeBray, E., Scott, J. and Lubienski, C. (2014) How policymakers define evidence: the politics of research use in New Orleans, Policy Futures in Education, 12(8): 101327.

    • Search Google Scholar
    • Export Citation
  • Jones, B.D. and Baumgartner, F.R. (2005) The Politics of Attention: How Government Prioritizes Problems, Chicago, IL: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Knight, C. and Lyall, C. (2013) Knowledge brokers: the role of intermediaries in producing research impact, Evidence & Policy, 9(3): 30916.

    • Search Google Scholar
    • Export Citation
  • Laugesen, M.J. and Isett, K.R. (2013) Evidence use in New York city public health policymaking, Frontiers in Public Health Services and Systems Research, 2(7): 37.

    • Search Google Scholar
    • Export Citation
  • Lavis, J.N., Robertson, D., Woodside, J.M., McLeod, C.B. and Abelson, J. (2003) How can research organizations more effectively transfer research knowledge to decision makers?, Milbank Quarterly, 81(2): 22148.

    • Search Google Scholar
    • Export Citation
  • Neal, J.W., Neal, Z.P., Kornbluh, M., Mills, K.J. and Lawlor, J.A. (2015) Brokering the research-practice gap: a typology, American Journal of Community Psychology, 56 (3–4): 42235.

    • Search Google Scholar
    • Export Citation
  • Patashnik, E.M. and Peck, J. (2017) Can congress do policy analysis: the politics of problem solving on Capitol Hill, in L.S. Friedman (ed) Does Policy Analysis Matter? Exploring its Effectiveness in Theory and Practice, Berkeley, CA: University of California Press, pp 85153.

    • Search Google Scholar
    • Export Citation
  • Peterson, M.A. (2018) In the shadow of politics: the pathways of research evidence to health policy making, Journal of Health Politics, Policy, and Law, 43(3): 34176.

    • Search Google Scholar
    • Export Citation
  • Radin, B. (2013) Beyond Machievelli: Policy Analysis at Midlife, 2nd edn, Washington, DC: Georgetown University Press.

  • Rigby, E. (2005) Linking research and policy on Capitol Hill: insights from research brokers, Evidence & Policy, 1(2): 195213.

  • Rigby, E. and Hatch, M. (2016) Incorporating economic policy into a health-in-all-policies agenda, Health Affairs, 35(11): 204452.

  • Rigby, E. and Morgan, K. (2018) Academic research and legislative advocacy: information use in the campaign against repeal of the ACA, Journal of Health Politics, Policy, and Law, 43(3): 51135.

    • Search Google Scholar
    • Export Citation
  • Smith, K.E., Kay, L. and Torres, J. (2013) Think tanks as research mediators? Case studies from public health, Evidence & Policy, 9(3): 37190.

    • Search Google Scholar
    • Export Citation
  • Sorian, R. and Baugh, T. (2002) Power of information: closing the gap between research and policy, Health Affairs, 21(2): 26473.

  • Squire, P. (2007) Measuring state legislative professionalism: the Squire Index revisited, State Politics & Policy Quarterly, 7(2): 21127.

    • Search Google Scholar
    • Export Citation
  • Tseng, V. (2012) The uses of research in policy and practice, Social Policy Report, Society for Research on Child Development, 26(2): 123.

    • Search Google Scholar
    • Export Citation
  • van de Goor, I., Hämäläinen, R-M., Syed, A., Lau, C.J., Sandu, P., Spitters, H., Karlsson, L.E., Dulf, D., Valente, A., Castellani, T., Arof, A.R. and on behalf of the REPOPA consortium (2017) Determinants of evidence use in public health policy making: results from a study across six EU countries, Health Policy, 121(3): 27381.

    • Search Google Scholar
    • Export Citation
  • Ward, V., House, A. and Hamer, S. (2009) Knowledge brokering: the missing link in the evidence to action chain?, Evidence & Policy, 5(3): 26779.

    • Search Google Scholar
    • Export Citation
  • Weiss, C.H. (1979) The many meanings of research utilization, Public Administration Review, 39(5): 42631.

  • 1 University of Missouri, , USA
  • | 2 George Washington University, , USA

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 23 23 22
PDF Downloads 3 3 2

Altmetrics

Dimensions