Co-creative technical assistance: essential functions and interim outcomes

View author details View Less
  • 1 University of North Carolina at Chapel Hill, , USA
  • | 2 The Implementation Group
Open access
Get eTOC alerts
Rights and permissions Cite this article

Background

A large national technical assistance (TA) centre in the US provides TA to Head Start regions to help strengthen their use of implementation science frameworks. In administration of TA to regions, the centre has used a ‘co-creation’ model.

Aims and objectives

This paper describes the co-creative TA approach, including level of dosage (frequency and duration) provided, and interim outcomes achieved.

Methods

The descriptive paper relies on secondary data analyses of information gathered in the course of delivering the technical assistance. The information gathered included observations of events, surveys, and listening sessions with TA participants.

Findings

Analyses revealed that the TA providers were successful in the provision of co-created TA and that interim outcomes of trust, mutual accountability, and some integration of implementation science concepts into ongoing regional work were achieved.

Discussion and conclusions

The TA providers succeeded in delivering TA that was perceived by both observers and participants as involving participants and being tailored to their settings and needs. Limitations include a small sample that was selected based on interest and readiness for engaging in TA and descriptive analyses that did not allow for claims of causality. The promising co-creation approach deserves additional exploration and may provide guidance to others designing technical assistance.

Abstract

Background

A large national technical assistance (TA) centre in the US provides TA to Head Start regions to help strengthen their use of implementation science frameworks. In administration of TA to regions, the centre has used a ‘co-creation’ model.

Aims and objectives

This paper describes the co-creative TA approach, including level of dosage (frequency and duration) provided, and interim outcomes achieved.

Methods

The descriptive paper relies on secondary data analyses of information gathered in the course of delivering the technical assistance. The information gathered included observations of events, surveys, and listening sessions with TA participants.

Findings

Analyses revealed that the TA providers were successful in the provision of co-created TA and that interim outcomes of trust, mutual accountability, and some integration of implementation science concepts into ongoing regional work were achieved.

Discussion and conclusions

The TA providers succeeded in delivering TA that was perceived by both observers and participants as involving participants and being tailored to their settings and needs. Limitations include a small sample that was selected based on interest and readiness for engaging in TA and descriptive analyses that did not allow for claims of causality. The promising co-creation approach deserves additional exploration and may provide guidance to others designing technical assistance.

key messages

  • A co-creative technical assistance approach was delivered in three Head Start regions.

  • Interim outcomes of trust and mutual accountability were achieved.

  • The small-scale analysis highlights a promising approach to providing technical assistance.

Background

A large national technical assistance (TA) centre in the US provides TA to Head Start Regions to strengthen their use of implementation science in the application of evidence-based practices to improve outcomes for children and families served by Head Start programmes. Implementation science has been defined as the study and application of methods and processes designed to support achievement of a programme or activity that has clearly defined components (Eccles and Mittman, 2006; Fixsen et al, 2005). The goal of the TA examined here was ‘to advance best practices in the identification, development, and promotion of the implementation of evidence-based development and teaching and learning practices that are culturally and linguistically responsive and lead to positive child outcomes across early childhood programs’ (National Center on Early Childhood Development, Teaching, and Learning, 2018). In administration of TA to regions, the centre has used a ‘co-creation’ model. Co-creation is the active involvement of stakeholders in all stages of the production and implementation process resulting in service models, approaches, and practices that are contextualised and tailored to settings (Metz and Bartley, 2015; Vargo and Lusch, 2004). Over a two-year period, three Head Start regions identified a set of improvement objectives, and the national centre used a responsive, co-creation TA approach to support the regions in meeting their objectives. The purpose of this analysis of secondary data was to describe the co-creative TA approach taken and interim outcomes achieved.

TA approaches can vary along dimensions related to dosage, delivery methods, and levels of collaboration (Wandersman et al, 2012). Knowledge of best practices for the provision of TA is at an early stage, and evidence suggests that most TA is delivered without consideration of an organising framework or conceptual model (Katz and Wandersman, 2016; Le et al, 2016). Generally, a higher dose of TA has been found to relate to greater improvements in capacity and/or implementation (Leeman et al, 2015). Both in-person and virtual TA can be effective, but generally on-site technical assistance, compared to TA that is telephone- or email-based, has been shown to be more likely to lead to chances for experiential learning and demonstration of skills (Becker et al, 2008; Feinberg et al, 2008). The quality of the collaborative relationship between TA providers and participants has emerged as critical to successful implementation of practices (Chilenski et al, 2016).

Co-creation approaches to providing TA have emerged as an important component of effective and sustainable implementation capacity building (Metz and Albers, 2014). Through co-creation, recipients and beneficiaries of TA are actively involved in all stages of planning and implementing the TA resulting in models, approaches, and practices that are contextualised and tailored to their settings (Metz and Bartley, 2015; Vargo and Lusch, 2004). The goal of contextualisation is to ensure there is a match between TA and the values, needs, skills, and resources of those delivering interventions, systems stakeholders, and service beneficiaries (Horner et al, 2014; Van Kerkhoff and Lebel, 2015). A recent systematic review of the academic literature on co-creation and co-production in the public sector suggested a need for greater specificity in defining co-creation and more examination of outcomes (Voorberg et al, 2014).

The co-creative TA approach examined in this paper was designed to strengthen the integration and application of implementation science concepts in the regions in order to increase capacity for the service and systems changes needed to enable the use of evidence-based approaches. There is increasing attention in the literature to specific competencies needed to facilitate implementation and evidence use (Berta et al, 2015). The co-creative TA approach for the current project relied on a model of change that specified essential functions and interim and project-end outcomes (see Figure 1). The essential functions that supported co-creation included co-learning, brokering, facilitation, addressing power differentials, co-design, and tailored support, with each of these defined as part of a practice profile for the implementation specialists delivering the TA (Metz et al, 2017; see Table 1).

Figure 1:
Figure 1:

Theory of change for co-creative technical assistance with an implementation science perspective

Citation: Evidence & Policy 15, 3; 10.1332/174426419X15468578679853

Table 1:

Descriptions of co-creation essential functions and core activities

Essential functionCore activities
Co-learning – work collaboratively with systems stakeholders to learn how applied knowledge on how implementation science can be effectively used in the local context. Implementation specialists are open to learning about the history and current priorities in the local context in order to assess the most feasible and relevant uses of implementation science.Understand the system and organisational context and culture.
Create spaces for new ideas to emerge (Space can be created through asking questions and structured facilitation processes; and physically created through meeting places and room setup).
Negotiate, build trust and respect for all perspectives.
Communicate and listen for the purpose of mutual understanding and for collaborative integration of different knowledge perspectives.
Seek ways to introduce and get buy-in for an implementation science approach that fits with existing programmes, practices and processes.
Synthesise diverse perspectives of thought, and check for understanding.
Seek opportunities to reflect on the problem, the implementation specialist’s personal experience, and the intention and interaction with others.
Support collaborative implementation planning.
Brokering – enable knowledge exchange and sharing among stakeholders to increase understanding of diverse perspectives and increase the application of implementation science to improve outcomes.Connect otherwise disconnected individuals or groups in the system by providing advice and serving as a relational resource.
Position themselves ‘in between’ people or groups in a system network who are disconnected but whose connections are vital for the success of the change effort.
Share evidence and data and promote opportunities for stakeholders and team members to engage with others in the use of data.
Address power differentials – address power imbalances between community members, stakeholders in the wider system, technical assistance providers, and researchers, by building trust, supporting two-way communication, cultivating opportunities for mutual consultation, and identifying many accountabilities.Include diverse expertise in team discussions.
Position the range of service beneficiary experiences at the centre of decision making and implementation activities.
Recognise and acknowledge loss of status and authority that can impede buy-in and engagement.
Develop an evolving ‘collective view’ or ‘shared understanding’, rather than pushing for consensus which is often artificial and perpetuates power structures.
Co-design – co-design tools, resources, and models through iterative processes and consensus building.Co-design tools, products, processes, governance structures, service models, and policy.
Facilitate design-centred activities that use collective sense-making and negotiation.
Conduct cyclical tests of change to iteratively improve prototypes of tools, products and processes to support implementation efforts.
Ongoing testing and improving of tools, products, processes, governance structures, service models, and policy to support implementation efforts.
Facilitation – enable a process of participatory problem solving and support that occurs in a context of a recognised need for improvement and supportive interpersonal relationships. Successful facilitation promotes cycles of ‘mutual consultations’ among stakeholders to ensure that different forms of knowledge and ways of knowing are integrated into planning and solutions (Powell et al, 2015). Implementation specialists are guided by four core values for participant engagement (Kaner, 2014):Serve as formal and informal facilitators as determined by analysis of context and strategy.
Support a balance of divergent and convergent thinking among team members, depending on the type of challenge faced.
For easily-named and easily-solved challenges (technical challenges), support stakeholders to evaluate alternatives, summarise key points, sort ideas into categories, and exercise judgement.
For complex challenges with no easy solution (adaptive challenges), support stakeholders to generate alternatives, free- flow open discussion, gather diverse points of view, and suspend judgement.
Create welcoming spaces for all participants in meetings.
Select and use structured facilitation method ahead of group discussion, depending on the type of challenge, to ensure that appropriate strategies are used to address different types of problems.
Support a communication protocol and process that facilitates interactions among stakeholders.
Full participation where all stakeholders are encouraged to share their perspectives.
Mutual understanding where stakeholders accept the legitimacy of one another’s needs and goals.
Inclusive solutions that emerge from the integration of everybody’s perspectives and needs.
Shared responsibility of stakeholders to implement proposals they endorse and to give and receive input before final decisions are made.
Tailored support – frequency, duration and intensity of implementation supports depends on the needs, goals and context of the implementation team and systems stakeholders. Implementation specialists refrain from assumptions that a certain level and type of support is always needed.Assess and agree to the implementation support to be made available to each individual site and/or collectively to a number of sites.
Schedule virtual and onsite meetings based on the goals of the team and stakeholders.
Tailor support based on ‘just in time’ needs of the team and systems stakeholders.
Assess the effectiveness of the level of support in meeting needs, goals, and context of the implementation effort.

The three regions that participated in the project began with a strong interest and readiness to engage the TA staff to help build their implementation capacity. Each region identified a core team that would participate in TA activities and communicate and coordinate with regional staff beyond the core team. Each region also identified one or two specific areas of need for TA support that were individualised for their region. One region (Region A) expressed an interest in a virtual webinar series that would most efficiently engage their core and broader staff in learning about implementation science principles, while the other two (Regions B and C) each expressed interest in a more intensive TA approach that would help to further assess specific needs within their broader interest of incorporating implementation science principles into their grantee support processes.

From the beginning of the process with the regions, the TA team used the co-creative approach by first discussing roles and expectations with each region using a ‘Give and Get’ activity that attempted to set the stage for shared expectations and clear roles. This was designed to allow each region to arrive at a shared agreement with the TA staff on what regional staff would contribute to the TA process (through participation and development work), and what to expect from the TA staff (through guidance and facilitation). The TA staff also used initial meetings to assess current capacity with the regions, and facilitated discussions on identifying next steps in their work. These specific activities were not prescribed but emerged out of a co-creative process with each region.

Over the two-year period, the TA staff met in-person and virtually and communicated electronically with regional staff in the provision of TA. Each meeting and interaction, and the frequency with which they occurred, was tailored to the region’s needs and goals. For example, in-person meetings were scheduled when the region felt as though they would benefit most from the in-person support. Tailored support can be contrasted with traditional TA models that prescribe the dosage of support prior to the engagement with the recipient. This paper summarises findings related to whether there was evidence that (1) participants experienced the TA as co-creative, and (2) interim outcomes were achieved.

Methods

TA was provided to the regions over a two-year period (June 2016–June 2018). The data gathered for purposes of informing the TA were analysed as secondary data for this article. This section describes the TA participants, data collection procedures, measures, and analytic approach.

Participants

The US is divided into ten regional offices that serve states, territories, tribes, and other grantees in a defined geographical area. Staff from three of these ten regions participated in the technical assistance. The regions were located in the South Central (five states), Central (four states), and Western (six states) US. The regions self-selected into this activity by expressing an interest in and readiness for engaging the national centre staff in building their region’s implementation capacity. A total of 52 regional staff participated in the TA provided by the national centre. The roles of these individuals included fiscal, programme, early childhood, and regional TA specialists and coordinators.

Procedures

Data to inform the TA were gathered through three mechanisms. First, national centre staff who were not directly involved in delivering the TA conducted observations of all virtual and on-site workshops to track utilisation of co-creative essential functions. Observations documented the specific examples of use of co-creative essential functions, as well as the frequency and length of time during which these functions were used in the TA activity. Second, TA staff administered anonymous paper or electronic feedback surveys at the end of all on-site workshops. The electronic surveys (Qualtrics) were emailed to participants, with a reminder sent a week later. Regions that preferred paper surveys completed them at the end of the event. Surveys were anonymous; we therefore do not know how many of the 52 participants completed one or more survey, but across all events, a total of 91 surveys were distributed, and 80 were received, for a response rate of 88%. Finally, national centre staff who were not involved in TA delivery moderated separate virtual listening sessions with each of the regions. Across the three listening sessions, 19 regional staff participated (range = two to ten per session). Questions were asked about participants’ perceptions of the TA and outcomes achieved. The sessions were audio-recorded and the audio was transcribed. Two team members coded the transcriptions independently. A detailed codebook was developed that included specific definitions of each co-creative essential function, activities that would signify co-creation in general implementation support, and finally specific examples from the work of the national TA centre for each co-creative function. The coders discussed the independently derived codes and resolved discrepancies by consensus. Overall, there was 95% agreement between observers for the codes (range = 90–96% for each listening session).

Measures

A standard observation protocol and coding scheme were used to document utilisation of co-creative essential functions during virtual and on-site workshops. Observers used a spreadsheet to code workshop activities for evidence of co-creation and noted the start and stop times of each activity. The same codebook described above for coding the listening sessions was used to live-code activities during the workshops by a single coder. The electronic surveys administered after each on-site event included questions about the relevance, feasibility, and usefulness of the specific TA provided as well as about interim outcomes. For example, the relevance and feasibility of a process-mapping activity were rated with Likert-type items, and outcomes of trust and use of implementation concepts were rated (for example, ‘I feel as though I can trust the TA team’; ‘We have integrated IS concepts into our products and processes to support regional efforts’). A facilitators’ guide was used for the listening sessions to provide standard procedures and questions across the three sessions. Copies of the instruments used are available upon request of the lead author.

Analytic plan

The TA data were analysed descriptively for this paper. Observation and survey data from the TA workshops were compiled across regions and events. The information from the three listening sessions was summarised with notes and analysed qualitatively in stages. First, a national centre staff read the notes from each session, listened to the audio recordings to fill in gaps in the written notes, and summarised the information by themes. A second staff member reviewed the notes and coded for reliability and provided feedback.

Findings

The data were analysed descriptively to understand how much TA was provided. Table 2 provides information on level of service, including both frequency of contact and number of hours of TA provided. Across regions, the average number of events was about 12 over the course of two years, and the average number of total hours was about 23. Region A received a lower dose, which matched their request for a virtual workshop series rather than the more intensive TA requested by regions B and C.

Table 2:

Number of technical assistance events and hours

REGION AREGION BREGION CTOTAL
On-site events0325
On-site hours0261238
Virtual events3191032
Virtual hours81211.531.5
Total events3221237
Total hours83823.569.5

The three sources of data – observation, survey, and listening session – were summarised to determine whether the TA providers succeeded in delivering TA that was co-creative. Observers of the events coded the activities during the workshops; survey and listening session data provided participant perceptions of the TA. All three sources of data suggested that the TA providers delivered TA that actively involved participants and was contextualised and tailored to each region’s needs. Figure 2 summarises the data from the listening sessions, which were coded for the number of times that co-creation essential functions were mentioned or described. Facilitation was mentioned most frequently for regions B and C, while tailored support was mentioned most frequently by participants in region A. Co-learning was the second most mentioned category overall, after facilitation. Table 3 provides an example from the data of each of the co-creation essential functions as reported by participants during the listening sessions.

Figure 2:
Figure 2:

Mentions of co-creation essential functions during listening sessions

Citation: Evidence & Policy 15, 3; 10.1332/174426419X15468578679853

Table 3:

Examples of co-creation from participant listening sessions

Co-creation functionExample from data
Co-learning[TA facilitator] was an excellent listener. He really tried to glean from us what we needed as a region and took our suggestions and integrated them into the sessions that we had. I think he was able to take what we said and turn it around into what that needed to look like in a webinar series and as a result I really think it was beneficial.
BrokeringUtilising this process has really strengthened the relationship between the regional office staff and the T/TA staff. T/TA has always been an integral part but now it seems that relationship has grown stronger through this process. It is much more deliberate.
FacilitationWe as a group will sit and talk and talk together and identify where we want to go and we keep talking and we’re not sure where we want to go and the IS team listens and after they listen intently for a while they will say what I think we heard you say was and then they distill it and summarise it and package it for us so that we can see clearly how to move forward.
Address power differentialsThey did a good job of honouring our expertise and experiences and not coming in with their own expert model. They know what’s best but they honoured where we were in the process and met us where we were but also being very gentle to move us along and also keep pushing it so we didn’t stay there….
Co-designWe have an assignment, we do it, we meet and we discuss, we tweak, we meet again, it’s that cycle.
Tailored supportShort of coming out and doing the training in person which to be honest I’m not sure would have been more effective I think the short bursts of information were good…. We didn’t want too many but I think our team really responded well to it. And even the spacing we did a month apart I think that was good too.

Finally, the TA data were examined descriptively for initial evidence that interim outcomes specified by the TA logic model were achieved. Figure 3 presents survey data, which demonstrate that participants generally agreed or strongly agreed that trust was established, there was mutual accountability in the work, and integration of implementation science was beneficial to regional work. There was less strong agreement that implementation science concepts were integrated into regional products and processes.

Figure 3:
Figure 3:

Region Perceptions of Outcomes (n = 37)

Citation: Evidence & Policy 15, 3; 10.1332/174426419X15468578679853

Data from the listening sessions supported the achievement of interim outcomes. Several participants described the respectful interactions between TA providers and participants that created trust. For example, one participant described the TA providers as follows:

What I appreciated was they were not condescending in any way shape or form, always very positive, very supportive, and knew and appreciated how hard we wanted to succeed at this and they were right there beside us every step of the way.

Regarding mutual accountability, several participants suggested that the TA providers and recipients shared responsibility for achieving the goals set forth for the TA. As one expressed:

I think one of the things that I appreciated just reflecting on it was we became a team together. It was not so much division with what our responsibilities were but we all were accountable and I felt [TA providers] were our cheerleaders in a lot of sense so I think it really helped us all to organise ourselves and understand each responsibility and role we had to support one another in this process and that was very meaningful for me.

The qualitative listening session data also supported the notion that integrating implementation science concepts into the regional products and processes was challenging. Several participants noted that there were challenges related to connecting implementation science concepts to their actual work. As one participant noted:

It was good to be using this as the process but I don’t know if I can pinpoint it in Implementation Science, ‘this is where we are’. I couldn’t draw those direct connections. However using IS was very effective. However I don’t know if I could pull it out the same stages or replicate that process and apply it to my work without the input of the team.

Discussion

This small-scale study provides initial evidence that using a co-creation approach in the delivery of TA is achievable and can be associated with positive outcomes. Using existing data from multiple sources, including observations, surveys, and listening sessions, the analyses showed that the TA providers succeeded in delivering TA that was perceived by both observers and participants as involving participants and being tailored to their settings and needs. Evidence also suggested that the interim outcomes of trust, mutual accountability, and some integration of implementation science concepts into ongoing regional work were achieved.

Previous research has found that in general, higher doses of TA have been related to greater improvements in capacity and implementation (Leeman et al, 2015). In the current examination, the frequency and dosage of service provided was relatively low overall and was particularly low for region A, which requested virtual TA. Even with this low dosage, a co-creation TA approach was successfully delivered and was related to interim outcomes. The study is small, and additional evidence is needed; nevertheless, as funds for professional development have become more limited, finding effective virtual delivery mechanisms that do not rely solely on didactic approaches will be critical. A co-creation approach that is tailored to recipients’ needs shows promise as one that works for both in-person and virtual delivery of TA, and should be further explored for evidence of effectiveness across providers and domains.

As noted previously, evidence suggests that much TA is provided without consideration of a conceptual model or framework (Katz and Wandersman, 2016; Le et al, 2016). This study fills a need in the literature by examining TA that was delivered within the confines of a specific framework, namely co-creation. It also provides a specific definition of co-creation functions (Voorberg et al, 2014) that corresponds to competencies needed to facilitate use of implementation strategies (Berta et al, 2015; Bornbaum et al, 2015). Within this co-creation framework, evidence suggests that the three different regions in this analysis may have experienced different levels of each of the co-creation components, based on mentions during listening sessions and coding of observation data. Since each region identified different goals in collaboration with the TA consultants, the mentions of specific co-creation activities would not be expected to be the same across regions. Rather, the mentions for specific co-creation activities highlights the specific focal areas of the work with each region and the tailoring of services to needs. For example, TA consultants working with Region C focused their efforts largely on facilitating a process with the Region C implementation team leadership, so it is not surprising that facilitation was mentioned more often than the other categories of co-creation. In comparison, Region A work was delivered virtually via webinar and therefore included a larger proportion of didactic content relative to the other regions; that facilitation was mentioned less often aligns with the focus of the activities that were conducted with Region A.

As noted in the background section, the three regions that participated in this activity all expressed interest in and readiness for engaging in TA around implementation science. That this co-creation approach to TA was successful in terms of implementation and interim outcomes may be an artifact of the specific regions that participated. The descriptive analyses are not causal, and no claims regarding direct links between the TA approach and interim outcomes can be made. In addition, the analyses were conducted with a very small number of participants. This work should be considered a pilot, descriptive examination of a promising approach to providing TA.

Although additional research and evaluation is needed with larger sample sizes to build the knowledge base on the relations between co-creation TA approaches and desired outcomes, several implications for TA providers and researchers are evident. First, TA providers should consider recipient readiness as they determine dosage and delivery mechanisms. The three regions that participated in this pilot expressed interest in receiving co-creative TA on this subject. Their interest may have influenced the regions’ willingness to engage virtually and as partners with the TA providers. This context is different than in other programmes, where TA is mandated for compliance or identified as necessary for corrective action, thus not approached co-creatively. Second, TA providers can use the essential functions and activities from this pilot as a roadmap for developing their own skills and competencies in co-creative work with recipients. National centres who wish to move beyond building recipient knowledge to changing practice may benefit from shifting their own approach to one of co-creation. Third, while progress toward interim outcomes was measured, the findings show that application of implementation science remained challenging. This suggests that ongoing support and more time may be needed for co-creation efforts to leverage the progress measured in service to sustained practice change. Fourth, this descriptive paper relied on secondary analysis of information gathered to inform and improve TA efforts. Research on TA is still in its infancy (Le et al, 2016) and could be advanced through larger prospective studies that assess the relations between TA strategies, core components, and interim and long-term outcomes. This descriptive paper provides insight into a co-creative approach for TA and interim outcomes that could be used in future studies of technical assistance. Finally, the use of mixed-methods to understand the co-creative TA approach employed in this study provided further insight into recipients’ perspectives on the value and benefit of co-creation strategies from multiple perspectives. It may be beneficial for future research to use such methods in order to deepen our understanding of effective co-creative TA and associated outcomes.

Funding

This work was supported by the Administration for Children and Families under Grant # 201709277N through a subcontract from ZERO TO THREE. Opinions or points of view expressed in this paper represent a consensus of the authors and do not necessarily represent the official position or policies of the Administration for Children and Families, US Department of Health and Human Services.

Acknowledgements

The authors gratefully acknowledge the contributions of the Head Start regional office staff, including the early childhood, programme, fiscal, and regional TA specialists and coordinators, who contributed their time and thoughts to inform the TA approach described in this paper, and whose data were used for the secondary analysis reported here.

Conflict of interest

The authors declare that there is no conflict of interest.

References

  • Becker, DR, Lynde, D, Swanson, SJ, 2008, Strategies for state-wide implementation of supported empoyment: the Johnson and Johnson–Dartmouth Community Mental Health Program, Psychiatric Rehabilitation Journal 31, 4, 2969

    • Search Google Scholar
    • Export Citation
  • Berta, W, Cranley, L, Dearing, JW, Dogherty, EJ, Squires, JE, Estabrooks, CA, 2015, Why, we think, facilitation works: insights from organizational learning theory, Implementation Science 10, 141

    • Search Google Scholar
    • Export Citation
  • Bornbaum, CC, Kornas K, Peirson L, Rosella, LC, 2015, Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: a systematic review and thematic analysis, Implementation Science 10, 162

    • Search Google Scholar
    • Export Citation
  • Chilenski, SM, Perkins, DF, Olson, J, Hoffman, L, Feinberg, ME, Greenberg, M, Welsh, J, Crowley, DM, Spoth, R, 2016, The power of a collaborative relationship between technical assistance providers and community prevention teams: a correlational and longitudinal study, Evaluation and Program Planning 54, 1929

    • Search Google Scholar
    • Export Citation
  • Eccles, MP, Mittman, BS, 2006, Welcome to Implementation Science, Implementation Science 1, 1

  • Feinberg, ME, Ridenour, TA, Greenberg, MT, 2008, The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania, Journal of Primary Prevention 29, 14565

    • Search Google Scholar
    • Export Citation
  • Fixsen, DL, Naoom, SF, Blase, KA, Friedman, RM, Wallace, F, 2005, Implementation research: a synthesis of the literature, Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network, FMHI Publication #231

    • Search Google Scholar
    • Export Citation
  • Horner, R, Blitz, C, Ross, S, 2014, The importance of contextual fit when implementing evidence-based interventions, ASPE Research Brief, Washington, DC: US Department of Health and Human Services

    • Search Google Scholar
    • Export Citation
  • Kaner, S, 2014, Facilitators guide to participatory decision-making (3rd edn), San Francisco, CA: Jossey-Bass

  • Katz, J, Wandersman, A, 2016, Technical assistance to enhance prevention capacity: a research synthesis of the evidence base, Prevention Science 17, 41728

    • Search Google Scholar
    • Export Citation
  • Le, LT, Anthony, BJ, Bronheim, SM, Holland, CM, Perry, DF, 2016, A technical assistance model of guiding service and system change, Journal of Behavioral Health Services and Research 43, 38095

    • Search Google Scholar
    • Export Citation
  • Leeman, J, Calancie, L, Hartman, MA, Escoffery, CT, Herrmann, AK, Tague, LE, Moore, AA, Wilson, KM, Schreiner, M, Samuel-Hodge, C, 2015, What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review, Implementation Science 10, 80

    • Search Google Scholar
    • Export Citation
  • Metz, A, Albers, B, 2014, What does it take? How federal initiatives can support implementation of evidence-based programs to improve outcomes for adolescents, Journal of Adolescent Health 54, 5926

    • Search Google Scholar
    • Export Citation
  • Metz, A, Bartley, L, 2015, Co-creating the conditions to sustain the use of research evidence in public child welfare, Child Welfare 94, 11539

    • Search Google Scholar
    • Export Citation
  • Metz, A, Louison, L, Ward, C, Burke, K, 2017, Active implementation practitioner profile, Chapel Hill, NC: University of North Carolina, National Implementation Research Network

    • Search Google Scholar
    • Export Citation
  • National Center on Early Childhood Development, Teaching, and Learning, 2018, Early Childhood Learning and Knowledge Center, viewed 17 November 2018, https://eclkc.ohs.acf.hhs.gov/ncecdtl

  • Powell, BJ, Waltz, TJ, Chinman, MJ, Damschroder, LJ, Smith, JL, Matthieu, MM, Proctor, EK, Kirchner, JE, 2015, A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change, ERIC, project, Implementation Science 10, 21

    • Search Google Scholar
    • Export Citation
  • Van Kerkhoff, L, Lebel, L, 2015, Coproductive capacities: rethinking science-governance relations in a diverse world, Ecology and Society 20, 1, 14

    • Search Google Scholar
    • Export Citation
  • Vargo, SL, Lusch, RF, 2004, Evolving to a new dominant logic for marketing, Journal of Marketing 68, 1, 117

  • Voorberg, WH, Bekkers, VJJM, Tummers, LG, 2014, A systematic review of co-creation and co-production: embarking on the social innovation journey, Public Management Review 17, 133357

    • Search Google Scholar
    • Export Citation
  • Wandersman, A, Chien, VH, Katz, J, 2012, Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance / quality improvement, American Journal of Community Psychology 50, 44559

    • Search Google Scholar
    • Export Citation
  • View in gallery

    Theory of change for co-creative technical assistance with an implementation science perspective

  • View in gallery

    Mentions of co-creation essential functions during listening sessions

  • View in gallery

    Region Perceptions of Outcomes (n = 37)

  • Becker, DR, Lynde, D, Swanson, SJ, 2008, Strategies for state-wide implementation of supported empoyment: the Johnson and Johnson–Dartmouth Community Mental Health Program, Psychiatric Rehabilitation Journal 31, 4, 2969

    • Search Google Scholar
    • Export Citation
  • Berta, W, Cranley, L, Dearing, JW, Dogherty, EJ, Squires, JE, Estabrooks, CA, 2015, Why, we think, facilitation works: insights from organizational learning theory, Implementation Science 10, 141

    • Search Google Scholar
    • Export Citation
  • Bornbaum, CC, Kornas K, Peirson L, Rosella, LC, 2015, Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: a systematic review and thematic analysis, Implementation Science 10, 162

    • Search Google Scholar
    • Export Citation
  • Chilenski, SM, Perkins, DF, Olson, J, Hoffman, L, Feinberg, ME, Greenberg, M, Welsh, J, Crowley, DM, Spoth, R, 2016, The power of a collaborative relationship between technical assistance providers and community prevention teams: a correlational and longitudinal study, Evaluation and Program Planning 54, 1929

    • Search Google Scholar
    • Export Citation
  • Eccles, MP, Mittman, BS, 2006, Welcome to Implementation Science, Implementation Science 1, 1

  • Feinberg, ME, Ridenour, TA, Greenberg, MT, 2008, The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania, Journal of Primary Prevention 29, 14565

    • Search Google Scholar
    • Export Citation
  • Fixsen, DL, Naoom, SF, Blase, KA, Friedman, RM, Wallace, F, 2005, Implementation research: a synthesis of the literature, Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network, FMHI Publication #231

    • Search Google Scholar
    • Export Citation
  • Horner, R, Blitz, C, Ross, S, 2014, The importance of contextual fit when implementing evidence-based interventions, ASPE Research Brief, Washington, DC: US Department of Health and Human Services

    • Search Google Scholar
    • Export Citation
  • Kaner, S, 2014, Facilitators guide to participatory decision-making (3rd edn), San Francisco, CA: Jossey-Bass

  • Katz, J, Wandersman, A, 2016, Technical assistance to enhance prevention capacity: a research synthesis of the evidence base, Prevention Science 17, 41728

    • Search Google Scholar
    • Export Citation
  • Le, LT, Anthony, BJ, Bronheim, SM, Holland, CM, Perry, DF, 2016, A technical assistance model of guiding service and system change, Journal of Behavioral Health Services and Research 43, 38095

    • Search Google Scholar
    • Export Citation
  • Leeman, J, Calancie, L, Hartman, MA, Escoffery, CT, Herrmann, AK, Tague, LE, Moore, AA, Wilson, KM, Schreiner, M, Samuel-Hodge, C, 2015, What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review, Implementation Science 10, 80

    • Search Google Scholar
    • Export Citation
  • Metz, A, Albers, B, 2014, What does it take? How federal initiatives can support implementation of evidence-based programs to improve outcomes for adolescents, Journal of Adolescent Health 54, 5926

    • Search Google Scholar
    • Export Citation
  • Metz, A, Bartley, L, 2015, Co-creating the conditions to sustain the use of research evidence in public child welfare, Child Welfare 94, 11539

    • Search Google Scholar
    • Export Citation
  • Metz, A, Louison, L, Ward, C, Burke, K, 2017, Active implementation practitioner profile, Chapel Hill, NC: University of North Carolina, National Implementation Research Network

    • Search Google Scholar
    • Export Citation
  • National Center on Early Childhood Development, Teaching, and Learning, 2018, Early Childhood Learning and Knowledge Center, viewed 17 November 2018, https://eclkc.ohs.acf.hhs.gov/ncecdtl

  • Powell, BJ, Waltz, TJ, Chinman, MJ, Damschroder, LJ, Smith, JL, Matthieu, MM, Proctor, EK, Kirchner, JE, 2015, A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change, ERIC, project, Implementation Science 10, 21

    • Search Google Scholar
    • Export Citation
  • Van Kerkhoff, L, Lebel, L, 2015, Coproductive capacities: rethinking science-governance relations in a diverse world, Ecology and Society 20, 1, 14

    • Search Google Scholar
    • Export Citation
  • Vargo, SL, Lusch, RF, 2004, Evolving to a new dominant logic for marketing, Journal of Marketing 68, 1, 117

  • Voorberg, WH, Bekkers, VJJM, Tummers, LG, 2014, A systematic review of co-creation and co-production: embarking on the social innovation journey, Public Management Review 17, 133357

    • Search Google Scholar
    • Export Citation
  • Wandersman, A, Chien, VH, Katz, J, 2012, Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance / quality improvement, American Journal of Community Psychology 50, 44559

    • Search Google Scholar
    • Export Citation

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 102 102 102
PDF Downloads 49 49 49

Altmetrics

Dimensions