key messages

• A co-creative technical assistance approach was delivered in three Head Start regions.

• Interim outcomes of trust and mutual accountability were achieved.

• The small-scale analysis highlights a promising approach to providing technical assistance.

Background

A large national technical assistance (TA) centre in the US provides TA to Head Start Regions to strengthen their use of implementation science in the application of evidence-based practices to improve outcomes for children and families served by Head Start programmes. Implementation science has been defined as the study and application of methods and processes designed to support achievement of a programme or activity that has clearly defined components (Eccles and Mittman, 2006; Fixsen et al, 2005). The goal of the TA examined here was ‘to advance best practices in the identification, development, and promotion of the implementation of evidence-based development and teaching and learning practices that are culturally and linguistically responsive and lead to positive child outcomes across early childhood programs’ (National Center on Early Childhood Development, Teaching, and Learning, 2018). In administration of TA to regions, the centre has used a ‘co-creation’ model. Co-creation is the active involvement of stakeholders in all stages of the production and implementation process resulting in service models, approaches, and practices that are contextualised and tailored to settings (Metz and Bartley, 2015; Vargo and Lusch, 2004). Over a two-year period, three Head Start regions identified a set of improvement objectives, and the national centre used a responsive, co-creation TA approach to support the regions in meeting their objectives. The purpose of this analysis of secondary data was to describe the co-creative TA approach taken and interim outcomes achieved.

TA approaches can vary along dimensions related to dosage, delivery methods, and levels of collaboration (Wandersman et al, 2012). Knowledge of best practices for the provision of TA is at an early stage, and evidence suggests that most TA is delivered without consideration of an organising framework or conceptual model (Katz and Wandersman, 2016; Le et al, 2016). Generally, a higher dose of TA has been found to relate to greater improvements in capacity and/or implementation (Leeman et al, 2015). Both in-person and virtual TA can be effective, but generally on-site technical assistance, compared to TA that is telephone- or email-based, has been shown to be more likely to lead to chances for experiential learning and demonstration of skills (Becker et al, 2008; Feinberg et al, 2008). The quality of the collaborative relationship between TA providers and participants has emerged as critical to successful implementation of practices (Chilenski et al, 2016).

Co-creation approaches to providing TA have emerged as an important component of effective and sustainable implementation capacity building (Metz and Albers, 2014). Through co-creation, recipients and beneficiaries of TA are actively involved in all stages of planning and implementing the TA resulting in models, approaches, and practices that are contextualised and tailored to their settings (Metz and Bartley, 2015; Vargo and Lusch, 2004). The goal of contextualisation is to ensure there is a match between TA and the values, needs, skills, and resources of those delivering interventions, systems stakeholders, and service beneficiaries (Horner et al, 2014; Van Kerkhoff and Lebel, 2015). A recent systematic review of the academic literature on co-creation and co-production in the public sector suggested a need for greater specificity in defining co-creation and more examination of outcomes (Voorberg et al, 2014).

The co-creative TA approach examined in this paper was designed to strengthen the integration and application of implementation science concepts in the regions in order to increase capacity for the service and systems changes needed to enable the use of evidence-based approaches. There is increasing attention in the literature to specific competencies needed to facilitate implementation and evidence use (Berta et al, 2015). The co-creative TA approach for the current project relied on a model of change that specified essential functions and interim and project-end outcomes (see Figure 1). The essential functions that supported co-creation included co-learning, brokering, facilitation, addressing power differentials, co-design, and tailored support, with each of these defined as part of a practice profile for the implementation specialists delivering the TA (Metz et al, 2017; see Table 1).

View Full Size

Theory of change for co-creative technical assistance with an implementation science perspective

Citation: Evidence & Policy 15, 3; 10.1332/174426419X15468578679853

• Download Figure
• Download figure as PowerPoint slide

The three regions that participated in the project began with a strong interest and readiness to engage the TA staff to help build their implementation capacity. Each region identified a core team that would participate in TA activities and communicate and coordinate with regional staff beyond the core team. Each region also identified one or two specific areas of need for TA support that were individualised for their region. One region (Region A) expressed an interest in a virtual webinar series that would most efficiently engage their core and broader staff in learning about implementation science principles, while the other two (Regions B and C) each expressed interest in a more intensive TA approach that would help to further assess specific needs within their broader interest of incorporating implementation science principles into their grantee support processes.

From the beginning of the process with the regions, the TA team used the co-creative approach by first discussing roles and expectations with each region using a ‘Give and Get’ activity that attempted to set the stage for shared expectations and clear roles. This was designed to allow each region to arrive at a shared agreement with the TA staff on what regional staff would contribute to the TA process (through participation and development work), and what to expect from the TA staff (through guidance and facilitation). The TA staff also used initial meetings to assess current capacity with the regions, and facilitated discussions on identifying next steps in their work. These specific activities were not prescribed but emerged out of a co-creative process with each region.

Over the two-year period, the TA staff met in-person and virtually and communicated electronically with regional staff in the provision of TA. Each meeting and interaction, and the frequency with which they occurred, was tailored to the region’s needs and goals. For example, in-person meetings were scheduled when the region felt as though they would benefit most from the in-person support. Tailored support can be contrasted with traditional TA models that prescribe the dosage of support prior to the engagement with the recipient. This paper summarises findings related to whether there was evidence that (1) participants experienced the TA as co-creative, and (2) interim outcomes were achieved.

Methods

TA was provided to the regions over a two-year period (June 2016–June 2018). The data gathered for purposes of informing the TA were analysed as secondary data for this article. This section describes the TA participants, data collection procedures, measures, and analytic approach.

Findings

The data were analysed descriptively to understand how much TA was provided. Table 2 provides information on level of service, including both frequency of contact and number of hours of TA provided. Across regions, the average number of events was about 12 over the course of two years, and the average number of total hours was about 23. Region A received a lower dose, which matched their request for a virtual workshop series rather than the more intensive TA requested by regions B and C.

The three sources of data – observation, survey, and listening session – were summarised to determine whether the TA providers succeeded in delivering TA that was co-creative. Observers of the events coded the activities during the workshops; survey and listening session data provided participant perceptions of the TA. All three sources of data suggested that the TA providers delivered TA that actively involved participants and was contextualised and tailored to each region’s needs. Figure 2 summarises the data from the listening sessions, which were coded for the number of times that co-creation essential functions were mentioned or described. Facilitation was mentioned most frequently for regions B and C, while tailored support was mentioned most frequently by participants in region A. Co-learning was the second most mentioned category overall, after facilitation. Table 3 provides an example from the data of each of the co-creation essential functions as reported by participants during the listening sessions.

View Full Size

Mentions of co-creation essential functions during listening sessions

Citation: Evidence & Policy 15, 3; 10.1332/174426419X15468578679853

• Download Figure
• Download figure as PowerPoint slide

Finally, the TA data were examined descriptively for initial evidence that interim outcomes specified by the TA logic model were achieved. Figure 3 presents survey data, which demonstrate that participants generally agreed or strongly agreed that trust was established, there was mutual accountability in the work, and integration of implementation science was beneficial to regional work. There was less strong agreement that implementation science concepts were integrated into regional products and processes.

View Full Size

Region Perceptions of Outcomes (n = 37)

Citation: Evidence & Policy 15, 3; 10.1332/174426419X15468578679853

• Download Figure
• Download figure as PowerPoint slide

Discussion

This small-scale study provides initial evidence that using a co-creation approach in the delivery of TA is achievable and can be associated with positive outcomes. Using existing data from multiple sources, including observations, surveys, and listening sessions, the analyses showed that the TA providers succeeded in delivering TA that was perceived by both observers and participants as involving participants and being tailored to their settings and needs. Evidence also suggested that the interim outcomes of trust, mutual accountability, and some integration of implementation science concepts into ongoing regional work were achieved.

Previous research has found that in general, higher doses of TA have been related to greater improvements in capacity and implementation (Leeman et al, 2015). In the current examination, the frequency and dosage of service provided was relatively low overall and was particularly low for region A, which requested virtual TA. Even with this low dosage, a co-creation TA approach was successfully delivered and was related to interim outcomes. The study is small, and additional evidence is needed; nevertheless, as funds for professional development have become more limited, finding effective virtual delivery mechanisms that do not rely solely on didactic approaches will be critical. A co-creation approach that is tailored to recipients’ needs shows promise as one that works for both in-person and virtual delivery of TA, and should be further explored for evidence of effectiveness across providers and domains.

As noted previously, evidence suggests that much TA is provided without consideration of a conceptual model or framework (Katz and Wandersman, 2016; Le et al, 2016). This study fills a need in the literature by examining TA that was delivered within the confines of a specific framework, namely co-creation. It also provides a specific definition of co-creation functions (Voorberg et al, 2014) that corresponds to competencies needed to facilitate use of implementation strategies (Berta et al, 2015; Bornbaum et al, 2015). Within this co-creation framework, evidence suggests that the three different regions in this analysis may have experienced different levels of each of the co-creation components, based on mentions during listening sessions and coding of observation data. Since each region identified different goals in collaboration with the TA consultants, the mentions of specific co-creation activities would not be expected to be the same across regions. Rather, the mentions for specific co-creation activities highlights the specific focal areas of the work with each region and the tailoring of services to needs. For example, TA consultants working with Region C focused their efforts largely on facilitating a process with the Region C implementation team leadership, so it is not surprising that facilitation was mentioned more often than the other categories of co-creation. In comparison, Region A work was delivered virtually via webinar and therefore included a larger proportion of didactic content relative to the other regions; that facilitation was mentioned less often aligns with the focus of the activities that were conducted with Region A.

As noted in the background section, the three regions that participated in this activity all expressed interest in and readiness for engaging in TA around implementation science. That this co-creation approach to TA was successful in terms of implementation and interim outcomes may be an artifact of the specific regions that participated. The descriptive analyses are not causal, and no claims regarding direct links between the TA approach and interim outcomes can be made. In addition, the analyses were conducted with a very small number of participants. This work should be considered a pilot, descriptive examination of a promising approach to providing TA.

Although additional research and evaluation is needed with larger sample sizes to build the knowledge base on the relations between co-creation TA approaches and desired outcomes, several implications for TA providers and researchers are evident. First, TA providers should consider recipient readiness as they determine dosage and delivery mechanisms. The three regions that participated in this pilot expressed interest in receiving co-creative TA on this subject. Their interest may have influenced the regions’ willingness to engage virtually and as partners with the TA providers. This context is different than in other programmes, where TA is mandated for compliance or identified as necessary for corrective action, thus not approached co-creatively. Second, TA providers can use the essential functions and activities from this pilot as a roadmap for developing their own skills and competencies in co-creative work with recipients. National centres who wish to move beyond building recipient knowledge to changing practice may benefit from shifting their own approach to one of co-creation. Third, while progress toward interim outcomes was measured, the findings show that application of implementation science remained challenging. This suggests that ongoing support and more time may be needed for co-creation efforts to leverage the progress measured in service to sustained practice change. Fourth, this descriptive paper relied on secondary analysis of information gathered to inform and improve TA efforts. Research on TA is still in its infancy (Le et al, 2016) and could be advanced through larger prospective studies that assess the relations between TA strategies, core components, and interim and long-term outcomes. This descriptive paper provides insight into a co-creative approach for TA and interim outcomes that could be used in future studies of technical assistance. Finally, the use of mixed-methods to understand the co-creative TA approach employed in this study provided further insight into recipients’ perspectives on the value and benefit of co-creation strategies from multiple perspectives. It may be beneficial for future research to use such methods in order to deepen our understanding of effective co-creative TA and associated outcomes.

Funding

This work was supported by the Administration for Children and Families under Grant # 201709277N through a subcontract from ZERO TO THREE. Opinions or points of view expressed in this paper represent a consensus of the authors and do not necessarily represent the official position or policies of the Administration for Children and Families, US Department of Health and Human Services.

Acknowledgements

The authors gratefully acknowledge the contributions of the Head Start regional office staff, including the early childhood, programme, fiscal, and regional TA specialists and coordinators, who contributed their time and thoughts to inform the TA approach described in this paper, and whose data were used for the secondary analysis reported here.

Conflict of interest

The authors declare that there is no conflict of interest.