How does policy learning take place across a multilevel governance architecture during crises?

Authors:
Bishoy Louis ZakiUniversity of Ghent, Belgium

Search for other papers by Bishoy Louis Zaki in
Current site
Google Scholar
Close
and
Ellen WayenbergUniversity of Ghent, Belgium

Search for other papers by Ellen Wayenberg in
Current site
Google Scholar
Close
Full Access
Get eTOC alerts
Rights and permissions Cite this article

Policy learning plays a significant role in shaping policy during crises. While scholarship has explored many of the mechanisms and outcomes of such learning, little is known about how policy learning takes place across different levels of a multilevel governance architecture. This is despite their prevalence and influence on crisis responses. Using a case of the Belgian COVID-19 policy response, we explore how policy learning takes place across different levels of multilevel governance within creeping crises, focusing on epistemic policy learning (learning from experts) as one of the most pronounced learning types within such contexts. By means of document analysis, supplemented by primary source data from expert and senior official interviews, we offer an exploratory account of how learning took place at the national and subnational levels. Our findings reveal how the inherent features of the COVID-19 crisis, and the existing multilevel governance architecture broke the policy learning process into smaller heterogenous learning processes at different levels. We find that decentralised approaches to learning provided the space for customised, yet often fragmented policy responses. We also find that institutional legacies, varying degrees of policymaker control over learning, and absence of common approaches to structuring and designing learning processes led policymakers in different jurisdictions to engage in varying policy learning processes. We take stock of these different learning processes and highlight their key features. We conclude by highlighting the implications of these findings for policy learning theory and practice.

Abstract

Policy learning plays a significant role in shaping policy during crises. While scholarship has explored many of the mechanisms and outcomes of such learning, little is known about how policy learning takes place across different levels of a multilevel governance architecture. This is despite their prevalence and influence on crisis responses. Using a case of the Belgian COVID-19 policy response, we explore how policy learning takes place across different levels of multilevel governance within creeping crises, focusing on epistemic policy learning (learning from experts) as one of the most pronounced learning types within such contexts. By means of document analysis, supplemented by primary source data from expert and senior official interviews, we offer an exploratory account of how learning took place at the national and subnational levels. Our findings reveal how the inherent features of the COVID-19 crisis, and the existing multilevel governance architecture broke the policy learning process into smaller heterogenous learning processes at different levels. We find that decentralised approaches to learning provided the space for customised, yet often fragmented policy responses. We also find that institutional legacies, varying degrees of policymaker control over learning, and absence of common approaches to structuring and designing learning processes led policymakers in different jurisdictions to engage in varying policy learning processes. We take stock of these different learning processes and highlight their key features. We conclude by highlighting the implications of these findings for policy learning theory and practice.

Introduction

In public policy, learning can be understood as the process by which policy actors interact with policy related knowledge and information to update their understandings and beliefs regarding policy issues and gain insights into potential policy adjustments (Zaki et al, 2022). Accordingly, policy learning is often leveraged in policy analysis to explain the how and why of policy change, or the lack thereof (for example, Bennett and Howlett, 1992; Dunlop and Radaelli, 2017).

During crises, policy learning can play a substantial role, as crises can serve as focusing events, or urgent technically complex situations that – to be effectively prepared for or handled – require or offer opportunities for learning. Hence, learning processes taking place around crises help explain variations in policymaking trajectories and thus outcomes (for example, Zaki and Wayenberg, 2021; Zaki et al, 2022a). This resulted in two main strands of crisis learning research emerging: inter-crisis and intra-crisis learning (see Kamkhaji and Radaelli, 2017). Inter-crisis (that is, post-crisis) learning focuses on the retrospective production of lessons, their types, depth and degrees of institutionalisation (for example, Raudla et al, 2018). Intra-crisis learning mainly focuses on how policy learning processes interact with the crisis context to produce different types of lessons as crisis episodes unfold (for example, Kamkhaji and Radaelli, 2017; Zaki et al, 2022a).

While the importance of inter-crisis learning has been long-established, understanding of intra-crisis learning processes is increasingly pressing, particularly with the emergence of ‘creeping crises’ (for example, the COVID-19 pandemic, migration, climate change, political and military tensions). These are lingering crises that shape the policymaking context for extended periods (Boin et al, 2020). There, policy learning continues to inform actors on critical issues in real time over years, rather than days or weeks, unlike shorter-term crises such as natural disasters or ‘single-episode’ shock events (for example, Kamkhaji and Radaelli, 2017).

Intra-crisis learning research has explored various areas, including: how do organisational actors manage learning and adapt to crisis evolutions (for example, Lee et al, 2020), how do politico-administrative legacies, crisis-related knowledge, and information influence learning (for example, Zaki and Wayenberg, 2021), and how do different policy learning types shift and interact within certain contexts leading to policy change as policy issues evolve (Zaki et al, 2022a). However, policy learning does not occur in a vacuum. While it is understood as an interaction between policy issues, actors, knowledge and structures within a policymaking context, we know relatively little about interactions between policy learning and the governance structures in which it occurs (Bomberg, 2007). In particular, how policy learning takes place across multilevel governance structures, especially within creeping crisis contexts (see Mavrot and Sager, 2018; Neij and Heikkinen, 2021; Zaki et al, 2021).

Most recently, the technically complex nature of the COVID-19 crisis has emphasised the role of epistemic policy learning, that is, learning from experts (Dunlop and Radaelli, 2013; Zaki and George, 2021). It also highlighted the influence of multilevel governance structures on the agility of crisis responses and policy outcomes (for example, Pattyn et al, 2021; Maggetti, 2020). While these structures are often associated with higher agility and scalability, they are also associated with coordination and alignment challenges especially during crises (Maggetti and Trein, 2018; Kettl, 2020): an issue that was highly visible in the COVID-19 context (for example, Schomaker et al, 2021; Boin and Lodge, 2021; Capano and Lippi, 2021).

Consistently, learning is expected to take place across the different levels of the governance architecture. For example, within such crises, federal or central government officials can engage in policy learning (for example, from experts) through interactions with dedicated national level expert advisory groups and use their learning to issue national directives (see, for example, Wayenberg et al, 2022). This is highly expected, given that crisis governance often starts at the central level (for example, see Kuhlmann et al, 2021). However, given the crisis’ large scale and extended period, crisis governance and policymaking can gradually tilt towards the subnational levels (for example, see the arguments by Maggetti and Trein, 2018; Maggetti, 2020). There, subnational policy actors also have access to their own expert groups, as well the policymaking leeway to implement their own lessons learned. However, how learning takes place across these different levels, and how different approaches to learning affect the uptake of national directives and subnational policymaking, remains largely underexplored.

Pursuing this inquiry is valuable, both for research and practice. Within such creeping crises contexts (for example, the recent COVID-19 pandemic), policymakers engage in systematic policy learning processes, mostly focused on epistemic policy learning (see Dunlop and Radaelli, 2013; Baekkeskov and Öberg, 2016). The ‘intense’ and systematic nature of this process is driven by inherent crisis complexity, perpetuity and continuous policy issue evolutions which necessitate ongoing and structured learning (see Dunlop and Radaelli, 2013; Boin et al, 2020; Zaki et al, 2022a). This can be different from non-crisis contexts where there is relatively less pressure for learning, prompt action, and higher tolerance for faults or delayed responses (Van Dooren and Noordegraaf, 2020). Crisis complexity, urgency and severity often leads to a rapid and high uptake of learning outcomes into policymaking, often entailing substantial implications for policy outcomes. These policy learning processes are highly sensitive to the governance structures in which they occur (for example, Pattyn et al, 2021). Features of these structures (for example, decentralisation, coordination, fragmentation) can foster or inhibit learning, and shape its outcomes (Zaki et al, 2022). This is particularly as decentralised and multilevel governance architectures boast varying politico-administrative traditions, competences and degrees of control known to influence how learning takes place (for example, Dunlop and Radaelli, 2013; Di Giulio and Vecchi, 2018).

Put together: such creeping crises necessitate and amplify the role of epistemic policy learning; said learning happens within decentralised governance architectures that influence how learning takes place. However, little is known on how this happens. Hence, in this article, using the case of the COVID-19 crisis, we explore the hitherto under-researched, question of how does epistemic policy learning take place across different levels of the multilevel governance architecture during creeping crises? To answer this question, we trace one of the most omnipresent types of policy learning within such crises, that is epistemic policy learning or learning from experts (Dunlop and Radaelli, 2013; Zaki and George, 2021). In doing so, we do not imply that other types of learning do not exist (or are muted) within the COVID-19 crisis. Rather, given the exploratory nature of this research, and scarcity of existing research on the issue, we elect to focus a relatively more pronounced and traceable type of learning to enable more robust analysis.

To do so, we conduct an exploratory case study (Greenfield and Natalia, 2022), using an instrumental cross-unit case study approach, that is, identifying a case highly suitable for exploring the phenomenon of interest. We use the Belgian highly pronounced multilevel governance architecture as an empirical setting. Belgium boasts a range of politico administrative traditions, and ethno-linguistic cleavages that allows us to explore a range of valuable within-case variations (see Valcke et al, 2008; Pattyn et al, 2019; Pattyn et al, 2021). We source data from 46 official documents, and media coverage supplemented by 13 high-level interviews with policymakers and experts. Our analysis aims at giving an exploratory account of how policymakers approached epistemic policy learning across the three of the key governance levels involved in the Belgian COVID-19 crisis response: national (federal) and subnational (provincial and municipal).

The contribution of this article is threefold. Theoretically, while learning literature often strives to explain when and how learning leads to policy change, this relationship is often challenging to explain (Borrás, 2011). By exploring variations in how learning takes place across different governance levels, we heed the call for a more multidimensional understanding of how learning takes place, and its potential contribution to policy change (Dunlop and Radaelli, 2017). This is through creating a better account of the microfoundations of learning and the key elements involved in the learning process, that is, how actors approach learning within different governance structures, using varying information and knowledge sources (see Kamkhaji and Radaelli, 2017; Zaki et al, 2022). Understanding how different governance tiers learn, offers a more nuanced explanation of why and how they adjust their policy responses, and why policy trajectories at the subnational levels can diverge or converge. Second, empirically, by offering a novel account of an under-researched phenomenon, this exploratory case study provides future research with testable key factors that influence policy learning across levels of government Third, practically, we leverage our findings to reflect on how policymakers can enhance the design and management of policy learning frameworks within crisis contexts.

This article proceeds as follows: in the second section, we explore the nexus of policy learning and multilevel governance and establish the mode, type and mechanism of policy learning observable within the COVID-19 crisis. In the third section, the methodological and analytical frameworks are constructed. In the fourth section, we present our case. We offer our discussion and conclusions in the fifth and final section.

Creeping crises, policy learning and multilevel governance

The manner by which policy learning takes place is highly influenced by the context surrounding the learning process. As such, different parameters of the context (for example, urgency, shock, issue complexity) can lead to different mechanisms, modes and types of learning taking place. These varying ways by which learning happens entail different operationalisation and empirical identification schemes. Hence, before appraising learning within a certain case, it is necessary to theorise ex-ante how learning is expected within a certain context. This helps us to establish and exclude the null hypothesis (that is, the non-learning condition), and exclude potentially confounding learning-like phenomena (Radaelli, 2009; Zaki, Wayenberg and George, 2022). In this section we begin by demarcating the context of policy learning and consequently, what mechanisms and modes of learning it invokes. Then we establish the plausibility of these learning processes taking place across different tiers of the multilevel governance architecture.

The COVID-19 crisis: context, mechanisms and modes of learning

An inferential learning mechanism

Boin et al (2020) conceptualise COVID-19 as a creeping crisis. Policymakers can see these crises coming a mile away. However, they do not necessarily dedicate them sufficient early-on attention. This is in contrast to fast-burning or one-time shock crises that invoke concerted consensual urgent action. Creeping crises are transboundary. Hence, they are ‘fuzzy’, challenging to define, they permeate societal and sectorial boundaries (Zaki and George, 2021), and demonstrate variations in intensity, that is, ‘occasional sparking’. Consequently, they lead to divisive threat perceptions, and create space for political contestation (Zaki et al, 2022a). Temporally, these crises are epochal, and present a sense of permanence as they manifest over extended periods. Thus, they require policymakers to establish long-term crisis-specific policymaking paradigms or ‘new normals’, that is, cognitive scripts for understanding the world throughout extended crisis periods (Zaki et al, 2022a). With this context in mind, what policy learning mechanisms take place?

Learning literature draws distinctions between two mechanisms by which learning occurs: contingent and inferential (Kamkhaji and Radaelli, 2017). Contingent learning occurs where crises induce shock and urgency, pushing policymakers to act quickly aiming to avoid harm. They draw rapid ‘cue–outcome associations’ leading to significant policy change, yet without an inferential (critical) reflection on – or revisions of – beliefs and priors. Ceteris paribus, under this mechanism, policy change happens, then critical reflections on priors ensue. Inferential learning on the other hand, often first involves genuine critical reflections on priors leading to updates in policy beliefs then potentially followed by belief or policy changes (for example, Heclo, 1974; Bennett and Howlett, 1992; Kamkhaji and Radaelli, 2017). Inferential learning as such happens when policy actors consume and critically reflect on policy related information and knowledge, often aided by knowledge production mechanisms and sensemaking capacities, and the existence of learning structures (see Rietig and Perkins, 2017; Zaki et al, 2022b). Conversely, contingent learning is enabled by: absence of institutional and epistemic entrepreneurship, limited cognitive and sensemaking capacities, crisis induced shock, fragmentation, or lack of jurisdictional authority (see Kamkhaji and Radaelli, 2017).

Looking at a two-year horizon in our analysis, COVID-19 policy learning research has shown that the crisis provides the space for inferential learning to occur (for example, see Ladi and Tsarouhas, 2020; Zaki and Wayenberg, 2021). There was a clear presence of expert communities in different capacities acting as producers of scientific knowledge with access to policymakers through formalised learning structures (Pattyn et al, 2021; Zaki and George, 2021). There was also substantial institutional entrepreneurship proffered by local and international specialised bodies (for example, Sciensano,1 World Health Organization, European Medical Agency). Furthermore, the epochal nature and interspersing intensity of the creeping crisis has availed significant room for the deliberate and critical reflection within the policymaking process, thus at this level of analysis neutralising the one-time shock effect, enabling foresight and critical reflection (Zaki et al, 2022a).

Put together, within the purview of our case, this creates space for inferential, rather than contingent learning. However, this does not exclude that contingent learning mechanisms can occur within other contexts, for example, when looking at different levels, or timeframes of analysis, varying actor constructions of the crisis, and political motivated crisis responses (for example, Bergeron et al, 2020; Ladi and Tsarouhas, 2020). Next, we home in on which specific mode of inferential learning prominently features within this analytical context.

Learning mode: COVID-19 and epistemic policy learning

Dunlop and Radaelli (2013) identify four main modes of learning: hierarchical, epistemic, bargaining-oriented and reflexive. The functionality (that is, suitability) of these learning modes is determined by two features of the policy issue: actor certification (that is, existence of socially endorsed groups that can act as ‘problem solvers’) and issue tractability (that is, degree of issue uncertainty). Given that COVID-19 is an issue of high certification of actors, and low issue tractability, epistemic policy learning is positioned as a highly functional learning mode (Zaki and Wayenberg, 2021). Epistemic policy learning is viewed as learning from certified groups of experts with substantive policy-issue knowledge and access to policymakers (Haas, 1992). Epistemic learning is also not unimodal. Epistemic actors can play different roles with varying degrees of influence based on a dynamic relationship with the learners as: contributors, producers of standards, facilitators or teachers.2 These positions are also largely determined by a combination of two key factors, pertinent to learners’ control over learning content (that is, means), and learning objectives (that is, ends). For example, when policymakers (learners) have limited control over the objectives and means of learning (for example, facing technical complexity and public pressure), epistemic communities can take a more compelling ‘teacher’ position and hold significant sway in defining interests, outlining lessons, and charting potential solutions. In contrast, when policymakers have control over learning means and outcomes, epistemic actors can be relegated to ‘contributors’ as policymakers have more space to: define their own agendas, contest inputs and reshuffle epistemic communities when needed (for example, Dunlop, 2017a; Zaki and Wayenberg, 2021).

While epistemic policy learning can be enlightening, undertaking it is onerous. Misdirected learning can engender policy failure and derail policy (for example, Dunlop, 2017b). Lapses in epistemic policy learning leading to sub-par outcomes can be attributed to several factors, including: mismanaged relationships with epistemic communities (for example, Dunlop, 2017a), epistemic entrenchments, the identification of expertise, the influence of institutional legacies on management of policy learning processes (for example, Zaki and Wayenberg, 2021), and the politicisation of expertise (for example, Dunlop, 2017b).

So, we have a creeping crisis, that – within our analytical context – invokes inferential learning, and where epistemic policy learning processes are at the forefront of policymaking. Next, we explore the theoretical nexus of multilevel governance and epistemic policy learning within the COVID-19 crisis context.

Epistemic policy learning and multilevel governance

Literature explores how epistemic policy learning takes place from different perspectives, commonly temporal or policy-issue specific. An example of this is looking at how the roles of epistemic communities vary over time, as functions of: knowledge production and dissemination (for example, Dunlop, 2017), perturbations in the policymaking context, and evolving policy issue perceptions (for example, Zaki et al, 2022a; Zaki and Wayenberg, 2021). However, the spatial perspective on how policy learning (particularly epistemic) takes place, that is, its geographical and governance level dispersion, has been rarely explored (see Mavrot and Sager, 2018; Neij and Heiskanen, 2021). This is despite knowing that problem-solving through learning and multilevel governance arrangements are being inherently entwined (see Maggetti and Trein, 2018), and that policy learning processes cannot be understood in separation from the structures in which they take place (for example, Bomberg, 2007; Pattyn et al, 2021).

Multilevelled and decentralised governance architectures are globally prevalent and play central roles in directing crisis responses. They also shape (empower or impede) policy responses, particularly during large-scale crises where the need for continuous adaptation is key (for example, Boin and Lodge, 2021; Capano and Lippi, 2021; Bergström et al, 2022). Major crisis responses often start at the central level; however, as a crisis continues, they gradually shift into more decentralised forms (see Kuhlmann et al, 2021). This is as policymakers attempt responding to challenges beyond the full reach of central governments (Maggetti and Trein, 2018). Coordinated decentralisation can allow for swift and localised responses that account for local specificities (for example, Schomaker et al, 2021). However, decentralised systems have also been associated with a range of coordination challenges leading to fragmented and siloed crisis responses.

Yet why would we expect learning processes to show potentially diverging patterns or behaviours across different levels? Literature shows that learning processes are influenced (catalysed, impeded or just simply shaped) by key factors that naturally vary across different governance levels. For example, learners’ control over the learning process can change how they engage in learning, and how they interact with expert communities (for example, Dunlop and Radaelli, 2013). Also, learning capacities, politico administrative traditions and political legacies can affect approaches to learning (for example, Di Giulio and Vecchi, 2018; Zaki et al, 2022). This naturally becomes more pronounced in countries with wider degrees of ethno-linguistic and political traditions at the subnational levels, such as, for example, Belgium (see Zaki et al, 2022a).

So, we have earlier identified epistemic policy learning as a central learning type within the COVID-19 crisis context. Now, we highlight why within this context, epistemic policy learning processes plausibly occur at different levels of the governance architecture. We put forward two main logics pertinent to the nature of COVID-19 as a creeping crisis.

First, COVID-19 is a large-scale crisis. Unfolding for over two years, the COVID-19 crisis has an epochal and persistent nature. It is also transboundary, as such it cuts through different sectors and communities. It is socially and behaviourally moderated, its evolutions and outcomes are shaped by local norms and cultures (Cao et al, 2020). Hence, managing the crisis needs to sustain consistent and long-term engagement with community and region-specific features, such as those concerning trust in government, economic conditions and local capacities, among several others (for example, Capano and Lippi, 2021; Zaki et al, 2022c; 2022d), while harmonising policies with national strategies (see Bosa et al, 2021). Accordingly, subnational governments need to customise or ‘tailor’ responses to their regional and local contexts and capacities, often yielding potentially varying and context-sensitive approaches to managing the crisis.

Second, being a technically complex policy issue, the knowledge underlying crisis response is highly specialised, and often expert-driven. Accordingly, policymakers at different levels of the governance architecture need to employ professional ‘sensemakers’ to: 1) Interpret and elaborate on national crisis strategies 2) Customise these strategies to fit subnational contexts, especially in countries where significant subnational cultural and politico-administrative cleavages exist (for example, Valcke et al, 2008; Pattyn et al, 2021). This becomes particularly important as subnational governments attempt to utilise the ‘leeway’ on policy adjustment and implementation created by national–subnational coordination mechanisms (see Maggetti, 2020). Consistently, findings from multilevel governance research emphasise that – around technically complex issues – subnational policymakers construct their own localised sensemaking groups to understand, contextualise and implement national policies through engaging with epistemic actors (for example, Neij and Heiskanen, 2021). There, localised policy learning processes can provide knowledge ‘at key entry points’ and improve policy suitability (Mavrot and Sager, 2018; Di Giulio and Vecchi, 2018), by leveraging situated knowledge, leading to better sensemaking and problem-solving capacities (for example, Leino and Peltomaa, 2012). Put together, the plausibility of policy learning occurring across multiple levels of the governance architecture is evidenced in extant literature (for example, Di Giulio and Vecchi, 2018; Mavrot and Sager, 2018), burgeoning COVID-19 literature (for example, Casula and Pazos-Vidal, 2021; Mattei and Del Pino, 2021) as well as the interactions between subnational actors and expert groups at their respective levels.

Hence, given our analytical context, we have a situation where policymakers engage in inferential learning (particularly epistemic) at different levels of the governance architecture. In the next section, we construct our analytical and methodological frameworks to explore how said processes of epistemic policy learning take place across the national (federal) and subnational (provincial and local/municipal) levels using the case of the Belgian COVID-19 response.

Methodological framework

Case design

To conduct our exploratory case analysis, we employ an embedded case study design with multiple units of analysis. This enables an exploration of complex empirical cases (Nowell and Albrecht, 2018). Since we focus on multilevel governance structures, we use an ‘across and within-unit’ cross sectional approach. This allows us to examine variations of different subsystems’ analytical units within a common larger structure (Gerring 2004). As the case selection is instrumental, we identify a case best suited to offer insights into the phenomenon of interest. Selecting Belgium was motivated by three main logics: first, the country’s established (and often perceived as complex) multilevel governance setting with emphasised central–local relationships and its conventionally dense advisory system (Valcke et al, 2008; Fobé et al, 2013; Happaerts, 2015); second, Belgium’s relatively diverse, and geographically dispersed clusters of scientific expertise and advisory structures (Pattyn et al, 2019); third, the fact that Belgium’s initial COVID-19 response has been subject to controversy, particularly given issues in response coordination, epidemiological conditions, legitimacy and expertise (Popelier, 2020; Zaki and Wayenberg, 2021). This avails opportunities for tracing pathways between different approaches to epistemic policy learning, adjustments of policy trajectories, and policy outcomes. Our observational timeframe is the two-year period between January 2020 and December 2021. This allows us to trace variations in learning processes as the crisis response scheme has moved between being federal and decentralised (see Wayenberg et al, 2022). Hence, this gives us a better understanding of the consistency by which learning was approached throughout the crisis.

Data

We employ data source triangulation by collecting data from 46 official documents (including advisory group reports, documents from parliamentary depositions and public statements), and media coverage.3 The primary use of documentary analysis and desk research supplemented by interviews is a highly effective and recognised approach in the policy learning methodological tradition that enables the tracing of ‘how’ learning takes place (for example, Kamkhaji and Radaelli, 2017; Cairney, 2021; Zaki, Wayenberg and George, 2022). Hence, we supplement our analysis with 13 high-level key informant interviews (four provincial governor cabinets, two municipal cabinets, seven senior experts involved in policy advisory bodies, and a national institution leading the government’s response).4

Analysis

The precise phenomenon which we are interested in observing is how epistemic policy learning takes place across different levels of the governance architecture. In other words, how did different government levels learn or update their understandings of the policy issue through interacting with expert communities. To structure our analysis, we use a background conceptualisation of policy learning. Background conceptualisations allow us to capture key analytical dimensions. Along with Haas’s (1992) definition of epistemic policy learning, at a background level, we view policy learning as a process of continuous interaction between actors, systems and structures, around policy related information and knowledge within a policymaking context (Zaki et al, 2022). In our analysis, expert communities and epistemic actors are advisory groups (that is, those existing within learning structures) or experts affiliated with scientific groups (for example, universities, scientific and research institutes). These groups were abductively identified through both document analysis and interview data. In analysing data, we followed an abductive approach (see Tavory and Timmermans, 2012), focusing on our key analytical dimensions (process of learning, knowledge and information, structures, actors) across the three governance levels. In doing so, we present a thick description of the policy learning process as per the policy learning methodological tradition (see Zaki et al, 2022). In Table 1, we highlight how the background conceptualisation informs the analytical foci of our case.

Table 1:

Analytical foci based on Zaki et al (2022) and Haas (1992)

Learning process dimension Analytical focus
Actors Policymakers and expert groups at the federal and subnational levels
Information and knowledge Knowledge and informational content generated by different expert groups and policymakers across different levels
Systems and structures
  • Epistemic advisory structures (for example, crisis cells, expert groups and so on)

  • Governance structures (federal, provincial, and municipal)

Context COVID-19 as a creeping crisis, over two years from early 2020 to late 2021
Policy issue COVID-19 as a societally embedded, large-scale crisis.
Process of learning Key features of the process: systematic versus non-systematic

Interviews were anonymised and coded along with documents on NVivo using qualitative content analysis. Coding frames were abductively developed in close connection to the research design (see Nowell and Albrecht, 2018). Consistent with the background conceptualisation used, we primarily coded for the occurrence of epistemic policy learning, and the ‘how’ dimension which accounts for: advisory structures engaged, knowledge and information leveraged, key actors involved, and the patterns of information consumption (systematic/non-systematic). In our discussion, we then draw on two typologies of policy learning to position how the identified epistemic actors were involved in the learning process (Dunlop and Radaelli, 2013), and the disciplinarity of the underlying knowledge and informational content involved (Zaki and Wayenberg, 2021).

In the next section, we present our case study. We begin by highlighting the regional and local specificities underlying COVID-19 evolutions, then proceed to account for how epistemic policy learning took place across the national and subnational levels.

Case description

National situation and subnational variations

In Belgium COVID-19 had a drastic start. Initially, the country had one of the world’s highest infection and mortality rates per capita. The government partially attributed this to ‘diligent counting’ while acknowledging shortcomings in initial pandemic response (Zaki and Wayenberg, 2021). Structurally, Belgium has pronounced ethno-linguistic divides, varying politico-administrative traditions, value perceptions, and cultural cleavages (Pattyn et al, 2021). In particular, these divides are visible between two main regions: Flemish (Dutch speaking) and Walloon (French speaking).5 Each of these regions comprises several provinces with each province comprising several municipalities (De Ceuninck et al, 2010). The spread of COVID-19 is moderated by societal, cultural and structural factors. Hence, subnational epidemiological variations were evident across regions, also across provinces and municipalities within the same regions (Clapson, 2021; Sciensano, 2021). Regionally (on the Flemish–Walloon level), perceptions of the virus, circulation and what needs to be done were seen quite differently where it was ‘Same country, same system, same laws, same regulations yet there were sometimes very different views in the public opinion’ (INT3-E). This applied, even within one region; one example was the city of Antwerp which had an exceptionally large cluster of infections that necessitated localised lockdowns in the summer of 2020, being on a substantially different epidemiological trajectory from other Belgian cities at the time (Zaki and Wayenberg, 2021). Other examples of within-country variations were observable in the different attitudes towards vaccinations and pandemic restrictions (for example, Walker, 2021a) as highlighted in Table 2.

Table 2:

Regional variations in epidemiological conditions

Region Fully vaccinated1 Confirmed cases (per 1000)2 Confirmed deaths (per 1,000)
Brussels capital region 45% 118.89 2.72
Flanders 69% 81.92 1.93
Wallonia 62% 119.99 2.53

Source: https://covid-vaccinatie.be/en as of 11 August 2021.

Sources: Cases and Death stats: Official Sciensano Data https://epistat.wiv-isp.be/covid/ as of 9 August 2021.

In the next section, we present an exploratory account of how – within this heterogenous context – each of the key governance levels approached policy learning. We focus on the key dimensions from our analytical framework of the learning process: advisory structures, actors involved, information and knowledge content. We start at the federal level.

Federal level learning

With the crisis being federally managed, the Belgian government initially established 14 national crisis response and expert advisory structures, later known as the ‘COVID-19 Labyrinth’ (Zaki and Wayenberg, 2021; INT3-E; INT5-G6). This led to significant leadership and coordination issues that were considered ‘fatal’ (Van Overbeke and Stadig, 2020). The two main technical bodies tasked with advising government were the then newly established GEMS (Group of Experts for Management Strategy) group (replacing the expert evaluation cell CELEVAL), and the more technically specialised Risk Assessment Group (RAG), a permanent group within Sciensano, Belgium’s main public health advisory body (Belgian Government, No Date[a]). In terms of expert actors involved within these groups, as many others, CELEVAL (which was later deactivated) mostly consisted of virologists and epidemiologists with an under-representation of sociological and behavioural experts (Zaki and Wayenberg, 2021; INT4-E). This issue persisted (albeit with less severity) in the case of GEMS and RAG (GEMS, 2020b; INT3-E). The logic of expert recruitment was to identify relevant expertise or as one interviewee expressed it: ‘expertise is where it is’, rather than localise learning or craft regionally representative advisory structures. As a result, the advisors’ line-up tilted towards a geographical overrepresentation of Flemish-based experts (GEMS, 2020b). This led to a perception among experts and the public that one region’s view ‘overpowered the other’, casting some legitimacy concerns over the learning process (INT3-E). Given that in a country like Belgium ‘it is hard to govern by imbalance’, the formation of these groups was later adjusted to create a more regionally balanced structure (INT3-E). This was found to be consistent with evidence on differences between how the two main Belgian regions and federal authorities have viewed the pandemic (for example, The Brussels Times, 2020; Chini, 2021; Walker, 2021b), particularly given the underlying subnational variations described by Pattyn et al (2021).

In terms of the knowledge and informational content, analysis of reports produced by the two main technical advisory bodies (14 reports from GEMS and 22 from RAG) consistently demonstrated limited regional contextualisation of learning. For example, GEMS recommendations on nursing homes, telework, epidemiological thresholds and contact tracing remain almost exclusively at the national level with limited localisation of insights (for example, Belgian Government, No Date[b]; GEMS, 2021a). Consistently, recommendations from RAG reflected regional granularity only when alarming spikes in epidemiological indicators are detected (that is, clusters of infections, hospitalisation and intensive care occupancy and so on). This did not include localised advice on what to do, or accounts on socio-economic or behavioural aspects in different regions. However, some medical experts at federal advisory bodies still occasionally provided insights on developments in their own local communities in committee meetings given that they ‘had their fingers on the pulse’ (INT1-E; INT3-E). This was, however, limited to epidemiological evolutions and conditions in critical healthcare facilities. It also remained non-systematic and on an ad hoc basis.

At the federal level, the policy learning processes therein were highly systematised and formalised. This included codified roles and task descriptions, and strong involvement of (predominantly) medical experts, for example, RAG, CELEVAL, GEMS, GEES (INT2-M; INT3-E). There were regular public updates, with established reporting standards. Large-scale changes in advisory bodies were pronounced and often explained through both official and non-official channels (for example, The Brussels Times, 2020b; Zaki and Wayenberg, 2021). This learning systematism was also reflected in experts playing a significant role in shaping crisis policymaking as well as directly addressing the public, sometimes on behalf of the government.

Next, we move one step down to explore how epistemic policy learning was approached at the provincial level.

Provincial level learning

Our analysis at the provincial level is consistent with findings from the federal level. Interviews with provincial officials show that federal level guidance was relatively generic, and was thus ‘interpreted differently’, sometimes leading to ‘confusion’ (INT5-G; INT3-E). This concern was echoed in calls from subnational authorities for clearer information and more precise courses of action (Moens, 2020). This generic guidance interacted with highly uncoordinated subnational governance structures. There, coordination challenges where exacerbated. Belgium had nine health ministers spanning different territorial jurisdictions at the subnational level (Pattyn et al, 2021). This resulted in fragmentation of competences and relatively divergent approaches to combating the pandemic (Van Overbeke and Stadig, 2020). The few evident aspects of local specificity (as perceived by the provincial government officials) seemed to be only in terms of epidemiological updates (for example, case counts, viral reproduction rates). At this level, it was up to provincial governors to steer the epistemic policy learning process or even decide if they wanted to have one.

Governors are considered executive officials appointed by the regional governments (for example, Flemish and Walloon). Their roles include directing policy formulation, implementation and provincial crisis responses (see Valcke et al, 2008). Consequently, variations in subnational approaches to learning were evident. Our analysis shows three main modalities by which provinces engaged in epistemic policy learning.

The first modality entailed intensive engagement with a relatively wide range of scientific experts. Some governors formed their own advisory structures, reached out to personal contacts and individual experts for consultation (INT1-E; INT5-G; INT6-G). This was next to interactions across municipalities with the permanent pre-established provincial crisis cells (for example, see National Crisis Center, No Date). These relatively generic expert cells exist on both regional and the local levels and include professionals from various fields known as the ‘disciplines’ (for example, those in the police force and fire brigades, logistics specialists, doctors, emergency planners). They often produced domain specific and siloed information and knowledge. The second modality entailed minimal engagement with expertise in general or in local expert communities (either from universities or experts within the permanent provincial crisis cells). As such, some provinces relied almost exclusively on the pre-established provincial crisis cells for interpretation and implementation of national policy guidance (INT6-G). The third modality involved a hybrid approach. Governors alternated between learning from their crisis cells on the one hand, and inter-provincial collaborations with other governors who engage with – and have access to – scientific experts within their own provinces, on the other. This was particularly salient in provinces where governors did not have strong existing collaborations with expert communities (or major university/research hubs). Across these three approaches, interactions with experts were largely systematised, mainly given protocols for holding regular meetings with provincial crisis cells (INT5-G; INT6-G). However, the learning structures and the actors therein largely varied.

While in all cases governors perceived their general role as ‘go-between-ers’ linking federal and municipal levels (INT5-G; INT6-G), two key factors seemed to influence their specific approaches to policy learning: perceptions of roles and past professional experiences. Some governors disidentified with being experts (or even policymakers) stating ‘I do not need to be an expert, I need to be a generalist who has sufficient knowledge and expertise of the team to be able to do my job, and so, I felt that was enough’ (INT6-G), while others identified more with the ‘policymaker’ and the ‘expert’ role and hence felt responsible for learning and disseminating sensible information to municipalities (INT5-G; INT6-G). While significant variations in the role perceptions of governors might be surprising, it is not alien to the Belgian case where there is little consensus on the roles, profiles and expectations from governors (see Valcke et al, 2008). As for the second factor, governors from academic or scientific backgrounds seemed to form closer connections and engaged more proactively with expert communities, particularly drawing on their pre-existing personal networks. For example, Antwerp’s governor’s former academic affiliation and networks seemed to foster more engagement with expert groups where intensive epistemic policy learning interactions took place. This helped address emerging clusters of infections within certain local communities and synthesise solutions for provincial outbreaks (INT3-E; INT5-G). It also led to critical sensemaking and contextualisation of policy advice from several layers of experts at the federal or regional levels (see Province of Antwerp, 2020). On the other hand, governors coming from different backgrounds (for example, former mayors or civil servants) tended to rely on pre-established crisis cells within the civil service (INT6-G). These findings are consistent with emerging research on the role of sectoral identities and their influence on policy preferences and action (for example, Hornung et al, 2019).

This leads us to our next inquiry into how municipal officials engaged in epistemic policy learning within the same crisis context.

Municipal level: citizen expectations and learning

With relatively limited coordination and the lack of a common approach to handling the pandemic, municipal action also diverged, and was even sometimes contradictory (for example, Het Laatste Nieuws, 2020). Consistently, approaches to engaging in epistemic policy learning also varied municipally (INT2-M). For example, Leuven, a city with a significant student population, and one of the highest-ranking universities in Belgium, augmented their municipal crisis cells by engaging with local university experts early on. This helped the city to spring to action, closing markets and schools, in the period when the federal government’s guidance was not yet available. The city also worked with local experts to provide guidance on public health practices, even before national directives were announced. Collaborations with local experts were also undertaken to enhance the city’s critical testing strategy and capacity, and process engineer vaccination rollouts (KU Leuven, 2020; INT2-M). The city government’s response to the demographic structure, and citizens’ expectations, seemed to catalyse engagement with epistemic communities where senior officials indicated that the population ‘expects us to take scientifically responsible action’ (INT2-M). This came consistent with the city’s culture of scientific decision making and innovation, particularly during the crisis (for example, Hope, 2020). Collaborations with epistemic communities at the municipal level were mainly a result of individual initiatives by some municipal governments, rather than pre-established institutional structures identifiable across municipalities (INT1-E; INT2-M; INT3-E; INT4-E).

Other city governments felt relatively less compelled to engage in epistemic policy learning processes, also seemingly catalysed by their populations’ expectations, where another city senior official said, ‘our citizens expect us to take reasonable decisions’ (INT2-M), while arguing for less drastic lockdown measures. Across both approaches to epistemic policy learning, municipal government officials have emphasised the need for regional and local specificities to be considered in decision-making (for example, business closures, lockdowns and so on), an issue which municipalities argued was not sufficiently considered by federal government. At this level, we find that the learning process were relatively less structured or systematised (that is, ad hoc). Whenever learning took place, the knowledge content was mostly focused on the core issue at hand (medical and epidemiological).

With that said, our account of policy learning processes across the federal, provincial and municipal levels shows diverging approaches to engaging in policy learning. In the next section we synthesise an answer to this article’s central research question, followed by the implications of these findings for theory and practice.

Discussion

In our analysis, we offered an exploratory account of how epistemic policy learning took place at different levels of the Belgian multilevel architecture. This analysis does not primarily aim to identify an exhaustive set of approaches to engaging in policy learning, particularly given inherent limitations on available data. Rather, it focused on identifying key features of the learning process across different levels. In this section, we draw on our findings to synthesise an answer to our central research question, followed by a discussion of the key implications for research and practice.

How does epistemic policy learning take place across different levels of the multilevel governance architecture?

Federal level advisory bodies had regular interactions and have grown more coordinated as opposed to how they were at the beginning of the pandemic. Steady government action and appointment of a ‘Corona Commissioner’ in October 2020 improved coordination (INT3-E). While crisis governance often has a central locus (particularly at the point of the initial crisis response), as the crisis unfolds, decentralised architectures play increasingly important roles (for example, Kuhlmann et al, 2021; Wayenberg et al, 2022). Here, Belgium’s complex institutional setting has clearly contributed to significant fragmentation and coordination issues (Pattyn et al, 2021). Consistently, this affected how policy learning took place. With the crisis being officially managed at the federal level, there was significant accountability and public pressure on the federal government. This resulted in reduced learner control over the objectives and content of the policy learning process (see Dunlop and Radaelli, 2013). Thus, the federal government was compelled to systematise, institutionalise and structure the policy learning process. While seeking to identify expertise, the regional representativeness of expert actors was compromised, undermining the legitimacy of the learning process. This was later addressed by regaining regional balance in expert committees, aiming to increase perceived legitimacy of learning outcomes (that is, policy recommendations); and in so doing, advancing the acceptance of COVID-19 policies through securing the buy-in of experts from different regions, which indicates a form of political learning as described by Heclo (1974).

Provincially, non-elected officials had higher control over learning objectives and content given that the crisis was mainly under federal management. Three main modalities of learning emerged from our data: Intensive, midrange cross-jurisdictional, to minimal. Ceteris paribus, given that the degree of learner’s control over the objectives and content of the learning process was almost constant at the provincial level, approaches to policy learning were seemingly influenced by two key factors. First, are the background and personal experience of provincial leaders (for example, in some cases, governors from academic backgrounds are more inclined towards intensive epistemic deliberations while those for example from civil service backgrounds are relatively less inclined to do so). The second factor are personal role perceptions. While some governors identified with their roles as policymakers responsible for aligning stakeholders (mayors, citizens and so on) with evidence-based action, others disidentified with such roles saying, ‘I am not a policymaker, I am a civil servant’ or ‘I do not need to be an expert on everything’. These variations were amplified by the role-ambiguity of governors and provincial governments in the Belgian system (see Valcke et al, 2008).

Municipally, elected officials were influenced by the accountability to their publics, yet also had more control over the learning process (again, given that all eyes were on the federal level). There, engagement with experts seemed to be somewhat reflective of the population’s expectations. While some municipal leaders intensively engaged with a wide range of local epistemic actors (university experts or local crisis cells) as their citizens ‘expect that they take scientific decisions’, others were more restrained and in line with their citizen’s expectations where the public pushed back for less drastic measures expecting ‘more reasonable’ decisions.

To position our findings within the broader policy learning debate, we draw on two frameworks/typologies: Dunlop and Radaelli’s (2013) genera of learning that allows us to position the roles of epistemic actors in the learning process as contributors, facilitators, producers of standards, or teachers, and Zaki and Wayenberg’s (2021) framework of epistemic policy learning inclusiveness. Additionally, we offer a description of the policy learning process by showcasing to what extent it was systematic, institutionalised and coordinated across different levels (Table 3).

Table 3:

State of epistemic policy learning operationalisation across multiple levels

Epistemic policy learning as Federal level (National) Lower learner control Provincial level (Subnational) High learner control Municipal level (Subnational) Higher learner control
An institutionalised practice
  • Relatively institutionalised.

  • Engagement of national research institutions

  • Mostly individual.

  • Minor collaboration in institutional forms.

  • Mostly individual.

  • Minor collaboration in an institutional form.

A systematised practice Systematic, periodic, and formalised. Adhoc, influenced by policymakers’ role perceptions and orientations. Adhoc, influenced by the civil service’s relationship with local clusters of experts.
A coordinated practice
  • Coordinated through established structures.

  • Varying yet clear inter-committee coordination.

  • Relatively less coordinated.

  • Contingent on policymakers’ coordination abilities and networks.

  • Relatively less coordinated.

  • Coordinated by individual policymakers.

Main outcomes Instrumental learning (May, 1992) Instrumental learning (May, 1992)
Inclusiveness of epistemic policy learning processes Relatively inter and intradisciplinary (health and behavioural experts inclusion evolved over time) Highly focused on medical experts
Role of experts
  • Teachers

  • Producers of standards

  • Producers of standards

  • Facilitators

  • Contributors

  • (Dependent on key policy actor preferences, role perceptions, and professional backgrounds)

The big picture: what did the crisis do to the policy learning process from a multilevel governance perspective?

Understanding the dynamics of policy learning as a key mechanism for policymaking and policy change within crises is ever pressing. Accounts of how learning takes place within these contexts need to consider the context of learning, its interaction with the policy issue, and the structures in which it occurs. Accordingly, in this article we explored how policy learning took place, not only at different levels of the Belgian multilevel governance architecture, but also across different units within the same levels. Next, we leverage our findings to answer an important inquiry for theory and practice. That is, how did COVID-19 shape the policy learning process from a multilevel governance perspective?

Creeping crises are substantially different from one-time shock crisis events such as natural disasters, for which there are standard crisis responses and operating procedures (for example, Boin and Lodge, 2021). Our findings show that the crisis broke down what would have otherwise been one overarching policy learning process into a set of smaller heterogenous processes across multiple levels. This is of course exacerbated by the existing complex institutional structure known to Belgium. Yet, why? And how? Here, three interlinked creeping crisis features come to the fore: policy issue context sensitivity, scale and duration.

The crisis is highly context sensitive. Crisis policymaking needed to account for socio-economic, behavioural and cultural contexts. It is also large-scale, that is, global in nature, and transboundary, that is, cuts across different sectors (Zaki and George, 2021). Hence, the COVID-19 crisis intersects with a range of subnational contexts and sectors (across different provinces and municipalities). The crisis is also dynamic in nature, that is, it has several outbursts of varying intensity. In terms of duration, unfolding over more than two years, the manifestation and social construction of the policy issue also evolved over time, from an initially exclusive public health issue to a complex socio-economic one (Zaki et al, 2022a; 2022c). This diminished the effectiveness of ‘one size fits all’ approaches to crisis policymaking. Accordingly, policymakers at different levels needed to continuously interpret and adjust often technically complex and generic policy guidance from the federal government to their local contexts.

This necessitated expert engagement through subnational policy learning processes. However, the absence of a common framework for policy learning rendered subnational learning processes visibly heterogenous. There, Dunlop and Radaelli’s (2013) degrees of learner control over epistemic policy learning robustly predicts the existence of variations across levels. We clearly see more structure, systematisation and accountability at the federal level (where policymakers had less control over the learning process) as opposed to the subnational levels (where policymakers had more). This created a set of divergent policy learning processes driven by heterogenous ‘epistemic islands’ across governance units.

So, what does this imply for theories of policy learning and policy change? Our findings show that within complex multilevel governance systems and during creeping crises, there is no monolithic policy learning process. Rather, there are several dispersed and context sensitive ones. Accordingly, theories aiming to account for the relationship between policy learning and policy change, need to consider the multilevelness of learning processes (for example, Dunlop and Radaelli, 2017; Zaki et al, 2022). Now, what does this imply for practice? Here, we offer two key recommendations towards enhancing policy learning across different levels within such contexts.

Bridging epistemic islands

Large-scale crises emphasise the value of decentralised policymaking and policy learning. However, their induced pressure often blurs the lines between decentralisation and fragmentation (for example, Schomaker et al, 2021; McConnell and Stark, 2021). Decentralised systems with legacies of institutional complexity and heterogenous politico-administrative traditions such as Belgium can be more prone to fragmentation under crisis pressure (see Mei, 2020). This overlaps with existing complex policymaking and advisory structures that are already challenging to govern (for example, see Valcke et al, 2008; Pattyn et al, 2021). Limited consistency, vertical and horizontal coordination of expert advisory mechanisms and policy learning processes can create heterogenous ‘epistemic islands’. In our findings, we see them as disconnected and relatively uncoordinated advisory structures across territories and regions. They have little to no common governance frameworks, and thus do not share a common formation rationale. Accordingly, they remain at diverging levels of disciplinarity and expertise, leading to diverging sensemaking and mean making frames (Zaki and Wayenberg, 2021; INT5-G). Critically, this influences understandings of policy problems, narratives and public perceptions. This can lead to subnational jurisdictions ending on diverging (yet not necessarily contextually valid) crisis response pathways. Limited local context sensitivity can constrain policy responses’ social fit legitimacy and compliance (thus, effectiveness). Hence, practitioners might need to consider: first, an overarching and systematic approach to identifying and integrating expertise into the policy learning process across multiple levels. This can be vertically (along different levels) and horizontally (across different territories at the subnational levels). Second, given crisis complexity, practitioners might need to consider utilising boundary-spanning expertise or learning ‘managers’ to coordinate disconnected islands into a network of coordinated advisory bodies. This needs to account for existing institutional legacies and politico-administrative traditions and capacities. This is an issue that emphasises the potential for integrating public administration experts in the design and coordination of policy learning solutions within complex settings.

Aligning structures, actors, knowledge and information

Varying politico-administrative traditions, existence of multiple advisory bodies across different levels and exogenous crisis pressures can contribute to fragmentation and weakened crisis governance (for example, Mei, 2020). Thus, for adequate learning governance, policymakers might need to consider a systems’ thinking approach that accounts for the interactions between advisory structures, their outcomes (for example, vis-a-vis features of informational content and socio-political embeddedness) and existing governance architectures. In part, this can be achieved through establishing institutional mechanisms to foster alignment between organisational and multilevel structures (see Carter and May, 2020). Additionally, heuristics across the governance architecture warrant consideration. Subnational decision makers (for example, provincial governors and mayors) can have surprisingly varying role perceptions, particularly as role descriptions can become blurred or malleable during long-term crises. Such perceptions can be shaped by several factors including (but not limited to) the decision makers’ own professional backgrounds and degrees of control over the learning process. This is congruent with our knowledge of how sectorial biases, beliefs, value perceptions, experiences and heuristics influence policymakers’ interactions with scientific expertise (for example, Hornung et al, 2019; Heikkila et al, 2020), particularly around novel non-structured issues (Hoppe, 2017). Thus, mapping the structural pathways that central level epistemic policy learning outcomes can take becomes critical for adequate learning governance and stakeholder alignment.

Conclusion

Our findings show that epistemic policy learning was varyingly employed across all different levels of the governance architecture during the COVID-19 crisis. These variations could be attributed to different degrees of policymakers’ control over learning objectives and content as proposed by Dunlop and Radaelli (2013). Variations in said degrees of control between the national and subnational levels could be explained given the official designation of policy issue management (being at the federal level). Within the subnational realm (where decision-makers had high control over the learning process), these variations could be partly explained by the policymakers’ role perceptions, professional backgrounds, and their local constituencies’ expectations. Our findings in this article also show that Dunlop and Radaelli’s (2013) and Dunlop’s (2017a) findings on shifts in epistemic communities’ engagement modalities as a function of learner’s degrees of control do not only hold valid over time (as knowledge becomes more public) but also across space.

In this article, we contributed to policy learning theory by exploring how policy learning takes place across multilevel governance systems and proposing potential explanatory parameters able to account for spatial variations in policy learning modalities. We also provided insights to policymakers on how to strengthen policy learning processes during creeping crises across different levels in multilevel governance systems.

Finally, our findings should be considered preliminary and exploratory. While these findings are consistent with our theoretical expectations, we are certainly limited by the number of interviews, which were determined by the available set of officials, number of provinces, and municipalities with major scientific/research institutes. Hence, we do not aim to make generalisable statements or offer an exhaustive listing of policy learning modalities or factors shaping them. However, our preliminary findings create space for a future research agenda. Here, we would like to outline three potentially valuable areas for future research: first, expanding the empirical base by exploring multiple cases in different contexts (for example, different countries or types of crises). This can allow for exploring new modalities of learning, their determinants, as well as refine the ones we highlight. On this note, future research can also explore how variations that come with multilevel governance structures (for example, administrative traditions, politics, levels of control and so on) could help provide better explanations for policy learning processes. This is through offering a comparative vantage point of how learning takes place across varying units with different configurations and institutional features. A second potentially valuable area for future research is to employ a range of methods to test the effects of the microfoundational factors we explored on learning preferences and their intersections, with policy actors being at different levels of control over the policy learning process (see Dunlop and Radaelli, 2013; Hornung et al, 2019). A third area for future research is at the nexus of knowledge, actors, learning structures and context, where future research can look into how paradigms of crisis-specific sharing of responsibilities impact knowledge production patterns across different advisory bodies, and the extent to which this shapes crisis learning. Similarly, future research can also explore how expert knowledge produced at one level influences knowledge production at other levels, that is, the interactions between multilevel knowledge production processes, and their effects. In the same direction, future research can explore how the disruptive impact of the crisis affects the paradigms underlying the identification of expertise and formation of expert advisory groups, within the Belgian system and elsewhere. In the future, this can utilise longitudinal analysis and historical research.

Notes

1

Sciensano is one of Belgium’s main public health institutes in charge of coordinating COVID-19 policy advisory structures.

2

Similar to epistemic policy learning other modes also involve different mechanisms based on learners’ control over learning means and ends (Dunlop and Radaelli, 2013). For example, within reflexive learning, learning can happen deliberatively, experimentally, framing-oriented, or evolutionary. In hierarchy-oriented learning, learning can happen with autonomy, instrumentally, through delegation, or is hetero-directed.

3

Triangulation was primarily conducted by cross-verification of inputs from different interviews (insights from experts, provincial and municipal officials) and documentary analysis.

4

All ten provincial cabinets were contacted, 4 have returned the invitation for interviews. 3 municipal cabinets with major scientific/ academic institutes were contacted, 2 have returned the invitation for interviews.

5

From a formal point of view, Belgium is divided into three regions (the Flemish, the Walloon and the Brussels region) and three communities (the Dutch speaking, the French speaking and the German speaking community). Regions have area-based competencies whilst communities are responsible for person-oriented tasks. Their respective territories do not coincide, what results out of various rounds of state reform over the last decades.

6

INT: Interview data. G: Provincial level, E: Experts, M: Municipal level.

Acknowledgements

The authors would like to thank our three anonymous reviewers and the editorial office for their constructive and helpful comments. The authors would also like to sincerely thank Prof. Dr. Claire Dupont from the Department of Public Governance and Management, Ghent University for her constructive and helpful comments on earlier versions of this paper. We would like to also thank Nadège from UCLouvain for interview assistance, and Sarah Brown, Senior Journals Manager at Policy & Politics for her continuous support.

Conflict of interest

The authors declare that there is no conflict of interest.

References

  • Baekkeskov, E. and Öberg, P. (2016) Freezing deliberation through public expert advice, Journal of European Public Policy, 24(7): 100626. doi: 10.1080/13501763.2016.1170192

    • Search Google Scholar
    • Export Citation
  • Belgian Government (No Date[a]) What is the government doing?, Coronavirus COVID-19, https://www.info-coronavirus.be/en/what-is-the-government-doing-about-it/[archived website].

    • Search Google Scholar
    • Export Citation
  • Belgian Government (No Date[b]) Verslagen adviesorgaan, Info Coronavirus, https://www.info-coronavirus.be/en/celeval-2022/.

  • Belgian Government. (2020, July 15) What is the government doing? Retrieved from Coronavirus COVID-19, https://www.info-coronavirus.be/en/what-is-the-government-doing-about-it/.

    • Search Google Scholar
    • Export Citation
  • Bennett, C.J. and Howlett, M. (1992) The lessons of learning: reconciling theories of policy learning and policy change, Policy Sciences, 25: 27594. doi: 10.1007/bf00138786

    • Search Google Scholar
    • Export Citation
  • Bergeron, H., Borraz, O., Castel, P. and Dedieu, F. (2020) Covid-19: une crise organisationnelle, SciencesPo. doi: 10.3917/scpo.berge.2020.01

    • Search Google Scholar
    • Export Citation
  • Bergström, T., Kuhlmann, S., Laffin, M. and Wayenberg, E. (2022) Special issue on comparative intergovernmental relations and the pandemic: how European devolved governments responded to a public health crisis, Local Government Studies, 48(2): 17990. doi: 10.1080/03003930.2022.2039636

    • Search Google Scholar
    • Export Citation
  • Boin, A. and Lodge, M. (2021) Responding to the COVID-19 crisis: a principled or pragmatist approach?, Journal of European Public Policy. doi: 10.1080/13501763.2021.1942155

    • Search Google Scholar
    • Export Citation
  • Boin, A., Ekengren, M. and Rhinard, M. (2020) Hiding in plain sight: conceptualizing the creeping crisis, Risks, Hazards, & Crisis in Public Policy, 11(2): 11638. doi: 10.1002/rhc3.12193

    • Search Google Scholar
    • Export Citation
  • Bomberg, E. (2007) Policy learning in an enlarged European Union: environmental NGOs and new policy instruments, Journal of European Public Policy, 14(2): 24868. doi: 10.1080/13501760601122522

    • Search Google Scholar
    • Export Citation
  • Borrás, S. (2011) Policy learning and organizational capacities in innovation policies, Science and Public Policy, 72534. doi: 10.3152/030234211X13070021633323

    • Search Google Scholar
    • Export Citation
  • Bosa, I., Castelli, A., Castelli, M., Ciani, O. et al. (2021) Corona-regionalism? Differences in regional responses to COVID-19 in Italy, Health Policy, 125(9): 117987. doi: 10.1016/j.healthpol.2021.07.012

    • Search Google Scholar
    • Export Citation
  • Cairney, P. (2021) The UK Government’s COVID-19 policy: what does ‘guided by the science’ mean in practice?, Frontiers in Political Science. doi: 10.3389/fpos.2021.624068

    • Search Google Scholar
    • Export Citation
  • Cao, C., Li, N. and Liu, L. (2020) Do national cultures matter in the containment of COVID-19?, International Journal of Sociology and Social Policy. doi: 10.1108/IJSSP-07-2020-0334

    • Search Google Scholar
    • Export Citation
  • Capano, G. and Lippi, A. (2021) Decentralization, policy capacities, and varieties of first health response to the COVID-19 outbreak: evidence from three regions in Italy, Journal of European Public Policy, 28(8): 1197218. doi: 10.1080/13501763.2021.1942156

    • Search Google Scholar
    • Export Citation
  • Carter, D.P. and May, P.J. (2020) Making sense of the US COVID-19 pandemic response: a policy regime perspective, Administrative Theory & Praxis, 42(2): 26577. doi: 10.1080/10841806.2020.1758991

    • Search Google Scholar
    • Export Citation
  • Casula, M. and Pazos-Vidal, S. (2021) Assessing the multi-level government response to the COVID-19 crisis: Italy and Spain compared, International Journal of Public Administration, 44(11-12): 9941005. doi: 10.1080/01900692.2021.1915330

    • Search Google Scholar
    • Export Citation
  • Chini, M. (2021) Covid Safe Ticket mandatory in Brussels, but no fines yet, 15 October, https://www.brusselstimes.com/brussels/189262/covid-safe-ticket-mandatory-in-brussels-from-tomorrow-but-no-fines-yet/.

    • Search Google Scholar
    • Export Citation
  • Clapson, C. (2021) Once again big differences in Covid data between Flanders and Wallonia, VRT News, 4 February, https://www.vrt.be/vrtnws/en/2021/02/04/once-again-big-differences-in-covid-data-between-flanders-and-wa/.

    • Search Google Scholar
    • Export Citation
  • Daigneault, P.M. (2013) Reassessing the concept of policy paradigm: aligning ontology and methodology in policy studies, Journal of European Public Policy, 21(3): 45369. doi: 10.1080/13501763.2013.834071.

    • Search Google Scholar
    • Export Citation
  • De Ceuninck, K., Reynaert, H., Steyvers, K. and Valcke, T. (2010) Municipal amalgamations in the low countries: same problems, different solutions, Local Government Studies, 36(3): 803822. doi: 10.1080/03003930.2010.522082

    • Search Google Scholar
    • Export Citation
  • Di Giulio, M. and Vecchi, G. (2018) Multilevel policy implementation and the where of learning: the case of the information system for school buildings in Italy, Policy Sciences, 52: 11935. doi: 10.1007/s11077-018-9326-4

    • Search Google Scholar
    • Export Citation
  • Dunlop, C. (2017a) The irony of epistemic learning: epistemic communities, policy learning and the case of Europe’s hormones saga, Policy and Society, 36(2): 21532. doi: 10.1080/14494035.2017.1322260

    • Search Google Scholar
    • Export Citation
  • Dunlop, C.A. (2017b) Policy learning and policy failure: definitions, dimensions and intersections, Policy & Politics, 45(1): 318. doi: 10.1332/030557316X14824871742750

    • Search Google Scholar
    • Export Citation
  • Dunlop, C.A. and Radaelli, C.M. (2013) Systematising policy learning: from monolith to dimensions, Political Studies, 61(3): 599619. doi: 10.1111/j.1467-9248.2012.00982.x

    • Search Google Scholar
    • Export Citation
  • Dunlop, C.A. and Radaelli, C.M. (2017) Learning in the Bath-tub: the micro and macro dimensions of the causal relationship between learning and policy change, Policy and Society, 36(2): 30419. doi: 10.1080/14494035.2017.1321232

    • Search Google Scholar
    • Export Citation
  • Fobé, E., Brans, M., Vancoppenolle, D. and Van Damme, J. (2013) Institutionalized advisory systems: an analysis of member satisfaction of advice production and use across 9 strategic advisory councils in Flanders (Belgium), Policy and Society, 32: 22540. doi: 10.1016/j.polsoc.2013.07.004

    • Search Google Scholar
    • Export Citation
  • GEMS (2020b) Advies Overlegcomité: Epidemiologische Situatie, Brussels: GEMS, https://d34j62pglfm3rr.cloudfront.net/celeval/GEMS_OCC_Epidemiological_update_measures_20201215.pdf.

    • Search Google Scholar
    • Export Citation
  • GEMS (2021a) Advies Overlegcomité: Contactberoepen, Brussels: Belgian Government, https://d34j62pglfm3rr.cloudfront.net/celeval/GEMS_OCC_Contactberoepen_20201215.pdf.

    • Search Google Scholar
    • Export Citation
  • Gerring, J. (2004) What is a case study and what is it good for?, The American Political Science Review, 98(2): 34154, https://www.jstor.org/stable/4145316. doi: 10.1017/S0003055404001182

    • Search Google Scholar
    • Export Citation
  • Haas, P. (1992) Introduction: epistemic communities and international policy coordination, International Organization, 46: 135. doi: 10.1017/S0020818300001442

    • Search Google Scholar
    • Export Citation
  • Happaerts, S. (2015) Climate governance in federal Belgium: modest subnational policies in a complex Multi-level setting, Journal of Integrative Environmental Sciences, 12(4): 285301. doi: 10.1080/1943815X.2015.1093508

    • Search Google Scholar
    • Export Citation
  • Heclo, H. (1974) Modern Social Politics in Britain and Sweden: From Relief to Income Maintenance, New Haven, CT: Yale University Press.

  • Hegele, Y. and Schnabel, J. (2021) Federalism and the management of the COVID-19 crisis: centralisation, decentralisation and (Non-)coordination, Western European Politics. doi: 10.1080/01402382.2021.1873529

    • Search Google Scholar
    • Export Citation
  • Heikkila, T., Weible, C.M. and Gerlak, A.K. (2020) When does science persuade (or not persuade) in High-conflict policy contexts?, Public Administration, 98(3): 53550. doi: 10.1111/padm.12655

    • Search Google Scholar
    • Export Citation
  • Het Laatste Nieuws (2020) Minister de crem fluit burgemeesters die te strenge coronarestricties oplegden terug, Het Laatste Nieuws (HLN) Online, 19 May, https://www.hbvl.be/cnt/dmf20200519_04965158.

    • Search Google Scholar
    • Export Citation
  • Hope, A. (2020) Leuven named European capital of innovation 2020, The Brussels Times, 24 September, https://www.brusselstimes.com/132644/leuven-is-named-european-capital-of-innovation-2020.

    • Search Google Scholar
    • Export Citation
  • Hoppe, R. (2017) Heuristics for practitioners of policy design: Rules-of-thumb for structuring unstructured problems, Public Policy and Administration, 33(4): 384408. doi: 10.1177/0952076717709338

    • Search Google Scholar
    • Export Citation
  • Hoppe, R. (2018) Rules-of-thumb for Problem-structuring policy design, Policy Design and Practice, 1(1): 1229. doi: 10.1080/25741292.2018.1427419

    • Search Google Scholar
    • Export Citation
  • Hornung, J., Bandelow, N.C. and Vogeler, C.S. (2019) Social identities in the policy process, Policy Sciences, 52: 21131. doi: 10.1007/s11077-018-9340-6

    • Search Google Scholar
    • Export Citation
  • Kamkhaji, J.C. and Radaelli, C.M. (2017) Crisis, learning and policy change in the European Union, Journal of European Public Policy, 24(5): 71434. doi: 10.1080/13501763.2016.1164744

    • Search Google Scholar
    • Export Citation
  • Kettl, D.F. (2020) States divided: the implications of American federalism for COVID‐19, Public Administration Review, 80(4): 595602. doi: 10.1111/puar.13243

    • Search Google Scholar
    • Export Citation
  • KU Leuven (2020) These scientists are tracing the source of corona infections among Leuven students, KU Leuven, 12 December, https://nieuws.kuleuven.be/en/content/2020/these-scientists-are-tracing-the-source-of-corona-infections-among-leuven-students.

    • Search Google Scholar
    • Export Citation
  • Kuhlmann, S., Hellström, M., Ramberg, U. and Reiter, R. (2021) Tracing divergence in crisis governance: responses to the COVID-19 pandemic in France, Germany and Sweden compared, International Review of Administrative Sciences, 87(3): 55675. doi: 10.1177/00208523209793

    • Search Google Scholar
    • Export Citation
  • Ladi, S. and Tsarouhas, D. (2020) EU economic governance and Covid-19: policy learning and windows of opportunity, Journal of European Integration, 42(8): doi: 10.1080/07036337.2020.1852231

    • Search Google Scholar
    • Export Citation
  • Lee, S., Hwang, C. and Moon, M.J. (2020) Policy learning and crisis Policy-making: quadruple-loop learning and COVID-19 responses in South Korea, Policy and Society, 39(3): 36381. doi: 10.1080/14494035.2020.1785195

    • Search Google Scholar
    • Export Citation
  • Leino, H. and Peltomaa, J. (2012) Situated knowledge – situated legitimacy: consequences of citizen participation in local environmental governance, Policy and Society, 31(2): 15968. doi: 10.1016/j.polsoc.2012.04.005

    • Search Google Scholar
    • Export Citation
  • Maggetti, M. (2020) Trust, coordination and multi-level arrangements: lessons for a European Health Union, European Journal of Risk Regulation, 11(4): 79099. doi: 10.1017/err.2020.97

    • Search Google Scholar
    • Export Citation
  • Maggetti, M. and Trein, P. (2018) Multilevel governance and Problem-solving: towards a dynamic theory of multilevel Policy-making?, Public Administration, 97(2): 35569. doi: 10.1111/padm.12573

    • Search Google Scholar
    • Export Citation
  • Mattei, P. and Del Pino, E. (2021) Coordination and health policy responses to the first wave of COVID-19 in Italy and Spain, Journal of Comparative Policy Analysis: Research and Practice, 23(2): 27481. doi: 10.1080/13876988.2021.1878886

    • Search Google Scholar
    • Export Citation
  • Mavrot, C. and Sager, F. (2018) Vertical epistemic communities in multilevel governance, Policy & Politics, 46(3): 391407. doi: 10.1332/030557316X14788733118252

    • Search Google Scholar
    • Export Citation
  • May, P.J. (1992) Policy learning and failure, Journal of Public Policy, 12(4): 33154. doi: 10.1017/S0143814X00005602

  • McConnell, A. and Stark, A. (2021) Understanding policy responses to COVID-19: the stars haven’t fallen from the sky for scholars of public policy, Journal of European Public Policy, 28(8): 111530. doi: 10.1080/13501763.2021.1942518

    • Search Google Scholar
    • Export Citation
  • Mei, C. (2020) Policy style, consistency and the effectiveness of the policy mix in China’s fight against COVID-19, Policy and Society, 39(3): 30925. doi: 10.1080/14494035.2020.1787627

    • Search Google Scholar
    • Export Citation
  • Moens, B. (2020) Not even coronavirus can unite Belgium, Politico, 12 March, https://www.politico.eu/article/coronavirus-belgium-division/.

    • Search Google Scholar
    • Export Citation
  • National Crisis Center (No Date) Coordination in an emergency, Nationaal Crisiscentrum, https://crisiscenter.be/en/what-do-authorities-do/crisis-management/coordination-emergency.

    • Search Google Scholar
    • Export Citation
  • NCCN (No Date) De vijf disciplines, Nationaal Crisiscentrum, https://crisiscentrum.be/nl/inhoud/de-vijf-disciplines.

  • Neij, L. and Heiskanen, E. (2021) Municipal climate mitigation policy and policy learning – a review, Journal of Cleaner Production. doi: 10.1016/j.jclepro.2021.128348.

    • Search Google Scholar
    • Export Citation
  • Nowell, B. and Albrecht, K. (2018) A reviewer’s guide to qualitative rigor, Journal of Public Administration Research And Theory, 34863. doi: 10.1093/jopart/muy052

    • Search Google Scholar
    • Export Citation
  • Pattyn, V., Blum, S., Fobe, E., Pekar-Milicevic, M. and Brans, M. (2019) Academic policy advice in Consensus-seeking countries: the cases of Belgium and Germany, International Review of Administrative Sciences, 117. doi: 10.1177/0020852319878780

    • Search Google Scholar
    • Export Citation
  • Pattyn, V., Matthys, J. and Van Hecke, S. (2021) High-stakes crisis management in the low countries: comparing government responses to COVID-19, International Review of Administrative Sciences, 87(3): 593611. doi: 10.1177/0020852320972472

    • Search Google Scholar
    • Export Citation
  • Popelier, P. (2020) COVID-19 legislation in Belgium at the crossroads of a political and a health crisis, The Theory and Practice of Legislation, 8(1–2): 13153. doi: 10.1080/20508840.2020.1771884

    • Search Google Scholar
    • Export Citation
  • Province of Antwerp (2020) Press release, Gouverneur Antwerpen, 26 August, https://www.gouverneurantwerpen.be/content/dam/cathy-berx/coronadocumenten/20200826_Persbericht%20toelichting%20epidemiologische%20situatie%20en%20de%20maatregelen%20in%20de%20strijd%20tegen%20het%20corona%20COVID-19%20virus%20in%20provincie%20Antwerpen.p.

    • Search Google Scholar
    • Export Citation
  • Radaelli, C. M. (2009) Measuring policy learning: regulatory impact assessment in Europe, Journal of European Public Policy, 16(8): 114564. doi: 10.1080/13501760903332647

    • Search Google Scholar
    • Export Citation
  • Raudla, R., Aleksandrs, C., Vytautas, K. and Rainer, K. (2018) Fiscal policy learning from crisis: comparative analysis of the baltic countries, Journal of Comparative Policy Analysis: Research and Practice, 20(3): 288303. doi: 10.1080/13876988.2016.1244947

    • Search Google Scholar
    • Export Citation
  • Rietig, K. and Perkins, R. (2017) Does learning matter for policy outcomes? The case of integrating climate finance into the EU budget, Journal of European Public Policy, 25(4): 487505. doi: 10.1080/13501763.2016.1270345

    • Search Google Scholar
    • Export Citation
  • Schomaker, R.M., Hack, M. and Mandry, A.K. (2021) The EU’s reaction in the first wave of the Covid-19 pandemic between centralisation and decentralisation, formality and informality, Journal of European Public Policy. doi: 10.1080/13501763.2021.1942153

    • Search Google Scholar
    • Export Citation
  • Sciensano (2021) COVID-19 wekelijks epidemiologisch bulletin (23 April 2021), Brussels: Sciensano, https://covid-19.sciensano.be/sites/default/files/Covid19/COVID-19_Weekly_report_NL.pdf.

    • Search Google Scholar
    • Export Citation
  • Stake, R.E. (1995) The Art of Case Study Research, London: Sage Publications.

  • Tavory, I. and Timmermans, S. (2012) Theory construction in qualitative research: from grounded theory to abductive analysis, Sociological Theory, 30(3): 16786. doi: 10.1177%2F0735275112457914

    • Search Google Scholar
    • Export Citation
  • The Brussels Times. (2020) Wallonia, Brussels, Flanders: Belgium’s Covid-19 state of play, 12 October, https://www.brusselstimes.com/135472/wallonia-brussels-flanders-belgiums-covid-19-state-of-play-leuven-ben-weyts-schools-sports-hospitality-bars-cafes-restaurants-figures-sciensano-steven-van-gucht-hainaut-liege-namur-luxembourg-walloon.

    • Search Google Scholar
    • Export Citation
  • The Brussels Times (2020a) Belgium in brief: Flanders falls behind, The Brussels Times, 26 October, https://www.brusselstimes.com/belgium/137635/belgium-in-brief-flanders-falls-behind-jan-jambon-virologist-steven-van-gucht-brussels-newest-measures-wallonia-curfew-average-infections-sciensano-public-health-institute-autumn-school-holiday/.

    • Search Google Scholar
    • Export Citation
  • The Brussels Times (2020b) Coronavirus: Belgium gets new group of expert advisors, The Brussels Times, 11 December, https://www.brusselstimes.com/news/belgium-all-news/health/144865/coronavirus-belgium-gets-new-group-of-expert-advisors-gees-celeval-consultative-committee-erika-vlieghe-yves-van-laethem-steven-van-gucht-marc-van-ranst/.

    • Search Google Scholar
    • Export Citation
  • Valcke, T., De Ceuninck, K., Reynaert, H. and Steyvers, K. (2008) Leadership, governance and legitimacy at an intermediate government level: the case of Belgian governors, Local Government Studies, 34(2): 24565. doi: 10.1080/03003930701852328

    • Search Google Scholar
    • Export Citation
  • Van Dooren, W. and Noordegraaf, M. (2020) Staging science: authoritativeness and fragility of models and measurement in the COVID‐19 crisis, Public Administration Review, 80(4): 61015. doi: 10.1111/puar.13219

    • Search Google Scholar
    • Export Citation
  • Van Overbeke, T. and Stadig, D. (2020) High politics in the low countries: COVID-19 and the politics of strained multi-level policy cooperation in Belgium and the Netherlands, European Policy Analysis, 6(2): 30517. doi: 10.1002/epa2.1101

    • Search Google Scholar
    • Export Citation
  • Walker, L. (2021a) Over one-third of French speakers in Belgium would refuse coronavirus vaccine, The Brussels Times, 14 May, https://www.brusselstimes.com/news/belgium-all-news/health/169499/over-one-third-of-french-speakers-in-belgium-would-refuse-coronavirus-vaccine-brussels-flanders-wallonia-uclouvain-ulb-ugent-motivatie-barometer/.

    • Search Google Scholar
    • Export Citation
  • Walker, L. (2021b) ‘Completely at odds with suggestions’: experts divided over Flanders’ summer plan, 10 May, https://www.brusselstimes.com/belgium/168776/completely-at-odds-with-suggestions-erika-vlieghe-marc-van-ranst-criticises-flanders-summer-plan-festivals-pukkelpop-tomorrowland-consultative-committee/.

    • Search Google Scholar
    • Export Citation
  • Wayenberg, E., Resodihardjo, S.L., Voets, J., Van Genugten, M., Van Haelter, B. and Torfs, I. (2022) The Belgian and Dutch response to COVID-19: change and stability in the mayors’ position, Local Government Studies, 48(2): 27190. doi: 10.1080/03003930.2021.1958787

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L. and George, B. (2021) New development: policy learning and public management – a match made in crisis, Public Money and Management, 42(2): 12932. doi: 10.1080/09540962.2021.1956212

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L. and Wayenberg, E. (2021) Shopping in the scientific marketplace: COVID-19 through a policy learning lens, Policy Design and Practice, 4(1): 1532. doi: 10.1080/25741292.2020.1843249

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Pattyn, V. and Wayenberg, E. (2022a) Policy learning mode shifts during creeping crises: a storyboard of COVID-19 driven learning in Belgium, European Policy Analysis. doi: 10.1002/epa2.1165

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Wayenberg, E. and George, B. (2022b) A systematic review of policy learning: tiptoeing through a conceptual minefield, Policy Studies Yearbook, 12: 152. doi: 10.18278/psy.12.1.2

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Nicoli, F., Wayenberg, E. and Verschuere, B. (2022c) Contagious inequality: economic disparities and excess mortality during the Covid-19 pandemic, Policy & Society, 41(2): 199216. doi: 10.1093/polsoc/puac011

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Nicoli, F., Wayenberg, E. and Verschuere, B. (2022d) “In Trust we Trust?”: The impact of trust in government on excess mortality during the COVID-19 pandemic, Public Policy and Administration, 37(2): 22652. doi: 10.1177%2F09520767211058003

    • Search Google Scholar
    • Export Citation
  • Baekkeskov, E. and Öberg, P. (2016) Freezing deliberation through public expert advice, Journal of European Public Policy, 24(7): 100626. doi: 10.1080/13501763.2016.1170192

    • Search Google Scholar
    • Export Citation
  • Belgian Government (No Date[a]) What is the government doing?, Coronavirus COVID-19, https://www.info-coronavirus.be/en/what-is-the-government-doing-about-it/[archived website].

    • Search Google Scholar
    • Export Citation
  • Belgian Government (No Date[b]) Verslagen adviesorgaan, Info Coronavirus, https://www.info-coronavirus.be/en/celeval-2022/.

  • Belgian Government. (2020, July 15) What is the government doing? Retrieved from Coronavirus COVID-19, https://www.info-coronavirus.be/en/what-is-the-government-doing-about-it/.

    • Search Google Scholar
    • Export Citation
  • Bennett, C.J. and Howlett, M. (1992) The lessons of learning: reconciling theories of policy learning and policy change, Policy Sciences, 25: 27594. doi: 10.1007/bf00138786

    • Search Google Scholar
    • Export Citation
  • Bergeron, H., Borraz, O., Castel, P. and Dedieu, F. (2020) Covid-19: une crise organisationnelle, SciencesPo. doi: 10.3917/scpo.berge.2020.01

    • Search Google Scholar
    • Export Citation
  • Bergström, T., Kuhlmann, S., Laffin, M. and Wayenberg, E. (2022) Special issue on comparative intergovernmental relations and the pandemic: how European devolved governments responded to a public health crisis, Local Government Studies, 48(2): 17990. doi: 10.1080/03003930.2022.2039636

    • Search Google Scholar
    • Export Citation
  • Boin, A. and Lodge, M. (2021) Responding to the COVID-19 crisis: a principled or pragmatist approach?, Journal of European Public Policy. doi: 10.1080/13501763.2021.1942155

    • Search Google Scholar
    • Export Citation
  • Boin, A., Ekengren, M. and Rhinard, M. (2020) Hiding in plain sight: conceptualizing the creeping crisis, Risks, Hazards, & Crisis in Public Policy, 11(2): 11638. doi: 10.1002/rhc3.12193

    • Search Google Scholar
    • Export Citation
  • Bomberg, E. (2007) Policy learning in an enlarged European Union: environmental NGOs and new policy instruments, Journal of European Public Policy, 14(2): 24868. doi: 10.1080/13501760601122522

    • Search Google Scholar
    • Export Citation
  • Borrás, S. (2011) Policy learning and organizational capacities in innovation policies, Science and Public Policy, 72534. doi: 10.3152/030234211X13070021633323

    • Search Google Scholar
    • Export Citation
  • Bosa, I., Castelli, A., Castelli, M., Ciani, O. et al. (2021) Corona-regionalism? Differences in regional responses to COVID-19 in Italy, Health Policy, 125(9): 117987. doi: 10.1016/j.healthpol.2021.07.012

    • Search Google Scholar
    • Export Citation
  • Cairney, P. (2021) The UK Government’s COVID-19 policy: what does ‘guided by the science’ mean in practice?, Frontiers in Political Science. doi: 10.3389/fpos.2021.624068

    • Search Google Scholar
    • Export Citation
  • Cao, C., Li, N. and Liu, L. (2020) Do national cultures matter in the containment of COVID-19?, International Journal of Sociology and Social Policy. doi: 10.1108/IJSSP-07-2020-0334

    • Search Google Scholar
    • Export Citation
  • Capano, G. and Lippi, A. (2021) Decentralization, policy capacities, and varieties of first health response to the COVID-19 outbreak: evidence from three regions in Italy, Journal of European Public Policy, 28(8): 1197218. doi: 10.1080/13501763.2021.1942156

    • Search Google Scholar
    • Export Citation
  • Carter, D.P. and May, P.J. (2020) Making sense of the US COVID-19 pandemic response: a policy regime perspective, Administrative Theory & Praxis, 42(2): 26577. doi: 10.1080/10841806.2020.1758991

    • Search Google Scholar
    • Export Citation
  • Casula, M. and Pazos-Vidal, S. (2021) Assessing the multi-level government response to the COVID-19 crisis: Italy and Spain compared, International Journal of Public Administration, 44(11-12): 9941005. doi: 10.1080/01900692.2021.1915330

    • Search Google Scholar
    • Export Citation
  • Chini, M. (2021) Covid Safe Ticket mandatory in Brussels, but no fines yet, 15 October, https://www.brusselstimes.com/brussels/189262/covid-safe-ticket-mandatory-in-brussels-from-tomorrow-but-no-fines-yet/.

    • Search Google Scholar
    • Export Citation
  • Clapson, C. (2021) Once again big differences in Covid data between Flanders and Wallonia, VRT News, 4 February, https://www.vrt.be/vrtnws/en/2021/02/04/once-again-big-differences-in-covid-data-between-flanders-and-wa/.

    • Search Google Scholar
    • Export Citation
  • Daigneault, P.M. (2013) Reassessing the concept of policy paradigm: aligning ontology and methodology in policy studies, Journal of European Public Policy, 21(3): 45369. doi: 10.1080/13501763.2013.834071.

    • Search Google Scholar
    • Export Citation
  • De Ceuninck, K., Reynaert, H., Steyvers, K. and Valcke, T. (2010) Municipal amalgamations in the low countries: same problems, different solutions, Local Government Studies, 36(3): 803822. doi: 10.1080/03003930.2010.522082

    • Search Google Scholar
    • Export Citation
  • Di Giulio, M. and Vecchi, G. (2018) Multilevel policy implementation and the where of learning: the case of the information system for school buildings in Italy, Policy Sciences, 52: 11935. doi: 10.1007/s11077-018-9326-4

    • Search Google Scholar
    • Export Citation
  • Dunlop, C. (2017a) The irony of epistemic learning: epistemic communities, policy learning and the case of Europe’s hormones saga, Policy and Society, 36(2): 21532. doi: 10.1080/14494035.2017.1322260

    • Search Google Scholar
    • Export Citation
  • Dunlop, C.A. (2017b) Policy learning and policy failure: definitions, dimensions and intersections, Policy & Politics, 45(1): 318. doi: 10.1332/030557316X14824871742750

    • Search Google Scholar
    • Export Citation
  • Dunlop, C.A. and Radaelli, C.M. (2013) Systematising policy learning: from monolith to dimensions, Political Studies, 61(3): 599619. doi: 10.1111/j.1467-9248.2012.00982.x

    • Search Google Scholar
    • Export Citation
  • Dunlop, C.A. and Radaelli, C.M. (2017) Learning in the Bath-tub: the micro and macro dimensions of the causal relationship between learning and policy change, Policy and Society, 36(2): 30419. doi: 10.1080/14494035.2017.1321232

    • Search Google Scholar
    • Export Citation
  • Fobé, E., Brans, M., Vancoppenolle, D. and Van Damme, J. (2013) Institutionalized advisory systems: an analysis of member satisfaction of advice production and use across 9 strategic advisory councils in Flanders (Belgium), Policy and Society, 32: 22540. doi: 10.1016/j.polsoc.2013.07.004

    • Search Google Scholar
    • Export Citation
  • GEMS (2020b) Advies Overlegcomité: Epidemiologische Situatie, Brussels: GEMS, https://d34j62pglfm3rr.cloudfront.net/celeval/GEMS_OCC_Epidemiological_update_measures_20201215.pdf.

    • Search Google Scholar
    • Export Citation
  • GEMS (2021a) Advies Overlegcomité: Contactberoepen, Brussels: Belgian Government, https://d34j62pglfm3rr.cloudfront.net/celeval/GEMS_OCC_Contactberoepen_20201215.pdf.

    • Search Google Scholar
    • Export Citation
  • Gerring, J. (2004) What is a case study and what is it good for?, The American Political Science Review, 98(2): 34154, https://www.jstor.org/stable/4145316. doi: 10.1017/S0003055404001182

    • Search Google Scholar
    • Export Citation
  • Haas, P. (1992) Introduction: epistemic communities and international policy coordination, International Organization, 46: 135. doi: 10.1017/S0020818300001442

    • Search Google Scholar
    • Export Citation
  • Happaerts, S. (2015) Climate governance in federal Belgium: modest subnational policies in a complex Multi-level setting, Journal of Integrative Environmental Sciences, 12(4): 285301. doi: 10.1080/1943815X.2015.1093508

    • Search Google Scholar
    • Export Citation
  • Heclo, H. (1974) Modern Social Politics in Britain and Sweden: From Relief to Income Maintenance, New Haven, CT: Yale University Press.

  • Hegele, Y. and Schnabel, J. (2021) Federalism and the management of the COVID-19 crisis: centralisation, decentralisation and (Non-)coordination, Western European Politics. doi: 10.1080/01402382.2021.1873529

    • Search Google Scholar
    • Export Citation
  • Heikkila, T., Weible, C.M. and Gerlak, A.K. (2020) When does science persuade (or not persuade) in High-conflict policy contexts?, Public Administration, 98(3): 53550. doi: 10.1111/padm.12655

    • Search Google Scholar
    • Export Citation
  • Het Laatste Nieuws (2020) Minister de crem fluit burgemeesters die te strenge coronarestricties oplegden terug, Het Laatste Nieuws (HLN) Online, 19 May, https://www.hbvl.be/cnt/dmf20200519_04965158.

    • Search Google Scholar
    • Export Citation
  • Hope, A. (2020) Leuven named European capital of innovation 2020, The Brussels Times, 24 September, https://www.brusselstimes.com/132644/leuven-is-named-european-capital-of-innovation-2020.

    • Search Google Scholar
    • Export Citation
  • Hoppe, R. (2017) Heuristics for practitioners of policy design: Rules-of-thumb for structuring unstructured problems, Public Policy and Administration, 33(4): 384408. doi: 10.1177/0952076717709338

    • Search Google Scholar
    • Export Citation
  • Hoppe, R. (2018) Rules-of-thumb for Problem-structuring policy design, Policy Design and Practice, 1(1): 1229. doi: 10.1080/25741292.2018.1427419

    • Search Google Scholar
    • Export Citation
  • Hornung, J., Bandelow, N.C. and Vogeler, C.S. (2019) Social identities in the policy process, Policy Sciences, 52: 21131. doi: 10.1007/s11077-018-9340-6

    • Search Google Scholar
    • Export Citation
  • Kamkhaji, J.C. and Radaelli, C.M. (2017) Crisis, learning and policy change in the European Union, Journal of European Public Policy, 24(5): 71434. doi: 10.1080/13501763.2016.1164744

    • Search Google Scholar
    • Export Citation
  • Kettl, D.F. (2020) States divided: the implications of American federalism for COVID‐19, Public Administration Review, 80(4): 595602. doi: 10.1111/puar.13243

    • Search Google Scholar
    • Export Citation
  • KU Leuven (2020) These scientists are tracing the source of corona infections among Leuven students, KU Leuven, 12 December, https://nieuws.kuleuven.be/en/content/2020/these-scientists-are-tracing-the-source-of-corona-infections-among-leuven-students.

    • Search Google Scholar
    • Export Citation
  • Kuhlmann, S., Hellström, M., Ramberg, U. and Reiter, R. (2021) Tracing divergence in crisis governance: responses to the COVID-19 pandemic in France, Germany and Sweden compared, International Review of Administrative Sciences, 87(3): 55675. doi: 10.1177/00208523209793

    • Search Google Scholar
    • Export Citation
  • Ladi, S. and Tsarouhas, D. (2020) EU economic governance and Covid-19: policy learning and windows of opportunity, Journal of European Integration, 42(8): doi: 10.1080/07036337.2020.1852231

    • Search Google Scholar
    • Export Citation
  • Lee, S., Hwang, C. and Moon, M.J. (2020) Policy learning and crisis Policy-making: quadruple-loop learning and COVID-19 responses in South Korea, Policy and Society, 39(3): 36381. doi: 10.1080/14494035.2020.1785195

    • Search Google Scholar
    • Export Citation
  • Leino, H. and Peltomaa, J. (2012) Situated knowledge – situated legitimacy: consequences of citizen participation in local environmental governance, Policy and Society, 31(2): 15968. doi: 10.1016/j.polsoc.2012.04.005

    • Search Google Scholar
    • Export Citation
  • Maggetti, M. (2020) Trust, coordination and multi-level arrangements: lessons for a European Health Union, European Journal of Risk Regulation, 11(4): 79099. doi: 10.1017/err.2020.97

    • Search Google Scholar
    • Export Citation
  • Maggetti, M. and Trein, P. (2018) Multilevel governance and Problem-solving: towards a dynamic theory of multilevel Policy-making?, Public Administration, 97(2): 35569. doi: 10.1111/padm.12573

    • Search Google Scholar
    • Export Citation
  • Mattei, P. and Del Pino, E. (2021) Coordination and health policy responses to the first wave of COVID-19 in Italy and Spain, Journal of Comparative Policy Analysis: Research and Practice, 23(2): 27481. doi: 10.1080/13876988.2021.1878886

    • Search Google Scholar
    • Export Citation
  • Mavrot, C. and Sager, F. (2018) Vertical epistemic communities in multilevel governance, Policy & Politics, 46(3): 391407. doi: 10.1332/030557316X14788733118252

    • Search Google Scholar
    • Export Citation
  • May, P.J. (1992) Policy learning and failure, Journal of Public Policy, 12(4): 33154. doi: 10.1017/S0143814X00005602

  • McConnell, A. and Stark, A. (2021) Understanding policy responses to COVID-19: the stars haven’t fallen from the sky for scholars of public policy, Journal of European Public Policy, 28(8): 111530. doi: 10.1080/13501763.2021.1942518

    • Search Google Scholar
    • Export Citation
  • Mei, C. (2020) Policy style, consistency and the effectiveness of the policy mix in China’s fight against COVID-19, Policy and Society, 39(3): 30925. doi: 10.1080/14494035.2020.1787627

    • Search Google Scholar
    • Export Citation
  • Moens, B. (2020) Not even coronavirus can unite Belgium, Politico, 12 March, https://www.politico.eu/article/coronavirus-belgium-division/.

    • Search Google Scholar
    • Export Citation
  • National Crisis Center (No Date) Coordination in an emergency, Nationaal Crisiscentrum, https://crisiscenter.be/en/what-do-authorities-do/crisis-management/coordination-emergency.

    • Search Google Scholar
    • Export Citation
  • NCCN (No Date) De vijf disciplines, Nationaal Crisiscentrum, https://crisiscentrum.be/nl/inhoud/de-vijf-disciplines.

  • Neij, L. and Heiskanen, E. (2021) Municipal climate mitigation policy and policy learning – a review, Journal of Cleaner Production. doi: 10.1016/j.jclepro.2021.128348.

    • Search Google Scholar
    • Export Citation
  • Nowell, B. and Albrecht, K. (2018) A reviewer’s guide to qualitative rigor, Journal of Public Administration Research And Theory, 34863. doi: 10.1093/jopart/muy052

    • Search Google Scholar
    • Export Citation
  • Pattyn, V., Blum, S., Fobe, E., Pekar-Milicevic, M. and Brans, M. (2019) Academic policy advice in Consensus-seeking countries: the cases of Belgium and Germany, International Review of Administrative Sciences, 117. doi: 10.1177/0020852319878780

    • Search Google Scholar
    • Export Citation
  • Pattyn, V., Matthys, J. and Van Hecke, S. (2021) High-stakes crisis management in the low countries: comparing government responses to COVID-19, International Review of Administrative Sciences, 87(3): 593611. doi: 10.1177/0020852320972472

    • Search Google Scholar
    • Export Citation
  • Popelier, P. (2020) COVID-19 legislation in Belgium at the crossroads of a political and a health crisis, The Theory and Practice of Legislation, 8(1–2): 13153. doi: 10.1080/20508840.2020.1771884

    • Search Google Scholar
    • Export Citation
  • Province of Antwerp (2020) Press release, Gouverneur Antwerpen, 26 August, https://www.gouverneurantwerpen.be/content/dam/cathy-berx/coronadocumenten/20200826_Persbericht%20toelichting%20epidemiologische%20situatie%20en%20de%20maatregelen%20in%20de%20strijd%20tegen%20het%20corona%20COVID-19%20virus%20in%20provincie%20Antwerpen.p.

    • Search Google Scholar
    • Export Citation
  • Radaelli, C. M. (2009) Measuring policy learning: regulatory impact assessment in Europe, Journal of European Public Policy, 16(8): 114564. doi: 10.1080/13501760903332647

    • Search Google Scholar
    • Export Citation
  • Raudla, R., Aleksandrs, C., Vytautas, K. and Rainer, K. (2018) Fiscal policy learning from crisis: comparative analysis of the baltic countries, Journal of Comparative Policy Analysis: Research and Practice, 20(3): 288303. doi: 10.1080/13876988.2016.1244947

    • Search Google Scholar
    • Export Citation
  • Rietig, K. and Perkins, R. (2017) Does learning matter for policy outcomes? The case of integrating climate finance into the EU budget, Journal of European Public Policy, 25(4): 487505. doi: 10.1080/13501763.2016.1270345

    • Search Google Scholar
    • Export Citation
  • Schomaker, R.M., Hack, M. and Mandry, A.K. (2021) The EU’s reaction in the first wave of the Covid-19 pandemic between centralisation and decentralisation, formality and informality, Journal of European Public Policy. doi: 10.1080/13501763.2021.1942153

    • Search Google Scholar
    • Export Citation
  • Sciensano (2021) COVID-19 wekelijks epidemiologisch bulletin (23 April 2021), Brussels: Sciensano, https://covid-19.sciensano.be/sites/default/files/Covid19/COVID-19_Weekly_report_NL.pdf.

    • Search Google Scholar
    • Export Citation
  • Stake, R.E. (1995) The Art of Case Study Research, London: Sage Publications.

  • Tavory, I. and Timmermans, S. (2012) Theory construction in qualitative research: from grounded theory to abductive analysis, Sociological Theory, 30(3): 16786. doi: 10.1177%2F0735275112457914

    • Search Google Scholar
    • Export Citation
  • The Brussels Times. (2020) Wallonia, Brussels, Flanders: Belgium’s Covid-19 state of play, 12 October, https://www.brusselstimes.com/135472/wallonia-brussels-flanders-belgiums-covid-19-state-of-play-leuven-ben-weyts-schools-sports-hospitality-bars-cafes-restaurants-figures-sciensano-steven-van-gucht-hainaut-liege-namur-luxembourg-walloon.

    • Search Google Scholar
    • Export Citation
  • The Brussels Times (2020a) Belgium in brief: Flanders falls behind, The Brussels Times, 26 October, https://www.brusselstimes.com/belgium/137635/belgium-in-brief-flanders-falls-behind-jan-jambon-virologist-steven-van-gucht-brussels-newest-measures-wallonia-curfew-average-infections-sciensano-public-health-institute-autumn-school-holiday/.

    • Search Google Scholar
    • Export Citation
  • The Brussels Times (2020b) Coronavirus: Belgium gets new group of expert advisors, The Brussels Times, 11 December, https://www.brusselstimes.com/news/belgium-all-news/health/144865/coronavirus-belgium-gets-new-group-of-expert-advisors-gees-celeval-consultative-committee-erika-vlieghe-yves-van-laethem-steven-van-gucht-marc-van-ranst/.

    • Search Google Scholar
    • Export Citation
  • Valcke, T., De Ceuninck, K., Reynaert, H. and Steyvers, K. (2008) Leadership, governance and legitimacy at an intermediate government level: the case of Belgian governors, Local Government Studies, 34(2): 24565. doi: 10.1080/03003930701852328

    • Search Google Scholar
    • Export Citation
  • Van Dooren, W. and Noordegraaf, M. (2020) Staging science: authoritativeness and fragility of models and measurement in the COVID‐19 crisis, Public Administration Review, 80(4): 61015. doi: 10.1111/puar.13219

    • Search Google Scholar
    • Export Citation
  • Van Overbeke, T. and Stadig, D. (2020) High politics in the low countries: COVID-19 and the politics of strained multi-level policy cooperation in Belgium and the Netherlands, European Policy Analysis, 6(2): 30517. doi: 10.1002/epa2.1101

    • Search Google Scholar
    • Export Citation
  • Walker, L. (2021a) Over one-third of French speakers in Belgium would refuse coronavirus vaccine, The Brussels Times, 14 May, https://www.brusselstimes.com/news/belgium-all-news/health/169499/over-one-third-of-french-speakers-in-belgium-would-refuse-coronavirus-vaccine-brussels-flanders-wallonia-uclouvain-ulb-ugent-motivatie-barometer/.

    • Search Google Scholar
    • Export Citation
  • Walker, L. (2021b) ‘Completely at odds with suggestions’: experts divided over Flanders’ summer plan, 10 May, https://www.brusselstimes.com/belgium/168776/completely-at-odds-with-suggestions-erika-vlieghe-marc-van-ranst-criticises-flanders-summer-plan-festivals-pukkelpop-tomorrowland-consultative-committee/.

    • Search Google Scholar
    • Export Citation
  • Wayenberg, E., Resodihardjo, S.L., Voets, J., Van Genugten, M., Van Haelter, B. and Torfs, I. (2022) The Belgian and Dutch response to COVID-19: change and stability in the mayors’ position, Local Government Studies, 48(2): 27190. doi: 10.1080/03003930.2021.1958787

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L. and George, B. (2021) New development: policy learning and public management – a match made in crisis, Public Money and Management, 42(2): 12932. doi: 10.1080/09540962.2021.1956212

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L. and Wayenberg, E. (2021) Shopping in the scientific marketplace: COVID-19 through a policy learning lens, Policy Design and Practice, 4(1): 1532. doi: 10.1080/25741292.2020.1843249

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Pattyn, V. and Wayenberg, E. (2022a) Policy learning mode shifts during creeping crises: a storyboard of COVID-19 driven learning in Belgium, European Policy Analysis. doi: 10.1002/epa2.1165

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Wayenberg, E. and George, B. (2022b) A systematic review of policy learning: tiptoeing through a conceptual minefield, Policy Studies Yearbook, 12: 152. doi: 10.18278/psy.12.1.2

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Nicoli, F., Wayenberg, E. and Verschuere, B. (2022c) Contagious inequality: economic disparities and excess mortality during the Covid-19 pandemic, Policy & Society, 41(2): 199216. doi: 10.1093/polsoc/puac011

    • Search Google Scholar
    • Export Citation
  • Zaki, B.L., Nicoli, F., Wayenberg, E. and Verschuere, B. (2022d) “In Trust we Trust?”: The impact of trust in government on excess mortality during the COVID-19 pandemic, Public Policy and Administration, 37(2): 22652. doi: 10.1177%2F09520767211058003

    • Search Google Scholar
    • Export Citation
Bishoy Louis ZakiUniversity of Ghent, Belgium

Search for other papers by Bishoy Louis Zaki in
Current site
Google Scholar
Close
and
Ellen WayenbergUniversity of Ghent, Belgium

Search for other papers by Ellen Wayenberg in
Current site
Google Scholar
Close

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 363 363 0
Full Text Views 669 669 232
PDF Downloads 419 419 133

Altmetrics

Dimensions