Introduction

The digital revolution has fundamentally reshaped the world in recent decades, making cyberspace and its key element, the internet, crucial to how economies, societies and politics operate. In this process, cyberspace has also become an object of security politics. The rise of cybersecurity – that is, activities geared towards anticipating, preventing and countering threats to users operating in and through cyberspace as well as to the underlying information technology (IT) infrastructure – has been undergirded by narratives that highlight the distinctiveness of cyberspace compared to other domains of security and emphasize the growing importance of developing political responses to cyber threats.

In this chapter, we explore the ecosystem of actors that produce and publish representations of threats and capabilities in cyberspace. How knowledge about cyber threats is produced and circulated is a question that has gained increasing attention in research on cybersecurity politics in recent years (see, for example, Dunn Cavelty, 2013; Stevens, 2020; Egloff and Dunn Cavelty, 2021; Maschmeyer et al, 2021; Slayton, 2021). This research has shed light on several facets of the use of comparative practices, such as the origins of computer risk management metrics (Slayton, 2015), the use of analogies and metaphors (Betz and Stevens, 2013; Branch, 2020) and big data analytics as an instrument used to identify actors that deviate from the usual patterns of activities (Aradau and Blanke, 2018; Shaurya and Singh, 2021). Little attention, though, has been paid to the production of knowledge about the evolving patterns of threats and capabilities in cyberspace. Yet, these comparative practices are just as crucial to cybersecurity politics, especially as they form the basis for arguments about which threats are trending and which states are improving their capabilities and enhancing their power. In particular, rankings of cyber threats and the cyber capabilities of states have become prominent tools through which actors seek to trigger policy developments and steer investments in cybersecurity resources.

Against this background, we analyse the ecology of publishers of comparative knowledge on threats and capabilities that has emerged in cyberspace. These publishers form three interrelated, yet distinct clusters: the first produces reports on the evolving patterns of cyber threats, the second evaluates the cybersecurity capacities of states and the third compares the cyber power of states. The production of comparative knowledge is much more datafied – that is, based on big data and data analytics in general – in the first cluster than in the other two. Drawing on sociological and IR approaches, we use the concept of ecosystems to tease out how the conditions of cyberspace have shaped the emergence of the ecology of publishers and help to explain differences in the comparative practices across the three clusters. The chapter thus contributes primarily to the first of the three themes highlighted in the introduction to this volume: how comparative knowledge is produced. Like other chapters (notably those of Jacobi and Herbst (Chapter 8), Bueger (Chapter 6), and Krause (Chapter 7)), it highlights the fragmented nature of the production of comparative knowledge and the ambiguity that this fragmentation entails. In addition, the chapter also provides insights into the other two themes: how comparative knowledge becomes politically relevant and how it (re)shapes politics. The three clusters produce representations of cybersecurity that reinforce and give substance to the narrative of a ‘fast evolving cyber threat landscape’ – to quote the 2016 Cyber Defence Pledge of the North Atlantic Treaty Organization (NATO, 2016) – that undergirds and shapes cybersecurity politics, which in turn generates further demands for comparative knowledge.

The chapter is structured as follows: after introducing our understanding of ecosystems and the ways they are represented through comparative practices, we map the ecology of publishers of comparative knowledge, distinguishing three clusters revolving around cyber threats, cybersecurity capacities and cyber power respectively. We then discuss two factors that help to explain the evolution of this ecology: the unequal distribution of relevant resources among the producers of representations and the effects of the struggle among states over the governance of the internet. While sharing a common narrative of a constantly evolving threat landscape, the three clusters differ in the logics of comparison they employ, resulting in dissimilar representations of the distribution of cyber capabilities.

Representational work in ecosystems

Is the volume of threats in cyberspace increasing, remaining stable or decreasing? Are the types of threats changing? Such questions can only be answered because there are actors that do representational work – that is, that produce abstract accounts of the patterns of threats and that successfully convince other actors that these accounts tell them something meaningful about the evolving state of cybersecurity. By doing so, the actors ‘present’ cyberspace in particular ways, emphasizing certain aspects while bracketing others (Bueger, 2015: 7). Like maps, the representations give actors an overview. But they also contribute to the construction of cyberspace as a governance object, that is, an issue that is deemed to have problematic aspects that require political action (see Allan, 2017). Comparative practices are fundamental to this representational work. To discern trends in the volume and types of threats over time, actors have to develop classifications of threats, collect data about threat incidents and analyse changes in the frequency of the threats – put differently, they have to assess similarities and differences between threats both at different moments in time and across these moments.

The starting point for our analysis of the role of comparative practices is the argument that cyberspace is not only the reference object of this representational work but also the social setting that shapes which actors do what forms of representational work. For this argument, we draw on a broad understanding of cyberspace as an ecosystem populated by a variety of actors that build, maintain use and/or seek to (re)shape the globalized network of computers and other digital technologies that has emerged in the last decades. Among these actors are tech companies, hackers, internet users, cybersecurity companies and various national, transnational and international governance institutions. The ecosystem metaphor is widely used among cybersecurity practitioners. The US Department of Homeland Security (2011: 2), for instance, described cyberspace in the following way: ‘Like natural ecosystems, the cyber ecosystem comprises a variety of diverse participants – private firms, non-profits, governments, individuals, processes, and cyber devices (computers, software, and communications technologies) – that interact for multiple purposes’. Practitioners, though, generally only use the term in a loosely defined sense to stress the diversity of actors involved in cyberspace and, relatedly, its complexity and the dynamic interplay between the actors, their practices and the technologies they use.

One key challenge of cybersecurity politics is the diversity of concepts and meanings. Different actors or communities of interests/expertise address the issue at hand with different emphases, normative evaluations and priorities. This has a significant impact on the way cybersecurity is assessed and the ecosystem is mapped (see also Calderaro and Craig, 2020: 920). Moreover, it has implications for the definition of cyber threats, the question of which threat perceptions are prioritized, which capacity-building elements are in the limelight and which sources of cyber power are used as points of references in comparisons. Put differently: it matters which actors do the representational work.

Two discussions in IR are particularly productive for giving the metaphor more substance and adapting it for the analysis of the representational work. The first is about organizational ecologies in world politics. The metaphor of ecosystems directs analytical attention to how environments shape the populations of actors that live in them. Ecological theorizing tries to explain why some populations thrive while others do not. In organizational ecology, the populations are different types of organizations (see Hannan and Freeman, 1989). Such theorizing has recently been applied to IR to analyse why the number of non-state actors involved in global governance activities is growing while the number of international organizations stagnates. Several factors are postulated: some organizations have institutional features that give them advantages over others. Non-state actors notably do not require negotiations among states to set up governance arrangements. Moreover, there are dynamics related to organizational density: the more organizations of one type exist, the easier it is for them to legitimate their activities vis-à-vis their environment, but at the same time the more intense their competition over valued resources becomes (Abbott et al, 2016). In this competition, organizations seek to find niches – for example, new governance domains and tasks – which allow them to thrive. Furthermore, there are interactional dynamics – so-called ‘regulatory processes’ – in play: ‘positive regulation’ in which actions by one type of organization enable activities by others types in a niche, ‘negative regulation’ in which the activities of one type of organization make it harder for other types to establish themselves in that niche and ‘double-negative regulation’ in which the activities of one type of organization prevent another type from establishing itself in a niche, which in turn leaves that niche open for a third type of organization (Lake, 2021: 349). Organizational ecology is in this sense not only about which type of organization is more numerous but also – and this is the more important aspect for present purposes – about the interplay of the activities of different types of organizations and the different governance arrangements that this interplay brings about.

The second discussion relates to the ecologies of indicators. The last three decades have witnessed a proliferation of quantitative forms of representation such as indices or rankings in many policy domains in world politics. A growing literature seeks to explain this proliferation and its effects (see Broome and Quirk, 2015; Kelley and Simmons, 2019; Rumelili and Towns, 2022). One explanation developed in this literature emphasizes the ‘self-reinforcing’ dynamics of the ‘ecology of indicators’: ‘as more indicators are produced, aggregations of indicators become more reliable, more indicators are used, more indicators are produced, and so on’ (Davis et al, 2012: 85). This explanation actually fuses two arguments. First, it postulates an enabling dynamic similar to the ‘positive regulation’ process mentioned earlier. The representational work already done by some actors facilitates and enables the representational work that other actors want to do – provided that the former actors make their representations available to the latter actors, for instance by publishing them. Second, it suggests that this enabling dynamic fuels an increasing use and production of representations in a given domain. Put differently: the representational work not only becomes easier, it also becomes more prevalent.

What the indicators literature has not yet discussed is what dampens this dynamic. Building on the organization ecology literature, the assumption would be that organizations produce and publish representations because they deem this representational work conducive to their success in the competition over resources such as public attention, market shares or political influence. In this logic, the proliferation of representations would slow down once organizations came to regard the production and publication of representations as no longer giving them advantages in the competition over resources.

Thus conceptualized, an ecosystem perspective helps to analyse and explain the conditions and dynamics that shape which organizations do which representational work in cyberspace. As the first step in the analysis, the next section identifies and maps three distinct, though interrelated, clusters of producers of representations.

Three clusters of representational work

In the last three decades, cybersecurity has morphed from a solely technical issue of securing computer networks into a political issue of promoting security in cyberspace. In this process, the prevalent understanding of cyber threats has broadened to encompass not only – as initially – crimes committed in computer networks but also attacks on critical infrastructures as well as cyber conflicts in which states come under attack by other states or non-state actors (see Carr, 2021: 54–7). In this process, cybersecurity has evolved into a broad field of application, incorporating technical, legal and organizational measures, with more cooperation across the public/private divide. Cyber threats assessments, in turn, have broadened beyond patterns of cyberattacks to include a wide range of aspects, for example the level or lack of technological and legal enforcement assets, privacy and data protections, threat intelligence exchange formats and infrastructural gaps.

As part of this process, a growing number of organizations have started to publish – some regularly, others irregularly – representations of various aspects of cybersecurity, ranging from overviews of trends in cyber threats through surveys of cybersecurity sentiments and estimations of the costs caused by cyberattacks to comparisons of cyber capabilities. The indicators and data collection methodologies vary across the different attempts to produce comparative knowledge, which has also led to a discourse on the value and shortcomings of the representations (see, for instance, Yarovenko et al, 2020).

In what follows, we do not aim to map the ecology of all of these organizations but focus more narrowly on three aspects – patterns of threats, cybersecurity capacities and cyber power – that are at the heart of cybersecurity politics. Interrelated but nonetheless distinct clusters of producers of representations have emerged for each of these three aspects: first, from the 2000s onwards, a cluster mapping and tracking of the patterns of threats based on a more comprehensive, datafied understanding of these patterns than previous computer risk management metrics (for these see Slayton, 2015), then in the 2010s a cluster evaluating the cybersecurity capacities of states and, in the last few years, a cluster developing representations of the distribution of cyber power. We discuss each of these clusters – summarized in Table 9.1 – in turn.

The threats cluster is both the oldest of the three clusters and the one with the highest density of organizations. Many companies active in the cybersecurity market publish some sort of statistics about the volume of and trends in cyber threats. A number of these companies, including big tech companies such as IBM and Microsoft and companies specializing in cybersecurity services such as CrowdStrike, FireEye Mandiant, Kaspersky and Symantec, regularly issue reports on the evolving patterns of threats. Cybersecurity companies publish statistics on the patterns of threats in order to secure valued resources, such as more customers and a reputation as cybersecurity experts in public debates. They dominate the cluster because they have a decisive advantage over other types of organization. By providing cybersecurity services to a large number of private, commercial and also public customers, they operate expansive networks of digital sensors and often state-of-the-art analytical tools that allow them to amass the key resource for statistical overviews of cyber threats: data on incidents – such as ‘indicators of compromise’ (IoCs) and ‘indicators of attack’ (IoAs)1 – compiled through the monitoring, recording and aggregating of malicious activity from the open, deep and dark web.

Organizations such as research institutes, think tanks or international organizations lack such networks of sensors deployed to numerous endpoints. Hence, most of these organizations do not have access either to the same amount of aggregated threat intelligence data or the respective analytical tools. They have dealt with this unfavourable setting in two ways. The first is to use the available quantitative and qualitative data published by cybersecurity companies as a base for producing their own representations. The European Union Agency for Cybersecurity (ENISA) has chosen this way. Since 2012 it has regularly published ‘Thread Landscape’ reports in which it tracks changes in cyber threats and ranks them according to their prevalence. In 2020, for instance, it listed malware, web-based attacks, phishing, web application attacks and spam as the top five cyber threats. In addition to such classifications of cyber threats, mostly based on quantifiable data, ENISA also produces detailed reports on specific cyber threats such as ransomware attacks. The second way is to compile their own incident databases. The US-based Center for Strategic and International Studies (CSIS), for instance, has maintained a list of ‘Significant Cyber Incidents’ on its website since 2015 (see CSIS, 2023). Another US-based think tank, the Council on Foreign Relations (CFR), compiles a list of state-sponsored cyber operations (see CFR, 2023).

The cybersecurity capacity cluster consists of organizations that evaluate the cybersecurity capacities of states, that is, their defensive cyber capabilities. In 2007 the Secretary-General of the International Telecommunication Union (ITU), the United Nations’ specialized agency for information and communication technologies, launched the Global Cybersecurity Agenda to promote cybersecurity efforts worldwide. The ITU translated the five working areas of the Global Cybersecurity Agenda – namely legal measures, technical measures, organizational structures, capacity building and international cooperation – into a five-dimensional framework of indicators. The resulting ranking, the Global Cybersecurity Index (GCI), has so far been published in four editions: the first in 2015, the second in 2017, the third in 2019 and the fourth, which evaluates 194 countries, in 2021 (see ITU, 2015; 2017; 2019; 2021). The governance niche also attracted other organizations. Among the most prominent: the US-based Potomac Institute proposed a Cyber Readiness Index in 2013 and published a revised version in 2015, while the Global Cyber Security Capacity Centre (GCSCC) of the University of Oxford developed a Cybersecurity Capacity Maturity Model for Nations (CCM) in 2014 which it has since revised twice (see GCSCC, 2021). Both organizations designed their comparative frameworks as multi-dimensional benchmarking tools meant to guide states in their cybersecurity capacity development. What sets them apart from the CGI is that they did not aggregate the benchmarking scores into overall rankings. The e-Governance Academy (no date) developed another ranking, the National Cyber Security Index (NCSI), which covers about 160 states and is distinct from the periodically published CGI in that it has been updated constantly since its launch in 2019. The e-Governance Academy is a non-profit foundation jointly created by the Estonian government, the Open Society Institute and the United Nations Development Programme.

The most recent of the three clusters is the cyber power cluster. This cluster goes beyond the cybersecurity capacity cluster by considering and comparing both the defensive and offensive cyber capabilities of states. Debates about cyber power and cyber powers have been going on for some time. In 2011, notably, a Cyber Power Index for the 19 state members of the G20 was published by the Economist Intelligence Unit in cooperation with Booz Allen Hamilton, but this index covered only defensive cyber capabilities. Fully developed comparative frameworks considering both defensive and offensive capabilities have only been published in the last few years, one by a research institute and another by a think tank. In 2020 Harvard University’s Belfer Center presented a National Cyber Power Index which, based on 32 intent indicators and 27 capability indicators, quantitatively ranks 30 states according to their cyber power (see Voo et al, 2020b). In 2021 the International Institute for Strategic Studies (IISS) published a qualitative net assessment in which it sorted 15 states into three tiers according to their relative cyber power (see IISS, 2021).

Table 9.1:

Three clusters of representational work in cybersecurity

Cluster Emergence Prevalent organizations
Threats 2000s Cluster dominated by cybersecurity companies, including
• Microsoft (Security Intelligence Report/Digital Defense Report, published since 2005)
• Kaspersky (Security Bulletin, published since at least 2007)
• FireEye Mandiant (M-Trend reports, published since 2011)
One international organization, the EU (via ENISA), aggregates such reports to produce ‘Thread landscape’ reports (since 2012)
Cybersecurity capacity 2010s Cluster features a diverse cast of organizations producing comparative frameworks:
• Cyber Readiness Index (Potomac Institute, first version 2013, second version 2015)
• Cybersecurity Capacity Maturity Model for Nations (GCSCC, University of Oxford, UK, launched in 2014, revised in 2016 and 2021)
• Global Cybersecurity Index (GCI, four editions published by ITU so far in 2015, 2017, 2019 and 2021)
• National Cyber Security Index (e-Governance Academy, Estonia, produced since 2016)
Cyber power late 2010s/early 2020s Cluster still in formation, with representations published so far by a research institute and a think tank:
• National Cyber Power Index (Belfer Center, Harvard University, US, published in 2020)
• Cyber Capabilities and National Power: A Net Assessment (IISS, UK, published in 2021)

The factors shaping the co-evolution of the three clusters

An ecosystem perspective suggests two factors that explain why the three clusters differ in their mix of organizations and why certain types of organization, and not others, dominate the representational work on the patterns of threats, the worldwide levels of cybersecurity capacity and the distribution of cyber power respectively. The first factor is unequal resources; the second, political struggles that prevent international organizations from occupying and dominating some of the clusters. These factors account for the dominance of cybersecurity companies in the first cluster, the absence of UN and ITU activities in the first and third cluster as well as the facilitating role that the first cluster plays for the third cluster.

Unequal resources

As already briefly mentioned, private software and hardware companies like Microsoft, IBM and Intel or cybersecurity companies such as Deepwatch, Fireye Mandiant, Infosec, Kaspersky and Palo Alto Networks have a special resource that most other organizations – apart from the intelligence services of some cyber powers – lack. Their networks of digital sensors give them a privileged and in many respects exclusive access to incident-level data on cyber threats, which in turn makes them the key gatekeepers to knowledge about cyber threats. They selectively share this knowledge with a wider audience through various channels including reports, statistics, working groups or expert hearings.

Civil society actors like non-governmental organizations (NGOs) often lack the resources to extensively collect data. This creates dependencies either on the publicity of data or on other actors producing data and statistical indicators which they can then use for the development of their own representations. They can, though, partly compensate for this disadvantage by developing analytical frameworks and data process methodologies based on open-source intelligence practices. Prominent examples are the CFR’s ‘Cyber Operations Tracker’ and the CSIS’s list of ‘Significant Cyber Incidents’ which both monitor cyber operations based on publicly assessable data. An example of a non-state actor using open-source intelligence to map the patterns of capabilities is the British NGO Privacy International. In its report The Global Surveillance Industry (Privacy International, 2016), it reconstructed the surveillance industry in five states (Germany, Israel, Italy, the UK and the US), tracking how private companies sell surveillance technologies to state actors.

The struggle over internet governance

All three clusters produce comparative knowledge in an ecosystem shaped by political struggles. These struggles have existed since the early days of the internet (see Mueller, 2017). A key point of contention is the nature of the governance of the internet. The US and the EU prefer a multi-stakeholder model in which various actors – including states, private companies and international organizations – partake in the management of an open internet. As the internet was created mainly by actors from the West, its governance mostly resembles this model. However, states such China and Russia lobby for a different model, one based on the principle of cyber sovereignty and the control of states over the internet. The debate over the governance of the internet thus features two competing camps, one advocating a liberal model, the other a sovereigntist model (see Flonk et al, 2020 and Price, 2018).

The struggles involve not only questions about the nature of governance, but also disputes over the technical infrastructure of the internet, as these have implications for how the internet works and can be controlled. The technical dimension is sometimes overlooked, but it is crucial to how open the internet is and how information is exchanged. Its bases are globally standardized data communication protocols. Internet Protocol (IP) addresses are crucial to the global internet expansion, but IP addresses are not an infinite resource. This regularly triggers controversial debates on new technical standards and management frameworks (Denardis, 2009: 1–3). The proposal for a new top-down internet protocol ‘New IP, Shaping Future Networks’, put forward by a Huawei-led group in the ITU in 2019, is one of the latest examples in a series of efforts to change the way the internet works in the name of making cyberspace fit for the high pace of the digital transformation and the integration of emerging technologies (see Murgia and Gross, 2020).

This struggle also affects cybersecurity politics. The Budapest Convention, signed in 2001, is a key framework document for the struggle against cybercrime. The convention was negotiated under the aegis of the Council of Europe but is open to all states (see Holder, 2022). China and Russia, though, are seeking to supersede it with a new cybercrime treaty and they have succeeded in convincing a majority of emerging and developing countries to join their endeavour. UN members are currently negotiating a possible UN convention on cybercrime. Western states fear that such a convention could, if cybercrimes are defined too broadly and data privacy and human rights are not adequately protected, help authoritarian regimes expand their control over cyberspace, thus undermining rather than strengthening the security of individuals (see for instance European Data Protection Supervisor, 2022). The struggles have made it difficult for international organizations such as the UN and the ITU, despite having mandates for (cyber) security governance, to establish an epistemic infrastructure tracking patterns of cyber threats or evaluating the cyber power of states.

Moreover, the struggles are part of a broader geopolitical struggle in which the US and other Western states compete with China and Russia over power in and over the international order. This geopolitical struggle has become more intense in the past decade, making questions of relative cyber power more relevant politically. The struggles have therefore not only prevented international organizations such as the UN and the ITU from positioning themselves as key knowledge producers in the three clusters. They have also increased the demand for comparisons of cyber power. The Belfer Center and the IISS have moved to occupy the resulting niche.

Enabling effects with side effects

The three clusters differ in the representations that they produce. Some clusters, though, have enabling effects on the work of other clusters. In particular, the first cluster facilitates the production of comparative knowledge in the third. The lists of cyber incidents compiled by the CFR and the CSIS were used by the Belfer Center and the IISS as a source for their own assessments of the cyber power of states. The Belfer Center drew on the CFR’s list to discern the objectives that states pursue in offensive cyber operations, which then informed the design of its indicator framework for the measurement of the distribution of cyber power (Voo et al, 2020a: 6). In addition, the CSIS’s list serves as the basis for its count of ‘state-based cyber attacks’, which in turn forms part of its capability indicators (Voo et al, 2020b: 61). The IISS (2021: 129) in turn used the CSIS’s list as one of its sources.

Put differently: some clusters depend on the work of other clusters. One corollary is that the clusters that draw on the other clusters carry over biases inherent in the latter’s work. To continue with the example of the CFR’s and CSIS’s lists: the estimated number of state-sponsored cyber operations is most likely higher and covers more countries than these lists suggest, but accurate data is collected and shared only within the community of intelligence services or between specific allies. The CFR and CSIS, in other words, face resource constraints that might lead to certain biases in their representation of the patterns of incidents. What is more, the perception of cyber threats remains a contested political issue and takes different meanings depending on the community or actor addressing the issue. The lists of the two organizations are informed by Western perceptions. It is therefore not surprising that their lists repeat and feed into the narrative set out in Western national security reports, according to which non-Western states – especially China, Russia, Iran and North Korea – are the states that most frequently conduct offensive cyber operations (see Figure 9.1).

A column chart plots units versus countries for the number of state-sponsored cyber operations by country from 2005 to 2020. The highest value is for China at 153 and the lowest value is 1 for 19 countries including Canada, Mexico, Panama, Netherlands, France, Spain, and Togo.
Figure 9.1:

Number of state-sponsored cyber operations by country, 2005–2020

Source: See CFR (2023)

A common threat narrative, but dissimilar logics of comparisons

The dynamics of the ecosystem not only influence the co-evolution of the three clusters and the mix of organizations in each, they also shape the comparative approaches pursued in each of them. While all three clusters share a common narrative of a constantly evolving threat landscape, this narrative is the product of the comparative practices of the first cluster and the background for the comparative practices of the other two. Though interrelated, their representational work is nonetheless distinct, with niche logics fostering disparate logics of comparisons. We discuss these different logics, summarized in Table 9.2, in this section.

Table 9.2:

Three distinct logics of comparison

First cluster: threats Second cluster: cybersecurity capacity Third cluster: cyber power
Comparisons serve to analyse Prevalence of different threats Cybersecurity capacity levels Cyber power differentials
Objective of representational work Guide development of more capable cybersecurity measures Create a comparative dynamic fostering the diffusion among states of best practices in cybersecurity Help policy makers navigate the interstate competition in cyberspace by clarifying the nature and distribution of cyber power
Data sources Digital sensors/ telemetry Questionnaires, strategy documents, statistical databases Questionnaires, strategy documents, statistical databases
Comparative approach Big data analysis Multi-dimensional frameworks of indicators Multi-dimensional frameworks of indicators

The organizations dominating the first cluster have the resources to compile and analyse huge amounts of data on cyber incidents. They often use figures for detected or blocked attacks to showcase their cybersecurity capabilities. Kaspersky’s 2021 report, for instance, notes that its cybersecurity tools ‘blocked 687,861,449 attacks launched from online resources across the globe’ between November 2019 and October 2021 (Kaspersky, 2021) while Microsoft’s 2021 report highlights that its tools blocked 9 billion ‘endpoint threats’, 31 billion ‘identity threats’ and 32 billion ‘email threats’ between July 2020 and June 2021 (Microsoft, 2021: 4). At the centre of the reports, however, is usually not an analysis of trends in the overall volume of cyber incidents but the disaggregation of the incident data into different types of threat and the discussion of the characteristics of and trends in those types. Put differently: the narrative that the reports want to sell is less a general ‘cyber incidents are on the rise’ story, rather a more differentiated story about which cyber threats are becoming more prevalent and dangerous and which less so. Underlying this story is a portrayal of cybersecurity as an ongoing contest between ‘defenders’ and ‘attackers’, in which the defenders learn to counter certain threats, the attackers in reaction seek new ways to achieve their aims, which in turn forces the defenders to step up their cybersecurity activities, and so on (see, for example, Microsoft, 2021: 5). The narrative, in short, is one of a constantly evolving threat landscape. ENISA buys into this narrative by structuring its aggregated overviews of the patterns of cyber treats in terms of top threats.

The first cluster’s representational work is not the only factor shaping the discourse on cybersecurity. Similarly important are episodes such as the cyberattack on Estonia in 2007, the Stuxnet attack on Iran’s nuclear programme in 2010, the Snowden revelations in 2013 or the Russian interference in the US elections in 2016 that prompted public and political debates about the changing nature of cyber threats and the best ways to deal with them. Nonetheless, it is the first cluster’s representational work that provides much of the publicly available knowledge on the patterns of threats in cyberspace. In so doing, it feeds and shapes the narrative of an evolving threat landscape that informs the representational work in which the other two clusters engage.

The organizations forming the second cluster seek to motivate and steer efforts by states to improve their cybersecurity capacities. They tend to leave the representation of the patterns of threats to others – and thus mainly to the first cluster – and instead seek to position themselves in the governance niche of cybersecurity capacity building. The ITU’s ranking publications are a case in point. The publications briefly highlight some threat statistics at the beginning to underscore the importance of cybersecurity, but the ITU’s own representational work centres on developing and updating a ranking of the cybersecurity capacities of states. The rationale is to promote best practices in cybersecurity. The Global Cybersecurity Index (GCI) is meant to provide ‘the right motivation to countries to intensify their efforts in cybersecurity’ (ITU, 2015: iii) and to serve as a ‘useful capacity development tool’ that identifies ‘areas for improvement’ and highlights ‘best practices for strengthening national cybersecurity’ (ITU, 2021: iv). The University of Oxford’s GCSCC (2021: 2) similarly describes the purpose of its benchmarking framework as to help ‘nations understand what works, what does not work and why, across all areas of cybersecurity’. As part of this positioning strategy, both organizations note that states have been making progress in improving their cybersecurity capacities and seek to portray themselves as facilitators and shapers of this progress while emphasizing the need for further capacity-building efforts ‘in the face of changing threats’ (GCSCC, 2021: 2; see also ITU, 2021: iv). The e-Governance Academy (no date) uses a different strategy for positioning itself in the governance niche. Rather than stressing best practices, it designs its ranking as a ‘global live index’ meant as ‘a comprehensive cyber security measurement tool that provides accurate and up-to-date public information about national cyber security’. In line with this objective, instead of publishing its ranking periodically, as the ITU does, it constantly updates it on its website.

The third cluster has emerged in reaction to the absence of cyber power rankings in the debate about offensive cyber activities among states. The organizations in this sense seek to fill a niche created by political developments but not so far occupied by other organizations. They build on the general narrative of an evolving threat landscape. The IISS (2021: 171), for instance, emphasizes the ‘rapidly evolving nature of cyber threats and opportunities’. However, their perspective on cyber threats is narrower than that of the other two clusters as they focus on a subset of cyber threats: attacks by states, or state-sponsored groups, on other states. The Belfer Center and the IISS not only invoke these attacks to underscore the relevance of their representational work but – as mentioned – also use them as empirical material for the development of their comparative frameworks. Differing from the second cluster, the logic of comparison is not primarily geared towards helping states improve their cybersecurity capabilities but towards teasing out power differentials in order to help policy makers navigate the interstate competition in cyberspace. While they integrate into their representations some of the indicators developed by the organizations in the second cluster, the Belfer Center and the IISS situate their representations in the practice of measuring power and classifying powers that has been part of great power politics for centuries. Consistent with this practice, they focus on the states deemed to be most important, which distinguishes them again from the representational work done in the second cluster which seeks to cover the cybersecurity capacities of all states.

Resource constraints partly explain why only the first cluster is characterized by a strong datafication of the comparative practices. The differences in the comparative practices are, however, also the result of niche strategies, with the producers of comparative knowledge in both the second and third cluster seeking to position themselves in niches that are distinct from the first cluster dominated by cybersecurity companies. The diverging niche logics – fostering worldwide cybersecurity capacity building versus understanding interstate competition in cyberspace – go a considerable way towards explaining the differences in the logic of comparison of the second and third cluster. As Table 9.3 shows, the dissimilar logics lead to differing representations of the distribution of cyber capabilities. Only 13 of the GCI’s top 20 states appear in the National Cyber Power Index (NCPI) and only 9 were included by the IISS in its Net Assessment. Only 6 of the NCSI’s top 20 states appear in the NCPI and only 3 in the IISS’ Net Assessment. China, to highlight the most prominent example, is absent from the top 20 of the two cybersecurity capacity rankings but appears in the top 10 of the two power rankings. A closer look at Table 9.3, however, also reveals that the niche dynamics have so far not fostered a common comparative approach within the two clusters. Neither the two cybersecurity capacity rankings nor the two cyber power rankings depict the same order of states. The cybersecurity capacity rankings, though, diverge more strongly (with only 8 states appearing in both top 20 lists) than the cyber power rankings (with 9 states in both top 15s). One explanation would be that the competition over attention and influence begets differentiation – the organizations seek to produce representations that differ from those of their competitors – which in turn translates into diverse representations and contributes to the continuing ambiguity of the distribution of cyber capabilities.

Table 9.3:

A comparison of the top 20 states in four prominent rankings

Rank Global Cybersecurity Index 2021 National Cyber Security Index 2022 National Cyber Power Index 2020 IISS Net Assessment

2021
1 US Greece US US the sole first-tier state
2 Great Britain Lithuania China Seven second-tier states (listed alphabetically):

Australia, Canada, China, France, Great Britain, Israel and Russia
3 Saudi Arabia Belgium Great Britain
4 Estonia Czech Republic Russia
5 South Korea Estonia Netherlands
6 Singapore Germany France
7 Spain Portugal Germany
8 Russia Spain Canada
9 UAE Poland Japan Seven third-tier states (listed alphabetically):

India, Indonesia, Iran, Japan, Malaysia, North Korea, Vietnam
10 Malaysia Finland Australia
11 Lithuania France Israel
12 Japan Sweden Spain
13 Canada Denmark Sweden
14 France Saudi Arabia Estonia
15 India Croatia New Zealand
16 Turkey Slovakia South Korea
17 Australia Netherlands Switzerland
18 Luxembourg Malaysia Singapore
19 Germany Italy Malaysia
20 Portugal US Vietnam

Note: The most recent version of each ranking was used. In the GCI, some states share the same ranks, which the table indicates through merged cells.

Conclusion

Cybersecurity politics features many comparative practices. In this chapter, we have shown that the widespread notion of cyberspace as an ecosystem can be analytically productive to explain how comparative knowledge is produced on three key aspects of cybersecurity politics: the patterns of cyber threats, the cybersecurity capacities of states as well as the distribution of cyber power. An ecosystem approach helps to tease out how a combination of three factors – (1) differences in resources, (2) political struggles preventing stronger roles for international organizations and fostering demands for different kinds of comparative knowledge and (3) strategies to carve out distinct niches of cybersecurity expertise – has given rise to three clusters of representational work populated by different types of organizations and characterized by different logics of comparison.

In addition to shedding light on how social settings shape the production of comparative knowledge, the chapter also probes into how comparative knowledge becomes politically relevant. The three clusters share a common threat narrative that emphasizes constantly changing patterns of threats and thus both feeds and legitimizes demands for political efforts to improve cybersecurity capabilities. The ecosystem approach highlights how organizations take advantage of these demands by strategically positioning themselves in two niches of cybersecurity politics through the publication of comparative frameworks: the debate about cybersecurity capacity building and the debate about interstate competition in cyberspace.

Comparative practices have political effects. The three clusters sustain a threat narrative that legitimizes demands for more cybersecurity activities. Cybersecurity politics constitute a promising case study for probing deeper into the effects of comparisons in future research. Both the cybersecurity capacity cluster and the cyber power cluster feature organizations that opt for quantitative comparative frameworks as well as organizations that opt for qualitative ones. The relative success of these organizations will thus provide insights into the impact that different comparative practices generate.

1

IoCs are signs of an attack such as login anomalies or suspicious file changes while IoAs are clues suggesting that an attack is planned.

References

  • Abbott, K.W., Green, J.F. and Keohane, R.O. (2016) ‘Organizational ecology and institutional change in global governance’, International Organization, 70(2): 24777.

    • Search Google Scholar
    • Export Citation
  • Allan, B.B. (2017) ‘Producing the climate: states, scientists, and the constitution of global governance objects’, International Organization, 71(1): 13162.

    • Search Google Scholar
    • Export Citation
  • Aradau, C. and Blanke, T. (2018) ‘Governing others: anomaly and the algorithmic subject of security’, European Journal of International Security, 3(1): 121.

    • Search Google Scholar
    • Export Citation
  • Betz, D.J. and Stevens, T. (2013) ‘Analogical reasoning and cyber security’, Security Dialogue, 44(2): 14764.

  • Branch, J. (2020) ‘What’s in a name? Metaphors and cybersecurity’, International Organization, 75(1): 3970.

  • Broome, A. and Quirk, J. (2015) ‘Governing the world at a distance: the practice of global benchmarking’, Review of International Studies, 41(5): 81941.

    • Search Google Scholar
    • Export Citation
  • Bueger, C. (2015) ‘Making things known: epistemic practices, the United Nations, and the translation of piracy’, International Political Sociology, 9(1): 118.

    • Search Google Scholar
    • Export Citation
  • Calderaro, A. and Craig, A. (2020) ‘Transnational governance of cybersecurity: policy challenges and global inequalities in cyber capacity building’, Third World Quarterly, 41(6): 91738.

    • Search Google Scholar
    • Export Citation
  • Carr, M. (2021) ‘A political history of cyberspace’, in P. Cornish (ed) The Oxford Handbook of Cyber Security, Oxford: Oxford University Press, pp 4966.

    • Search Google Scholar
    • Export Citation
  • CFR (Council on Foreign Relations) (2023) Cyber Operations Tracker, www.cfr.org/cyber-operations/ [Accessed 4 December 2023].

  • CSIS (Center for Strategic and International Studies) (2023) Significant Cyber Incidents, www.csis.org/programs/strategic-technologies-program/significant-cyber-incidents [Accessed 4 December 2023].

  • Davis, K.E., Kingsbury, B. and Merry, S.E. (2012) ‘Indicators as a technology of global governance’, Law & Society Review, 46(1): 71104.

    • Search Google Scholar
    • Export Citation
  • Denardis, L. (2009) Protocol Politics: The Globalization of Internet Governance, Cambridge, MA: The MIT Press.

  • Dunn Cavelty, M. (2013) ‘From cyber-bombs to political fallout: threat representations with an impact in the cyber-security discourse’, International Studies Review, 15(1): 10522.

    • Search Google Scholar
    • Export Citation
  • Egloff, F.J. and Dunn Cavelty, M. (2021) ‘Attribution and knowledge creation assemblages in cybersecurity politics’, Journal of Cybersecurity, 7(1): 112.

    • Search Google Scholar
    • Export Citation
  • e-Governance Academy (no date) National Cyber Security Index, https://ncsi.ega.ee/ [Accessed 11 February 2022].

  • European Data Protection Supervisor (2022) Opinion 9/2022 on the Recommendation for a Council Decision Authorising the Negotiations for a Comprehensive International Convention on Countering the Use of Information and Communications Technologies for Criminal Purposes, 18 May, https://edps.europa.eu/system/files/2022-05/2022-05-18-opinion_on_international_convention_en.pdf [Accessed 10 May 2023].

    • Search Google Scholar
    • Export Citation
  • Flonk, D., Jachtenfuchs, M. and Obendiek, A. (2020) ‘Authority conflicts in internet governance: liberals vs. sovereigntists?’, Global Constitutionalism, 9(2): 36486.

    • Search Google Scholar
    • Export Citation
  • GCSCC (Global Cyber Security Capacity Centre) (2021) Cybersecurity Capacity Maturity Model for Nations, https://gcscc.ox.ac.uk/files/cmm2021editiondocpdf [Accessed 11 February 2022].

    • Search Google Scholar
    • Export Citation
  • Hannan, M.T. and Freeman, J. (1989) Organizational Ecology, Cambridge: Cambridge University Press.

  • Holder, M. (2022) ‘Cyberspace in a state of flux: regulating cyberspace through international law’, Groningen Journal of International Law, 9(2): 26680.

    • Search Google Scholar
    • Export Citation
  • IISS (International Institute for Strategic Studies) (2021) Cyber Capabilities and National Power: A Net Assessment, 28 June, www.iiss.org/blogs/research-paper/2021/06/cyber-capabilities-national-power [Accessed 11 February 2022].

    • Search Google Scholar
    • Export Citation
  • International Telecommunication Union (ITU) (2015) Global Cybersecurity Index and Cyberwellness Profiles, Geneva: International Telecommunication Union.

    • Search Google Scholar
    • Export Citation
  • ITU (2017) Global Cybersecurity Index 2017, Geneva: International Telecommunication Union.

  • ITU (2019) Global Cybersecurity Index 2018, Geneva: International Telecommunication Union.

  • ITU (2021) Global Cybersecurity Index 2020: Measuring Commitment to Cybersecurity, Geneva: International Telecommunication Union.

  • Kaspersky (2021) Kaspersky Security Bulletin 2021. Statistics, 15 December. https://securelist.com/kaspersky-security-bulletin-2021-statistics/105205 [Accessed 11 February 2022].

    • Search Google Scholar
    • Export Citation
  • Kelley, J.G. and Simmons, B.A. (2019) ‘Introduction: the power of global performance indicators’, International Organization, 73(3): 491510.

    • Search Google Scholar
    • Export Citation
  • Lake, D.A. (2021) ‘The organizational ecology of global governance’, European Journal of International Relations, 27(2): 34568.

  • Maschmeyer, L., Deibert, R. and Lindsay, J.R. (2021) ‘A tale of two cybers – how threat reporting by cybersecurity firms systematically underrepresents threats to civil society’, Journal of Information Technology & Politics, 18(1): 120.

    • Search Google Scholar
    • Export Citation
  • Microsoft (2021) Digital Defense Report, October, https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RWMFIi?id=101738 [Accessed 11 February 2022].

    • Search Google Scholar
    • Export Citation
  • Mueller, M. (2017) Will the Internet Fragment? Sovereignty, Globalization and Cyberspace, Cambridge, MA: Polity.

  • Murgia, M. and Gross, A. (2020) ‘Inside China’s controversial mission to reinvent the internet’, Financial Times, 27 March, www.ft.com/content/ba94c2bc-6e27-11ea-9bca-bf503995cd6f [Accessed 10 May 2023].

    • Search Google Scholar
    • Export Citation
  • NATO (North Atlantic Treaty Organization) (2016) ‘Cyber Defence Pledge’, Press Release 124, 8 July, www.nato.int/cps/en/natohq/official_texts_133177.htm [Accessed 11 February 2022].

    • Search Google Scholar
    • Export Citation
  • Price, M. (2018) ‘The global politics of internet governance. A case study in closure and technological design’, in D.R. McCarthy (ed) Technology and World Politics. An Introduction, London: Routledge, pp 12645.

    • Search Google Scholar
    • Export Citation
  • Privacy International (2016) The Global Surveillance Industry, July, www.privacyinternational.org/sites/default/files/2017-12/global_surveillance_0.pdf [Accessed 31 March 2023].

    • Search Google Scholar
    • Export Citation
  • Rumelili, B. and Towns, A.E. (2022) ‘Driving liberal change? Global performance indices as a system of normative stratification in liberal international order’, Cooperation and Conflict, 57(2): 15270.

    • Search Google Scholar
    • Export Citation
  • Shaurya and Singh, M. (2021) ‘Cyber threats of modern era’, in V.E. Balas, A.E. Hassanien, S. Chakrabarti and L. Mandal (eds) Proceedings of International Conference on Computational Intelligence, Data Science and Cloud Computing. IEM-ICDC 2020, Singapore: Springer, pp 65970.

    • Search Google Scholar
    • Export Citation
  • Slayton, R. (2015) ‘Measuring risk: computer security metrics, automation, and learning’, IEEE Annals of the History of Computing, 37(2): 3245.

    • Search Google Scholar
    • Export Citation
  • Slayton, R. (2021) ‘Governing uncertainty or uncertain governance? Information security and the challenge of cutting ties’, Science, Technology & Human Values, 46(1): 81111.

    • Search Google Scholar
    • Export Citation
  • Stevens, T. (2020) ‘Knowledge in the grey zone: AI and cybersecurity’, Journal of Digital War, 1(1): 16470.

  • US Department of Homeland Security (2011) Enabling Distributed Security in Cyberspace: Building a Healthy and Resilient Cyber Ecosystem with Automated Collective Action, 23 March, www.dhs.gov/xlibrary/assets/nppd-cyber-ecosystem-white-paper-03-23-2011.pdf [Accessed 12 May 2023].

    • Search Google Scholar
    • Export Citation
  • Voo, J., Hemani, I., Jones, S., DeSombre, W., Cassidy, D. and Schwarzenbach, A. (2020a) Reconceptualizing Cyber Power: Cyber Power Index Primer, Cambridge, MA: Belfer Center for Science and International Affairs.

    • Search Google Scholar
    • Export Citation
  • Voo, J., Hemani, I., Jones, S., DeSombre, W., Cassidy, D. and Schwarzenbach, A. (2020b) National Cyber Power Index 2020: Methodology and Analytical Considerations, Cambridge, MA: Belfer Center for Science and International Affairs.

    • Search Google Scholar
    • Export Citation
  • Yarovenko, H., Kuzmenko, O. and Stumpo, M. (2020) ‘Strategy for determining country ranking by level of cybersecurity’, Financial Markets, Institutions and Risks, 4(3): 12437.

    • Search Google Scholar
    • Export Citation