How is comparative knowledge produced?

The chapters show that the various issue areas of security politics – whether traditional or new – all feature epistemic infrastructures that produce comparative knowledge, usually in the form of statistics. Some of these epistemic infrastructures are maintained by international organizations, others by non-governmental organizations, and yet others by for-profit companies. These epistemic infrastructures are not only important as publishers of comparative knowledge, they also serve as enablers of the comparative practices of other actors. By compiling statistics on piracy incidents, cybersecurity incidents or the military arsenals of states, the epistemic infrastructures make it easier for other actors to analyse any trends behind these incidents or compare the military capabilities of selected states. Yet, as Bastian Giegerich and James Hackett (Chapter 3) and Christian Bueger (Chapter 6) underscore, the compilation of the statistics involves a range of decisions on definitions, classifications and methods of calculation that affect the outcome of the comparisons. In this sense, epistemic infrastructures not only simply serve as enablers of actors’ comparative practices, they also shape these practices. They preconfigure them by publishing datasets that reflect particular methodologies for operationalizing and quantifying security issues and, relatedly, privilege some forms of comparison over others.

The chapters demonstrate that the available comparative knowledge on security issues is often ambiguous. To give but a few examples. There are differing statistics on the same crimes (see Jakobi and Herbst, Chapter 8), disputes about the number of piracy incidents (see Bueger, Chapter 6) and different rankings and lists of fragile states (see Krause, Chapter 7). These ambiguities, though, are not peculiar to more recent subfields of security politics. They have also persisted in longstanding subfields, as Giegerich and Hackett (Chapter 3) illustrate with their discussion of the different methods for calculating military expenditure. Put differently, the epistemic infrastructures on arms dynamics – that is, on the issue that for a long time has been most central to global security politics – are no more consolidated than those on newer security issues such as maritime security, global crimes, state fragility or cybersecurity.

What accounts for these ambiguities and their persistence? The authors point to three factors in particular. One is the difficulty of collecting reliable data. This can restrict the number of data producers when special resources are required that only a few actors possess, for example large networks of digital sensors in the case of cybersecurity (see Myatt and Müller, Chapter 9). It can also lead to a situation that renders data permanently ambiguous, when it forces the actors producing the comparative knowledge to rely on estimates and best guesses rather than robust figures. For instance, the different levels of transparency that states allow with respect to their military capabilities and expenditures complicates the statistical work of the IISS (see Giegerich and Hackett, Chapter 3). Similarly, the opacity of most crimes makes it hard to compile accurate statistics on them (see Jakobi and Herbst, Chapter 8). Another factor is a lack of consolidation of, and standardization among, the relevant epistemic infrastructures. Many of the subfields – from arms dynamics through maritime security and global crimes to state fragility – feature a plurality of epistemic infrastructures, with a number of different actors producing and publishing comparative knowledge, each employing their own methodology.

The third factor is the political struggle over the governance of the issues. Research on rankings discusses political contention as a factor that spurs the production of comparative knowledge. Interested actors, the argument goes, seek to influence the governance of issues by publishing comparative knowledge that supports their policy proposals, resulting in a situation characterized by a plurality of representations (see Kelley and Simmons, 2019: 496). The present volume highlights another effect of political contention. It hampers the consolidation of epistemic infrastructures, that is, the development of shared methodologies and shared institutional frameworks for the production of comparative knowledge. In the case of maritime security, for instance, disputes over the best governance architecture have prevented the consolidation of the fragmented assemblage of epistemic infrastructures (see Bueger, Chapter 6). The Cold War arms control negotiations were for a long time characterized by methodological disputes between Western and Eastern states over how to count and compare military capabilities. These disputes were only resolved in the second half of the 1980s (see Müller and Albert, 2021). In his chapter on the Treaty on Conventional Forces in Europe (Chapter 11), Hans-Joachim Schmidt unpacks how Western and Eastern states developed a shared comparative framework for the conventional balance in Europe, revealing how closely intertwined the methodological debates were with political considerations of what constituted an acceptable distribution of military capabilities.

The study of comparative practices thus underscores how fuzzy structures and governance objects are in world politics. There is no natural way of producing comparative knowledge about the distribution of power or any other governance object. Rather, which comparative practices are the most appropriate and pertinent ones to make sense of structures such as the distribution of power and status hierarchies or to produce knowledge on governance objects such as maritime security, global crime or state fragility remains an open and continuously renegotiated question. Even if actors do not have political motives for favouring some comparative practices over others – which they often have – there are usually several plausible ways of conceptualizing the phenomena and hence producing comparative knowledge on them. It is not only the balance of power that is ‘elusive’ (Wohlforth, 1993). All governance objects are.

A productive avenue for further probing into how political contention affects the production of comparative knowledge would therefore be to explore how actors cope with the ambiguities. One common strategy that actors employ is to depoliticize comparative practices and to treat ambiguities as something that can be reduced through better data collection or better methodologies (on the depoliticization of knowledge production, see Louis and Maertens, 2021). As part of this strategy, actors may also acknowledge that there are different plausible representations of the issues, as the IISS does for instance by providing more than one set of figures on military expenditures (see Giegerich and Hackett, Chapter 3). This strategy essentially seeks to decouple the mapping tools dimension of comparative practices from their ordering tools dimension. The two dimensions are, however, closely intertwined. Relatedly, at least two politicization strategies can be observed in the practice of security politics: the first politicization strategy is to emphasize the ambiguities. When established comparative practices do not support the policy proposals that actors want to promote, then they can reopen the debate on the best comparative practices by demonstrating that the issues can also be represented differently. Governance arrangements, however, often require some agreement on how to produce comparative knowledge on the respective issues. The second politicization strategy is, therefore, to problematize the ambiguities and to push for a common comparative framework to be negotiated and developed. In global security politics, such problematizations are not uncommon, but have so far led only in some instances – for example during arms control negotiations – to a consolidation and standardization of the epistemic infrastructures. One crucial reason is that the actors know that different comparative methodologies may produce different governance outcomes – in short, that the comparative practices are ordering tools – which means that they usually cannot develop a common comparative framework without resolving the underlying political questions.

How does comparative knowledge become politically relevant?

Actors produce comparative knowledge both in order to influence ongoing debates and to put new issues on the agenda. Instead of asking whether the production of comparative knowledge is spurred by political debates or initiates such debates, it seems more productive, therefore, to adopt a processual perspective and delve into the complex dynamics in which comparative practices are used to increase political attention for issues, prompting debates that then create more demand for comparative knowledge, thus making the comparative practices more relevant politically. Several chapters provide insights into these dynamics, which underpinned the growing political attention paid to relative naval capabilities in the late 19th century (Langer, Chapter 10), the proliferation of epistemic infrastructures on maritime security (Bueger, Chapter 6), the evolving governance of state fragility (Krause, Chapter 7) and supply and demand with respect to statistics on global crimes (Jakobi and Herbst, Chapter 8). Cybersecurity is an example of how a dynamic of this kind can become self-perpetuating, with the producers of comparative knowledge sustaining a narrative of an ever-evolving threat landscape that demands constant monitoring – and hence a continual production of comparative knowledge (see Myatt and Müller, Chapter 9).

These dynamics do not unfold automatically. Rather, they depend on the ability of actors – not necessarily the same actors as those producing the comparative knowledge – to convince relevant constituencies that some issues deserve more political attention and require more or renewed political action. Comparative practices are powerful tools in this endeavour. Actors resort to several strategies to increase the impact of their arguments. They condense the arguments into simple statistics (see Friedberg, 1988: 283), a strategy that works particularly well in policy fields in which governance activities are informed by calls for evidence-based policy making (see Jakobi and Herbst, Chapter 8). They craft special booklets that present and visualize the arguments, as NATO, for instance, did during the East–West disputes over the military balance in the 1980s (see Schlag, Chapter 4). And they use narratives to impose certain frames on developments. One example is the narrative of a tech race between the US and China, which is both substantiated through comparisons and creates further demands for comparisons to track which of the two states is winning the ‘race’ (see Retzmann, Chapter 12).

However, more research is needed to unpack these dynamics. The chapters suggest several avenues. One is to delve deeper into why particular forms of comparison – rather than comparative practices as such – come to be relevant. Two already mentioned possible factors are, first, the fit to the policy arguments that actors want to make and, second, governance modes that value particular forms of comparison. Notably, evidence-based policy making foregrounds statistical forms of comparative knowledge. Paul Beaumont (Chapter 2) stresses another factor: he shows that US policy makers chose equality in aggregates as the key comparison for the SALT II arms control negotiations because they thought that this comparison mattered most to how audiences would judge the outcome of the negotiations. This suggests that actors choose those forms of comparison that they deem to resonate most with the audiences that they want to influence. Both producers of comparative knowledge and their audiences may re-evaluate the pertinence and salience of particular ways of comparing in the wake of events that confound their expectations. One example, mentioned by Giegerich and Hackett (Chapter 3), is the weak performance of the Russian military in the war in Ukraine, which has led to debates on whether past assessments overestimated the relative strength of the Russian military, with analysts arguing for comparisons that pay more attention to qualitative factors such as morale, cohesion and logistics. Last but not least, Schmidt (Chapter 11) hints at what might be termed a ‘form follows function’ logic. The negotiations on the CFE Treaty put a premium on the verifiability of the agreed distribution of military power, which meant that dynamic force comparisons (such as complex combat simulations) were unsuitable and made simple statistical comparisons of select major weapon systems the preferred ordering tool for managing the military balance.

The relevant constituencies of actors are another avenue. In a narrow sense, the constituency consists in the politicians, diplomats and experts that partake in the governance of issues. Research on comparative practices tends to focus on how this constituency compares and debates comparisons. In a wider sense, the constituency of actors also comprises the publics that observe global security politics and potentially put pressure on policy makers to adopt, modify or end certain policies. What Paul MacDonald and Joseph Parent (2021: 375) emphasize with regard to status politics holds true for research on comparisons more broadly. More research is needed on this broader constituency, which played a role in the naval competition of the late 19th century, was the target of NATO’s Force Comparisons booklets and a factor in the US decision making on the SALT II comparisons. While Beaumont (Chapter 2) teases out that the US government assumed that it knew how its domestic public would assess status, Paul Musgrave and Steven Ward (Chapter 5) show that the US public is in fact pluralistic in its status understandings, with different subgroups using different markers to evaluate international status. This raises a double question: to what extent do the narrow and the broader constituency of actors differ in their prevalent comparative knowledge? Does this matter and, if yes, in what ways?

The final avenue is the comparative ordering of the agenda of global security politics. Several chapters, as mentioned, show how comparative practices were key to the processes through which particular security issues gained in prominence and importance within the agenda of global security politics. What is more, as Anja Jakobi and Lena Herbst (Chapter 8) highlight, comparisons have also been mobilized by actors to debate the significance of different issues of security governance – that is, for attempts to order security issues in terms of their relative salience. Research on comparative practices has, however, so far paid scant attention to the practices that are used in global security politics to compare the evolving salience of security issues as diverse as, say, arms dynamics, global crime and state fragility. Yet, the ordering of the agenda of global security politics involves not only the identification of the security issues that should be addressed as part of global governance. It also involves comparisons about which of these security issues (should) matter and how much. And by foregrounding some issues, these comparisons also make the comparative practices relating to these issues more relevant. One example is the varying salience of naval force comparisons. Debates about world power status made them central to debates about great power competition in the late 19th and early 20th century. During the Cold War, the nuclear balance and the European conventional balance were widely regarded as the most crucial dimensions of the East–West conflict, making the related comparisons more salient than naval force comparisons. In the past two decades, naval force comparisons have become more prominent again in the wake of growing competition between the US and China.

How do comparative practices shape security politics?

Comparative practices are ordering tools. They are, in fact, indispensable for most ordering purposes. When actors compete over power or status, they make comparisons of how much power or status they and other actors have. When actors negotiate governance arrangements, they make decisions on how to allocate rights and duties among themselves (and potentially also to other actors), which – like all matters of distribution – involve comparisons. When actors debate how much some security issues matter or should matter, they make comparisons about the importance of the security issues in question relative to other security issues. The question is, in this sense, not whether the ordering of global security politics involves comparative practices. It does. The question rather is which comparative practices it involves and what effects they have.

Put differently, some particular comparative practices may be associated with a specific mode of governance, for instance indicators and rankings with indirect modes of governance (see Broome and Quirk, 2015; Kelley and Simmons, 2020). But all modes of governance involve – in one way or another – comparative practices. Comparisons are integral to the construction of the governance objects, the debates on how much different actors have to contribute to the governance, the debates on how successful the governance is and, last but not least, the debates on which governance objects should get how much attention in world politics.

For all their differences, research on balance of power politics, status politics and indicators/rankings all associate comparative practices primarily with competitive dynamics in world politics. The chapters in this volume underscore that the effects on these competitive dynamics are variable. Comparative practices may either fuel competitive dynamics (see Langer, Chapter 10; Retzmann, Chapter 12) or be part of governance arrangements that tame them (see Beaumont, Chapter 2; Schmidt, Chapter 11). What determines in these cases which of the effects comes into play is not so much the difference between absolute and relative standards of comparison – the factor that Ann Towns and Bahar Rumelili (2017) emphasize to explain variations in competitive dynamics – but rather the type of ordering that the relevant actors strove for: unilateral ordering through competition versus ordering through joint governance.

That said, the chapters underscore that the modulation of competitive dynamics is only one effect that comparative practices have. They are also integral to the construction of governance objects and influence collective action in at least two ways. The first effect, discussed earlier, pertains to ordering the agenda of global security politics through distinguishing between, and ascribing salience to, policy issues. The second effect consists in steering the governance of the issues. These steering effects can occur in one of two ways: when states – or other governance actors such as international organizations – draw on comparative practices to decide how they deal with the issues, as in the use of state fragility rankings and ratings to decide on the allocation of development aid (see Krause, Chapter 7); or when other actors use comparative practices to put pressure on states or other governance actors to deal with issues in particular ways. This pressure game may have a competitive dimension, notably when the other actors use rankings to suggest that states are in a competition for best performance with regard to the governance of some issues, but it does not have to have one. The pressure can be generated in other ways, for instance through temporal comparisons that highlight and frame some trends, such as increases in piracy attacks, crimes or cyberattacks, as problems that states or other governance institutions should react to and tackle.

There is a strategic dimension to the effects. The fundamental ambiguity of security issues – the fact that they can be represented in multiple ways through comparative practices – gives actors leeway to promote those representations that are most conducive to the effects they prefer. In the CFE negotiations, for instance, the NATO members were able to make their problematization of the balance – the claim that there existed an imbalance favouring the Warsaw Pact – the basis for the arms control measures that entailed far bigger reductions for the Warsaw Pact members than for the NATO members (see Schmidt, Chapter 11). The effects of the comparative practices thus depend to a considerable degree on how successful the various actors involved in global security politics are in making their preferred representations central to the debates on and governance of the issues. This is neither to say that the production of comparative knowledge is always shaped by such strategic considerations, nor that the actors are always able to control the effects that the comparative practices have. The point rather is that the various actors are well aware that different comparative practices have different ordering repercussions and that, as a corollary, the ordering of global security politics often involves battles over the comparative practices that are to inform and guide the ordering.

One argument in research on rankings is that the more the published rankings differ in their representations – that is, the more ambiguous the available comparative knowledge is – the more diluted the ordering effects are (see Sauder and Espeland, 2006; Rumelili and Towns, 2022). If the representations of the issue are too incompatible, then ambiguity can indeed hamper or prevent governance arrangements. Before the CFE negotiations, West and East had been engaged in long and unsuccessful negotiations on conventional arms control since the early 1970s. These negotiations had been stalled by their inability to agree on a common interpretation of the conventional military balance in Europe. That being said, besides preventing collective modes of ordering, disagreements may also fuel more individualistic modes of ordering. Ambiguities about the relative standing of the different great powers were a factor fuelling the late 19th century naval competition (see Langer, Chapter 10). Divergent interpretations of the theatre nuclear and conventional balances in Europe were the engine of the Cold War arms race between West and East (see Müller and Albert, 2021). Moreover, ambiguities seem to prevent some ordering effects, not all. They may make some statistics inadequate as mapping tools, but that has not prevented actors from using them as ordering tools in global crime governance (see Jakobi and Herbst, Chapter 8). And despite their differences the state fragility rankings and ratings have had an impact on the flows of development aid (see Krause, Chapter 7).

Broadening the debate: the comparative ordering of world politics

The main aim of this volume is to foster a common debate in security studies on the use of comparisons in global security politics and to show how integral these practices are to the ordering of the policy field. Global security politics is, however, far from being the only policy field shaped by comparative practices. They are also pervasive features of other policy fields in world politics, such as – to name just a few – economic politics, development politics and climate politics. So, the debate should not remain limited to global security politics. In this spirit, this chapter ends with some suggestions for how to broaden the debate beyond global security politics. There are at least three productive ways for doing so.

The first is to study how comparative practices shape the agenda of world politics. The volume has highlighted how comparisons underpin debates about the importance of different security issues. The uses and effects of comparisons, though, go beyond that. Global security politics is sometimes described as ‘high politics’ and other policy fields as ‘low politics’. That description is itself comparative in that it orders world politics in terms of more and less important policy fields. The comparative ordering of world politics thus has two dimensions: the use of comparisons as ordering tools within policy fields and their use to establish order among policy fields – that is, to differentiate policy fields from one another and to assign political importance to them.

The second is to probe into how different policy fields deal with the ambiguity of comparative knowledge. Ordering is an act of pattering social relations, for instance by sorting actors into a hierarchy. This, however, only works as long as there is some shared understanding among relevant actors of what these patterns are. As the chapters in this volume have shown, the comparative knowledge available on political issues is often ambiguous, though that does not preclude that it has effects on how the issues are governed. This raises questions about the interplay between the level of ambiguity and the ordering effects of comparative knowledge. One crucial aspect of this interplay is that comparative practices can themselves be the objects of ordering, that is, of attempts by actors to make the available comparative knowledge less ambiguous through negotiating and agreeing on common frameworks of comparison. To what extent do policy fields differ in the ordering of comparative practices? For instance, why is the consolidation of knowledge infrastructures happening in some policy fields – think of climate politics and the authoritative role of the IPCC (see Edwards, 2010) – but not in others? Why, to give further examples, has the UN been successful in establishing itself as a key source of comparative knowledge on development politics (think of the Sustainable Development Goals and the related indicators framework, see Tichenor et al, 2022), but not on matters of arms dynamics, despite armaments and disarmament being a crucial dimension of security governance?

The third way is to explore the ecologies of comparative practices that underpin and shape policy fields. The volume underscores that comparisons order world politics in many ways: besides sorting the agenda of world politics, comparisons undergird hierarchies, fuel competition and guide governance efforts. Some comparative practices – for example rankings – potentially combine two or more of these modes of ordering. Still, the ordering of policy fields is not usually the product of one type of comparative practice, but of a combination of comparative practices, some of which enable, and reinforce, or, alternatively, challenge and contest other comparative practices. It seems, therefore, productive to not only study the ordering effects of particular types of comparative practices (such as indicators or rankings), but to delve further into the ecologies of comparative practices that, in their combination and juxtaposition, impart both structure and dynamics – and thus both stability and change – to policy fields.

Such a broadening of the debate will further underscore the point that this volume seeks to make: comparisons are one of many practices used in world politics. But they underpin and shape almost all, if not all, of its aspects, from hierarchies through competitions and the production of knowledge about governance objects to the distribution of (governance) rights and duties. World politics is deeply comparative in nature – and comparative practices are, accordingly, key to understanding and explaining how it is ordered and evolves.