Automating social media content moderation: implications for governance and labour discretion

Authors:
Sana Ahmad WZB Berlin Social Science Center and Free University Berlin, Germany

Search for other papers by Sana Ahmad in
Current site
Google Scholar Close
and
Maximilian Greb WZB Berlin Social Science Center, Germany

Search for other papers by Maximilian Greb in
Current site
Google Scholar Close
Article Category:
Research Article
Copyright:
© Authors 2022
Online Publication Date:
27 Oct 2022
Pages:
176–198
Publisher:
Bristol University Press
Volume/Issue:
Volume 2: Issue 2
DOI:
https://doi.org/10.1332/273241721X16647876031174
Restricted access
Get eTOC alerts
Rights and permissions Cite this article
Buy article/chapter (
GBP  £25.00
)

Content moderation is key to platform operations. Given the largely outsourced character of content moderation work and the dynamic character of social media platforms, technology firms have to address the accompanying high degrees of uncertainty and labour indeterminacy. Central to their managerial strategies is the use of automated technology that allows them to organise work by incorporating the social media user activities within the production processes, and control workers for ensuring the accuracy of content moderation decisions. The labour process analysis is informed by two workshops with ten participants at a Berlin-based IT-services firm providing content moderation services to a lead firm based in the USA. The research design combines together the design thinking method and the focus group interview method to examine the worker–machine interaction. The research findings indicate that technical control results in continuous standardising of content moderation work through routinisation of tasks and codification of time. Its combination with bureaucratic control through the supply-side managerial functions aims to ensure the quality service delivery and points to the continued significance of human supervision. Correspondingly, there are two main contributions of our study: first, regarding the governance in content moderation value chains and second, regarding the worker experiences of technical-driven control. On account of the limited resistance observed in the labour process, we conclude that instead of seeing it as the totalisation of technical control, our findings point towards the structural conditions in Germany that restrict migrant workers’ agency.

  • Ahmad, S. (nd) Work and Workers in Content Moderation Value Chains [Unpublished doctoral dissertation], Berlin: Free University Berlin.

  • Ahmad, S. (2019) ‘It’s Just the Job’: investigating the influence of culture in india’s commercial content moderation industry, preprint, SocAXiv. doi: 10.31235/osf.io/hjcv2

    • Search Google Scholar
    • Export Citation

  • Ahmad, S. and Krzywdzinski, M. (2022) Moderating in obscurity: how Indian content moderators work in global content moderation value chains, in M. Graham and F. Ferrari (eds) Digital Work in the Planetary Marke, Cambridge, MA: The MIT Press, International Development Research Centre, pp 77–95.

    • Search Google Scholar
    • Export Citation

  • Anderson, B. (2010) Migration, immigration controls and the fashioning of precarious workers, Work, Employment and Society, 24(2): 300–17. doi: 10.1177/0950017010362141

    • Search Google Scholar
    • Export Citation

  • Appelman, N. and Leerssen, P. (2022) On ‘trusted’ flaggers, December. https://law.yale.edu/sites/default/files/area/center/isp/documents/trustedflaggers_ispessayseries_jul2022.pdf.

    • Search Google Scholar
    • Export Citation

  • Ascher, D.L. and Noble, S.U. (2019) Unmasking hate on Twitter: disrupting anonymity by tracking trolls, in Free Speech in the Digital Age, New York: Oxford University Press. doi: 10.1093/oso/9780190883591.003.0011

    • Search Google Scholar
    • Export Citation

  • Barrett, P.M. (2020) “Tech - Content Moderation June 2020.” NYU stern center for business and human rights. June 2020, https://bhr.stern.nyu.edu/tech-content-moderation-june-2020.

    • Search Google Scholar
    • Export Citation

  • Batt, R. and Moynihan, L. (2002) The viability of alternative call centre production models, Human Resource Management Journal, 12(4): 14–34. doi: 10.1111/j.1748-8583.2002.tb00075.x

    • Search Google Scholar
    • Export Citation

  • Bauder, H. (2005) Institutional capital and labour devaluation: the non-recognition of foreign credentials in Germany, European Journal of Economics and Economic Policies: Intervention, 2(1): 75–93. doi: 10.4337/ejeep.2005.01.09

    • Search Google Scholar
    • Export Citation

  • Beverungen, A., Böhm, S. and Land, C. (2015) Free labour, social media, management: challenging marxist organization studies, Organization Studies, London: SAGE Publications.

    • Search Google Scholar
    • Export Citation

  • Birke, P. (2022) Grenzen aus Glas: Arbeit, Rassismus und Kämpfe der Migration in Deutschland, Wien: Mandelbaum Verlag eG.

  • Braverman, H. (1974) Labor and Monopoly Capital: The Degradation of Work in the Twentieth Century, New York and London: Monthly Review Press.

    • Search Google Scholar
    • Export Citation

  • Callaghan, G. and Thompson, P. (2001) Edwards revisited: technical control and call centres, Economic and Industrial Democracy, 22(1): 13–37. doi: 10.1177/0143831X01221002

    • Search Google Scholar
    • Export Citation

  • Carter, B., Danford, A., Howcroft, D., Richardson, H., Smith, A. and Taylor, P. (2011) ‘All they lack is a chain’: lean and the new performance management in the British civil service. doi: 10.1111/j.1468-005X.2011.00261.x

    • Search Google Scholar
    • Export Citation

  • Eurostat (2020) Durchschnittliche Wochenarbeitszeit Von Vollzeitbeschäftigten Erwerbstätigen in Deutschland von 2009 bis 2019 nach Geschlecht [Graph], Statista.

    • Search Google Scholar
    • Export Citation

  • Chen, A. (2012) Inside Facebook’s outsourced anti-porn and gore brigade, where Camel toes are more offensive than crushed heads, Gawker, https://www.gawker.com/5885714/inside-facebooks-outsourced-anti-porn-and-gore-brigade-where-camel-toes-are-more-offensive-than-crushed-heads.

    • Search Google Scholar
    • Export Citation

  • Colucci, E. (2007) ‘Focus groups can be fun’: the use of activity-oriented questions in focus group discussions, Qualitative Health Research, 17(10): 1422–33. doi: 10.1177/1049732307308129

    • Search Google Scholar
    • Export Citation

  • Cova, B. and Dalli, D. (2009) Working consumers: the next step in marketing theory?. doi: 10.1177/1470593109338144

  • Crawford, K. and Gillespie, T. (2016) What is a flag for? Social media reporting tools and the vocabulary of complaint, New Media & Society, 18(3): 410–28. doi: 10.1177/1461444814543163

    • Search Google Scholar
    • Export Citation

  • Dijck, J.van. (2013) Facebook and the engineering of connectivity: a multi-layered approach to social media platforms, Convergence, 19(2): 141–55. doi: 10.1177/1354856512457548

    • Search Google Scholar
    • Export Citation

  • Dunkel, W. and Kleemann, F. (2013) Customers at Work: New Perspectives on Interactive Service Work, London: Springer.

  • Ebrahim, S. (2017) There is no freedom of speech on Facebook, activists say, The Daily Vox (blog), (Accessed: 6 Dec 2017), https://www.thedailyvox.co.za/there-is-no-freedom-of-speech-on-facebook-activists-say-shaazia-ebrahim/.

    • Search Google Scholar
    • Export Citation

  • Edwards, R. (1979) Contested Terrain: The Transformation of the Workplace in the Twentieth Century, New York: Basic Books.

  • Ellis, V. and Taylor, P. (2006) ‘You don’t know what you’ve got till it’s gone’: re-contextualising the origins, development and impact of the call centre, New Technology, Work and Employment, 21(2): 107–22. doi: 10.1111/j.1468-005X.2006.00167.x

    • Search Google Scholar
    • Export Citation

  • Etzold, B. (2017) Capitalising on asylum – the reconfiguration of refugees’ access to local fields of labour in Germany, Refugee Review, 3: 82–102.

    • Search Google Scholar
    • Export Citation

  • Fairwork. (2022) Fairwork Germany Ratings 2021: Labour Standards in the Platform Economy, Berlin, Germany and Oxford, UK.

  • Flecker, J., Haidinger, B. and Schönauer, A. (2013) Divide and serve: the labour process in service value chains and networks, Competition & Change, 17(1): 6–23. doi: 10.1179/1024529412Z.00000000022

    • Search Google Scholar
    • Export Citation

  • Flecker, J. and Schönauer, A. (2016) The production of ‘placelessness’: digital service work in global value chains, in J. Flecker (ed) Space, Place and Global Digital Work, London: Palgrave Macmillan, pp 11–30.

    • Search Google Scholar
    • Export Citation

  • Frenkel, S., Korczynski, M., Donoghue, L. and Shire, K. (1995) Re-constituting work: trends towards knowledge work and info-normative control, Work, Employment and Society, 9(4): 773–96. doi: 10.1177/095001709594008

    • Search Google Scholar
    • Export Citation

  • Fuchs, C. (2014) Digital Labour and Karl Marx, New York: Routledge.

  • Fuller, L. and Smith, V. (1991) Consumers’ reports: management by customers in a changing economy, Work, Employment and Society, 5(1): 1–16. doi: 10.1177/0950017091005001002

    • Search Google Scholar
    • Export Citation

  • Gabriel, Y., Korczynski, M. and Rieder, K. (2015) Organizations and their consumers: bridging work and consumption, Organization, 22(5): 629–43. doi: 10.1177/1350508415586040

    • Search Google Scholar
    • Export Citation

  • Gabriel, Y. and Lang, T. (2015) The Unmanageable Consumer, London: SAGE.

  • Gandini, A. (2019) Labour process theory and the gig economy, Human Relations, 72(6): 1039–56. doi: 10.1177/0018726718790002

  • Gillespie, T. (2018) Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media, New Haven, CT and London: Yale University Press.

    • Search Google Scholar
    • Export Citation

  • Gillespie, T. (2020) Content moderation, ai, and the question of scale, Big Data & Society, 7(2): 2053951720943234. doi: 10.1177/2053951720943234

    • Search Google Scholar
    • Export Citation

  • Glucksmann, M.A. (2004) Call configurations: varieties of call centre and divisions of labour, Work, Employment and Society, 18(4): 795–811. doi: 10.1177/0950017004047965

    • Search Google Scholar
    • Export Citation

  • Grimmelmann, J. (2015) The virtues of moderation, Yale Journal of Law and Technology, 17: 42.

  • Habermas, J. (1962) Strukturwandel Der öffentlichkeit, Neuwied, Berlin, Vol 6, pp 42–75.

  • Habermas, J. (2022) Ein Neuer Strukturwandel der Öffentlichkeit und die Deliberative Politik, Berlin: Suhrkamp Wissenschaft, https://www.suhrkamp.de/buch/juergen-habermas-ein-neuer-strukturwandel-der-oeffentlichkeit-und-die-deliberative-politik-t-9783518587904.

    • Search Google Scholar
    • Export Citation

  • Hochschild, A.R. (1983) The Managed Heart: Commercialization of Human Feeling, Berkeley, CA: University of California Press.

  • Holtgrewe, U. and Doellgast, V. (2012) A service union’s innovation dilemma: limitations on creative action in German industrial relations, Work, Employment and Society, 26(2): 314–30. doi: 10.1177/0950017011432913

    • Search Google Scholar
    • Export Citation

  • Houlihan, M. (2004) Tensions and variations in call centre management strategies, Call Centres and Human Resource Management, London: Palgrave Macmillan, pp 75–101.

    • Search Google Scholar
    • Export Citation

  • Howcroft, D. and Richardson, H. (2012) The back office goes global: exploring connections and contradictions in shared service centres, Work, Employment and Society, 26(1): 111–27. doi: 10.1177/0950017011426309

    • Search Google Scholar
    • Export Citation

  • Ith, T. (2015) Microsoft’s PhotoDNA: protecting children and businesses in the cloud. Stories, https://news.microsoft.com/features/microsofts-photodna-protecting-children-and-businesses-in-the-cloud/.

    • Search Google Scholar
    • Export Citation

  • Jourová, V. (2019) How the code of conduct helped countering illegal hate speech online, Directorate-General for Justice and Consumers. European Commission, https://ec.europa.eu/info/sites/default/files/hatespeech_infographic3_web.pdf.

    • Search Google Scholar
    • Export Citation

  • Kellogg, K.C., Valentine, M.A. and Christin, A. (2019) Algorithms at work: the new contested terrain of control, Academy of Management Annals, 14(1): 366–410. doi: 10.5465/annals.2018.0174

    • Search Google Scholar
    • Export Citation

  • Klonick, K. (2017) The new governors: the people, rules, and processes governing online speech, Harvard Law Review, 131: 73.

  • Korczynski, M. (2002) Human Resource Management in the Service Sector, Basingstoke: Palgrave.

  • Krings, T. (2021) ‘Good’ bad jobs? The evolution of migrant low-wage employment in Germany (1985–2015), Work, Employment and Society, 35(3): 527–44. doi: 10.1177/0950017020946567

    • Search Google Scholar
    • Export Citation

  • Krzywdzinski, M. and Gerber, C. (2021) Between automation and gamification: forms of labour control on crowdwork platforms, Work in the Global Economy, 1(1–2): 161–84. doi: 10.1332/273241721X16295434739161

    • Search Google Scholar
    • Export Citation

  • Littler, C.R. and Salaman, G. (1982) Bravermania and beyond: recent theories of the labour process, Sociology, 16(2): 251–69. doi: 10.1177/0038038582016002006

    • Search Google Scholar
    • Export Citation

  • Mackenzie, A. (2019) From API to AI: platforms and their opacities, Information, Communication & Society, 22(13): 1989–2006. doi: 10.1080/1369118X.2018.1476569

    • Search Google Scholar
    • Export Citation

  • MacKenzie, R. and Forde, C. (2009) The rhetoric of thegood worker’versus the realities of employers’ use and the experiences of migrant workers, Work, Employment and Society, 23(1): 142–59. doi: 10.1177/0950017008099783

    • Search Google Scholar
    • Export Citation

  • McGovern, P. (2007) Immigration, labour markets and employment relations: problems and prospects, British Journal of Industrial Relations, 45(2): 217–35. doi: 10.1111/j.1467-8543.2007.00612.x

    • Search Google Scholar
    • Export Citation

  • Miozzo, M. and Ramirez, M. (2003) Services innovation and the transformation of work: the case of uk telecommunications, New Technology, Work and Employment, 18(1): 62–79. doi: 10.1111/1468-005X.00111

    • Search Google Scholar
    • Export Citation

  • Mirchandani, K. and Hande, M.J. (2020) The hidden work of challenging precarity, Canadian Journal of Sociology, 45(3): 265–88.

  • Noble, D.F. (1984) Forces of Production: A Social History of Industrial Automation, New York: Knopf.

  • Postigo, H. (2016) The socio-technical architecture of digital labor: converting play into youtube money, New Media & Society, 18(2): 332–49. doi: 10.1177/1461444814541527

    • Search Google Scholar
    • Export Citation

  • Remesh, B.P. (2008) Work organisation, control and ‘empowerment’: managing the contradictions of call centre work, in Upadhya, C. and Vasavi, A.R. (eds) An Outpost of the Global Economy: Work and Workers in India’s Information Technology Industry, London: Routledge India, pp 235–62.

    • Search Google Scholar
    • Export Citation

  • Reuter, M., Dachwitz, I., Fanta, A. and Beckedahl, M. (2019) Exklusiver einblick: so funktionieren facebooks moderationszentren, netzpolitik.org (blog), (Accessed: 5 Apr 2019), https://netzpolitik.org/2019/exklusiver-einblick-so-funktionieren-facebooks-moderationszentren/.

    • Search Google Scholar
    • Export Citation

  • Ritzer, G., Dean, P. and Jurgenson, N. (2012) The coming of age of the prosumer, American Behavioral Scientist, 56(4): 379–98. doi: 10.1177/0002764211429368

    • Search Google Scholar
    • Export Citation

  • Roberts, S.T. (2019) Behind the Screen: Content Moderation in the Shadows of Social Media, New Haven, CT and London: Yale University Press.

    • Search Google Scholar
    • Export Citation

  • Sapir, A. (1987) International trade in services: comments, in Giarini, O. (ed) The Emerging Service Economy, Oxford: Pergamon Press.

  • Silver, E. (2018) Hard questions: who reviews objectionable content on Facebook – and is the company doing enough to support them?, Meta (blog), (Accessed: 26 Jul 2018), https://about.fb.com/news/2018/07/hard-questions-content-reviewers/.

    • Search Google Scholar
    • Export Citation

  • Sinders, C. and Ahmad, S. (2021) The labor behind the tools: Using design thinking methods to examine content moderation software, ACM Interactions, https://interactions.acm.org/archive/view/july-august-2021/the-labor-behind-the-tools.

    • Search Google Scholar
    • Export Citation

  • Smith, C. and Thompson, P. (1998) Re-evaluating the labour process debate, Economic and Industrial Democracy, 19(4): 551–77. doi: 10.1177/0143831X98194002

    • Search Google Scholar
    • Export Citation

  • Smith, C. (2006) The double indeterminacy of labour power: labour effort and labour mobility, Work, Employment and Society, 20(2): 389–402. doi: 10.1177/0950017006065109

    • Search Google Scholar
    • Export Citation

  • Taylor, P. (2015) Labour and the changing landscapes of the call centre, in Putting Labour in Its Place, edited by Kirsty Newsome, Phil Taylor, Jennifer Bair, and Al Rainnie, Basingstoke: Palgrave Macmillan, pp 266–86. https://strathprints.strath.ac.uk/51919/.

    • Search Google Scholar
    • Export Citation

  • Thompson, P. (2003) Fantasy island: a labour process critique of the ‘age of surveillance’, Surveillance & Society, 1(2): 138–51. doi: 10.24908/ss.v1i2.3350

    • Search Google Scholar
    • Export Citation

  • Thompson, P. (2010) The capitalist labour process: concepts and connections, Capital & Class, 34(1): 7–14. doi: 10.1177/0309816809353475

    • Search Google Scholar
    • Export Citation

  • Thompson, P. and Smith, C. (2009) Labour power and labour process: contesting the marginality of the sociology of work, Sociology, 43(5): 913–30. doi: 10.1177/0038038509340728

    • Search Google Scholar
    • Export Citation

  • Wang, J. (2018) Regulating Hosting ISPs’ Responsibilities for Copyright Infringement, Singapore: Springer.

  • Wright, E.O. (2000) Working-class power, capitalist-class interests, and class compromise, American Journal of Sociology, 105(4): 957–1002. doi: 10.1086/210397

    • Search Google Scholar
    • Export Citation

  • Zahra, L. (2020) A new resource to fight unfair content takedowns, Mozilla Foundation (blog), (Accessed: 13 May 2020), https://foundation.mozilla.org/en/blog/new-resource-fight-unfair-content-takedowns/.

    • Search Google Scholar
    • Export Citation

  • Ziegler, A. (2021) Das Tech-unternehmen. Zum fundament eines neuen Unternehmenstypus, ISF München. doi: 10.36194/ArbSozForschung-2021-001

    • Search Google Scholar
    • Export Citation
Sana Ahmad WZB Berlin Social Science Center and Free University Berlin, Germany

Search for other papers by Sana Ahmad in
Current site
Google Scholar Close
and
Maximilian Greb WZB Berlin Social Science Center, Germany

Search for other papers by Maximilian Greb in
Current site
Google Scholar Close

sana.ahmad@wzb.eu
maximilian.greb@wzb.eu

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 1409 770 92
Full Text Views 2193 659 1
PDF Downloads 1185 340 3

Altmetrics

Dimensions

Cover Work in the Global Economy
Work in the Global Economy