Research Statement

Research Questions

The research questions that this work addresses are:
How are contemporary creative industry practitioners employing digital automation?
How do creative industry practitioners and media and cultural studies academics perceive the probable future trajectories of automation and media production?
Do these practices and imagined futures contribute to a contemporary politics of digital automation?

Recent years have seen a surge of interest surrounding how digital automation may transform labour and society in the near future. This has been heavily influenced by high-profile studies which suggest that within the next twenty years 47%-80% of jobs in developed countries could be automated (Frey and Osborne 2017, Elliott 2014). While automation has historically been associated with machines conducting routine and repetitive mechanical tasks, advances in artificial intelligence and machine learning have led to predictions that soon many ‘creative’, decision-making processes will be automated (Brynjolfsson and McAfee 2014, Susskind and Susskind 2015, Ford 2015).

Exemplifying this potential trend, today we find software tools such as Magisto and MAGIX Fastcut that are marketed as completely automated video editing programmes powered by artificial intelligence, effectively purporting to replace human video editors with software for particular forms of production. At the same time, however, contemporary creative work is heavily reliant upon specific processes and practices of digital automation that enable numerous key production techniques for video, photography, music and games. Indeed, many contemporary forms of software and hardware are advertised as being ‘revolutionary’ upgrades over their predecessors precisely because they incorporate advanced automated tools. This includes the various content-aware tools present within Adobe Photoshop; the type of computational photography exemplified by portrait mode on the iPhone; automatic image stabilization and object tracking tools within nonlinear video editing software such as Adobe Premiere, Avid Media Composer and DaVinci Resolve; automated object tracking and hazard avoidance systems on drones, and the automatic pitch correction, tuning and noise reduction tools employed within audio editing software such as Adobe Audition and Avid Pro Tools.

Digital automation is therefore central to enabling forms of contemporary creative media production and a potential source of concern surrounding precarity and the future of employment in the creative industries. The film explores why creative industry practitioners employ automated tools, what benefits and new possibilities these tools present for contemporary media production, and the potentially detrimental impacts these tools and technologies may have upon the production of creative work and more generally upon culture and society.

Context

There have been numerous recent publications that focus upon the societal impacts of automation, including Nicholas Carr’s (2015) The Glass Cage, Safiya Noble’s (2018) Algorithms of Oppression, Virginia Eubanks’ (2018) Automating Inequality, Cathy O’Neil’s (2016) Weapons of Math Destruction and Bernard Stiegler’s (2017) Automatic Society. Where this research project departs from these existing lines of enquiry is the specific focus upon the employment of automated tools within the creative industries and the generation of audio-visual work about automation that reflexively employs automated tools in its own production. While the documentary does not explicitly discuss the use of these tools, I wish to highlight this element of the work here.

A key example is the production of the soundtrack to the film, which employed a software tool called Smartsound Sonicfire Pro 6. This allows users to customise the length of each song to a frame of video; the software will not merely stop the music but rearranges it to last the precise duration requested. It also allows users to alter the mood or instrumentation mix with no expert audio-related knowledge. So, for example, when dialogue is present, the volume of the music can not only be reduced, but layers that would potentially compete for attention with the dialogue, such as drums or prominent bass lines, can be removed. The software also has multiple alternate arrangements of each audio track, allowing subtle variations on a theme to be employed. In the past having this level of control over timing, mood and mix would have necessitated employing a sound designer and/or mixer, whereas using Sonicfire Pro 6 allowed the documentary to largely be approached as a solo project.

This exemplifies the tendencies within contemporary automated media production tools to both enable individuals or small independent production companies to perform tasks that would have previously required significantly larger teams, whilst concurrently promoting forms of neoliberal subjectivity that are based upon individual entrepreneurship and service provision within knowledge or creative economies. These tendencies should be situated within two divergent strands of scholarship. The first strand emphasises how digital technologies afford increasing numbers of citizens to participate in sophisticated modes of mediated production (Jenkins 2006, Bruns 2008, Burgess and Green 2009) and typically celebrates this as an empowering and democratising convergence culture. The second strand, by contrast, makes visible the connections between networked digital information and communication technologies, and the political economy, culture and infrastructures of neoliberalism and platform capitalism. The quantifiability that is integral to computation is understood to enable the forms of comparison and measurability that are a pre-requisite for assessing the results of competition. Computation is therefore positioned as extending what can be commodified (Harvey 2005, Dean 2009, Beer 2015, Beller 2016). Consequently, within this literature computation is commonly aligned with marketisation and neoliberalisation (Peck 2010) rather than empowerment and democracy.

The documentary is part of a larger research project that examines the relationships between digital automation and creative media production and consumption. This larger project will include written research outputs that explore how media production distribution and consumption are being altered through various tools and practices associated with automation and machine learning.

Methods

The film is a documentary which consists of a series of qualitative interviews conducted with creative media professionals and academics. It features some expository narration tying them together.

Outcomes

Creative professionals raised concerns about the rollout of creative forms of automation. In particular, they argued that automation can produce a standardization that reduces the ability to push boundaries, break conventions and do things differently. Similarly, automation was seen as having the potential to inhibit an artist’s understanding of their creative practice and ability to critically reflect on their work, as black-boxed systems conceal processes behind consumer-friendly GUIs. Despite these specific concerns, however, creative professionals were in general quite positive about the ways that automation has improved their work, both in terms of enabling new forms of creative practice and reducing the amount of tedious, repetitive work they had to undertake.

Academics, perhaps unsurprisingly, were rather more critical of some of the potential pitfalls of digital automation. In particular concerns were raised about the ways that automation and artificial intelligence reinforce problematic hierarchies surrounding gender, leverage the unremunerated labour and knowledge of indigenous and other minority communities, and form increasingly standardised modes of commodified knowledge.

Partially, this divergence derives from differences surrounding a humanist model of agency that posits a creative individual who employs tools (that was common to most creative workers), and a posthuman model of distributed agency that instead posits assemblages of humans, machines and other entities. While some artists felt that elements of the latter perspective was emerging through creative interactions with technologies stating that at times ‘It definitely does feel like you’re collaborating with the machine, and you literally are,’ this was at odds with numerous statements that technologies are simply tools that are purposively employed by humans.

One of the key findings from the interviews with creative media practitioners was the difference between ways that automation is used in various contexts to save time on the one hand, and on the other, how automation enables new types of creative activities that arise from novel collaborations between humans and machines which leverage the more-than-human calculative speeds of contemporary computational assemblages.

The time that is saved by employing massively automated digital systems allows companies to exist at far smaller scales than was previously viable. As one interviewee commented; “We have five people on this project, without a [game] engine we’d need 45”. Such independent creative ventures can be flexible, agile and innovative in ways that align closely with neoliberal business rhetoric. Nevertheless, this also enables creative professionals to derive a significant sense of self-worth, agency and pleasure from their labour. There are, however, questions that this raises surrounding the concentration of power in the designers of these tools within the broader context of platform capitalism (Srnicek 2016). For example, consider how Amazon’s Lumberyard game engine ties in with Amazon Web Services, thereby further entrenching Amazon’s position as a dominant provider in back-end web services. Consequently, the apparent democratization of media production frequently fosters dependence upon a relatively small number of corporate actors which operate within the paradigms of walled gardens and subscription models. In some cases, this requires regular payments to be made for media producers to continue accessing project files, with Adobe’s Creative Cloud being emblematic of such predatory practices that are designed to effectively lock creative producers into specific software ecosystems.

Alongside these practices that are primarily designed to save time, automated systems allow novel types of collaborative creativity. Here computational systems are increasingly understood as active participants in the creative process via techniques such as procedural generation and machine learning. As Joanna Zylinska (2017, 14) argues, while we commonly encounter a false and reductive opposition between humans and machines, a more productive approach positions ‘the human as part of a complex assemblage of perception in which various organic and machinic agents come together—and apart—for functional, political, or aesthetic reasons.’ This potentially leads to a more progressive politics of automation, one based upon recognising creative processes as being necessarily more-than-human and collective affairs, resonating with recent accelerationist calls for automated futures that escape political logics of neoliberalism (Srnicek and Williams 2015, Frase 2016).

Dissemination

The work has been funded by a Massey University research fund grant of NZD$10,000. So far, the work has been screened at several public talks in Aotearoa-New Zealand that have examined the interface between automation and labour. A short excerpt from the film alongside a short essay about artificial intelligence, machine learning and inequality is due to be published shortly in the open access online journal Spheres.

References

Beer, David. 2016. Metric power. London: Palgrave Macmillan.
Beller, Jonathan. 2016. “Informatic Labor in the Age of Computational Capital.” Lateral 5 (1).
Bruns, Axel. 2008. Blogs, Wikipedia, Second Life, and beyond: From production to produsage. New York: Peter Lang.
Brynjolfsson, Erik, and Andrew McAfee. 2014. The second machine age: Work, progress, and prosperity in a time of brilliant technologies. New York and London: WW Norton & Company.
Burgess, Jean, and Joshua Green. 2008. YouTube: Online video and participatory culture. Cambridge: Polity.
Carr, Nicholas. 2015. The glass cage: Where automation is taking us. New York and London: W.W Norton & Company.
Dean, Jodi. 2009. Democracy and other neoliberal fantasies: Communicative capitalism and left politics. Durham: Duke University Press.
Elliott, Stuart W. 2014. “Anticipating a Luddite revival.” Issues in Science and Technology 30 (3):27-36.
Eubanks, Virginia. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. New York: St. Martin’s Press.
Ford, Martin. 2015. Rise of the Robots: Technology and the Threat of a Jobless Future. New York: Basic Books.
Frase, Peter. 2016. Four futures: Life after capitalism. London and New York: Verso books.
Frey, Carl Benedikt, and Michael A Osborne. 2017. “The future of employment: how susceptible are jobs to computerisation?” Technological forecasting and social change 114:254-280.
Harvey, David. 2005. A brief history of neoliberalism. Oxford: Oxford University Press.
Jenkins, Henry. 2006. Convergence culture: Where old and new media collide. New York: NYU press.
Noble, Safiya Umoja. 2018. Algorithms of Oppression: How search engines reinforce racism. New York: NYU Press.
O’Neil, Cathy. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. USA: Allen Lane.
Peck, Jamie. 2010. Constructions of Neoliberal Reason. Oxford: Oxford University Press.
Srnicek, Nick. 2016. Platform capitalism. Cambridge: Polity.
Srnicek, Nick, and Alex Williams. 2015. Inventing the Future: Postcapitalism and World Without Work. London: Verso.
Stiegler, Bernard. 2017. Automatic Society: The Future of Work:  Hoboken NJ: John Wiley & Sons.
Susskind, Richard E, and Daniel Susskind. 2015. The future of the professions: How technology will transform the work of human experts. Oxford: Oxford University Press.
Zylinska, Joanna. 2017. Nonhuman photography. Cambridge MA: MIT Press.

Peer Reviews

All reviews refer to original research statements which have been edited in response to what follows:

Review 1: Accept work and statement for publication with no alterations
The film reflects on both creativity and automation, and the filmmaker engages with practitioners and experts in the fields of creative practice and media and technology, and the work presents an interesting summation of current fact and opinion on the topic. The originality lies in its pertinence to contemporary media practice, prosumer practice and in assessing a potential future landscape for digital creativity. The use of Smartsound Sonicfire Pro 6 in the creation of the soundtrack of the film is very interesting. This use of automation could have been taken further in terms of shooting style and technique. The film is quite classic in style, broadly taking a participatory documentary form. More synergy between form and content would dramatically increase the outputs research value. Shifts in shooting style for documentary interviews occur in the first half of the film. There are unusual filters used at various moments, but there doesn’t seem to be any clear thematic or narrational justification for the use of such techniques. If this use of filters were chosen through a process of automation, this would be interesting and innovative, but this clear that this is the case. The author may be interested in Lars Von Trier’s use of ‘automated cinematography’ in his film The Boss of it All (2008) as an example of a film that implements an automated approach to creativity, which could potentially be used in future practice.

The research statement is clear and well written. The research questions frame the screenwork well. They are pertinent and important questions, which are thoroughly engaged with in the screenwork itself. The author has a strong knowledge of contemporary arguments in the field of automation and creativity, and they engage with some of the key literature through a reflection on the production of the soundtrack. Equally, the author engages with the political ramification of a move toward automation, and considers both the neoliberal positioning of this phenomena on the one hand, and the potential advantages of automation for creative entrepreneurship and creative economies (although, perhaps, these are two sides of the same coin). I would be interested to know more about the author’s own assessment of the political ramifications of these developments, but this is perhaps beyond the scope of this submission.

Review 2: Accept work subject to minor revisions of statement
The Screenwork and the Statement share a related problem: an insufficiently developed argument. Before turning to that issue, though, I would point to a smaller problem, where there seemed to be a curious disjunction between the Statement’s description of the video and the video’s approach/organization.

When asked about the criteria for judging the research of the piece, the author states: “I would suggest that the reviewers assess this submission as a documentary film about automation and creativity that reflexively uses automated tools to build its argument.” This statement seems to promise something that the “documentary” does not actually deliver, which is a “reflexive” consideration of automated creative tools. It may well be that automated tools were used in making this documentary, but in what ways is this usage “reflexive”? Where does the video discuss the tools used in its making (have I simply missed this)? I think it would certainly be a much more interesting documentary if it did draw attention to the tools used in its own making. I think many viewers would indeed be interested to know how automated tools were used to “build its argument,” which seems to imply that the content of the argument was assembled automatically. If this were the case, we would truly see a piece that challenges traditional notions of creativity and its relation to machinic or automatic assemblage. Of course, this issue is in part simply a matter of imprecise phrasing, which could be corrected by editing the Statement.

In a broader sense, however, the issue of reflexivity in relation to argument-building points to what I see as the central flaw of the piece, which is its argument. Put simply, what is the argument that the documentary and statement attempt to make? The Statement notes that there are “two divergent strands of scholarship” with one “celebrat[ing mediated production] as an empowering and democratising convergence culture” while in the other, “computation is commonly aligned with marketisation and neoliberalisation.” This divergence in large part structures the discussion of the documentary, with creative professionals generally taking a more positive view of digital automation as time-saving and empowering to artists, and academics, as the Statement notes, being “more critical of some of the potential pitfalls of digital automation.” The problem is not, however, that these views are divergent; it is that this piece seems content merely to take these different views for granted, never really addressing the assumptions that underlie their views or cause them to diverge. In this sense, I would claim that this piece does not ultimately make an argument about creativity and automation; it mainly summarizes these conflicting views, without explaining their basis, much less reconciling them. The author does claim that the video offers “some expository narration tying them [i.e., “interviews with creative media professionals and academics”] together,” but it is hard to see what “ties” them, because there is little discussion of the underlying issues. Although the video is a competent, professional documentary, with excellent interviews of smart, articulate artists and thinkers, it seems unable to address these deeper issues, such as ”Does creativity require human creators? Are digital technologies ‘empowering’ only if they remain under human control? How empowering can they be if the results accrue value mainly for corporate or state powers?” Nor do more pragmatic concerns receive much attention in this piece, beyond arguments that these technologies enable artists to do more, or to do more for less cost. What about questions such as “What is the basis for artistic judgement when creative work can be produced entirely by algorithmic processes? (Or will judgement itself also come to be automated?) At what point does automated or autonomous creativity begin to replace human artists (or academics for that matter)?” While some of the interviews do address these questions, the piece as a whole does not seem to look beyond old-fashioned ideas of “technology as either a tool or a threat.” The academic references that it mobilizes are relevant, but it does not consider them in enough depth to address the deeper issues of technology’s changing relations to human beings, and the redefinition of both.

Nevertheless, having made this extended objection, I am fully aware that the documentary is a polished, competent work, that the issues that the piece raises are certainly significant, and although it does not advance a rigorous argument in my view, it does contain valuable information that will likely be of interest to many. I therefore recommend for publication, whilst asking that the author address the point about reflexivity. I do not believe that my concerns about the generality of the piece’s argument can be addressed without major revisions that would likely be impossible in the present context.

All reviews refer to original research statements which have been edited in response.