Research Statement

Research Questions

The video essay Sound Stack queries what value there is in the quest for sonic authenticity within film soundtracks, and what are the apparent extents of this authenticity? 

The tutorial video interrogates the particular means by which the Sound Stack video essay employs volumetric visualisation. It is my attempt to address how formal methods of videographic research and experimentation might be shared within the wider community of practice.  

The portfolio as a whole questions the extent to which a scholar (me) might go before they consider a piece of videographic research to be ‘complete’ or at least to have been engaged with as completely as possible. 

Context / Significance

This work is a direct response to and extension of my previous video essay, Sonic Chronicle, Post Sound (2020). In that work, I focused on reading and categorising the film soundtrack using the soundscape research of R. Murray Schafer (1993). Schafer’s research focused on the fugitive nature of the soundscape (an analogue of the landscape), noting in particular how radically it had changed since the industrial revolution. In my video essay, I took Schafer’s methods of soundscape categorisation and applied them to the soundtracks of 3 newsrooms: The San Francisco Chronicle from Zodiac (Fincher, 2007), and the Washington Post from both The Post (Spielberg, 2017) and All the President’s Men (Pakula, 1976). This was a work that I had some ambition to revisit, but was unclear how this might manifest until I became aware of Michael Southworth’s soundscape research (1967), which originated in the late 1960s (as did Schafers) but was instead focused on urban design and the soundscape of the city.

Southworth completed his masters thesis, The Sonic Environment of Cities, in 1967. The aims of the thesis as set out suggested to me something of a road map for the video essay that would become Sound Stack. Southworth aimed to develop “techniques and language” for recording the soundscape of the city, Boston in his case (1967). I was keen, through my initial annotation experiments, to find a similar method of recording the soundscape of a film, extending my work beyond the broader classifications I had used previously and which were informed by Schaefer’s research. Though the methods I used to record and annotate my soundscape differed from those employed by Southworth, I found significant value in the analytical method he applied to his graphical recordings of the soundscape. In particular, his suggestion that most of the sound settings in his soundwalk sequence (the same sequence  I undertook for my recording) “did not seem to be singular” (Southworth, 1967). This concept of singularity, that one location might be sonically defined and identifiable from another, leaked into my own analysis of the soundscape of Zodiac (Fincher, 2007), and specifically that of the San Francisco Chronicle newsroom.

In revisiting and extending this previous research, I am acknowledging that in my own video essay practice, I have perhaps too easily accepted the end credits of my video essays as synonymous with the completion of a particular piece of research. The self-contained nature of the published video essay on Vimeo does not perhaps readily invite revision and expansion; rather, there is a compelling completeness to a finished video essay which often encourages me to move on.

Returning to the work allowed me to approach the newsroom soundscape employing a novel form of visualisation via the volumetric process. This is a method of visualisation pioneered by Kevin L. Ferguson in his 2015 video essay, Volumetric Cinema. The process makes use of the medical imaging program ImageJ (or the slightly newer version I use here known as Fiji) to visualise time along the Z axis of a cube, making it possible to manipulate a volumetric ‘stilled’ version of a video clip. In Sound Stack, I returned to the soundscape of the San Francisco Chronicle as portrayed in Zodiac, and through a process of sound annotation within the volume inspired by Southworth’s work, I came to question the sonic authenticity of the city soundscape as portrayed in the film (as opposed to the richly detailed and sonically authentic newsroom). In my analysis, I found a direct and fascinating connection between the creative design of the city soundscape in film and that of the real city as observed by Southworth. The tantalising question of authenticity then balances on the fine line between fiction and reality: whether the fictional city soundscape should be reflective of the largely anonymous real city soundscape observed by Southworth, or whether it might reflect an ambition to creatively render a soundscape unique to its location through some sonic delight?

It has long been a gripe of film sound researchers (such as Altman in 1980 and Beck in 2008) that film studies is an image-led field, a sentiment echoed in Southworth’s research, where he notes that city design “has been visually prejudiced” (1967). In Sound Stack, I have sought to turn this visual prejudice against itself by employing the eye-catching volumetric process to bring focus to the film's soundtrack.

Following the screening and discussion of Sound Stack at the ‘Theory and Practice of the Video Essay’ Conference at UMass in September 2022, the work languished somewhat as I debated what I should do with it. I wanted to seek a venue to publish, but the conference experience had left me questioning if the work was ‘complete’. Reflecting on my prior experience publishing video essays, I was drawn to the headings that Screenworks used for this very statement, specifically ‘method’. In the making of Sound Stack,I employed a specific formal method (the volumetric process), which, whilst not new, had taken me some time to tease out. In this method, I saw perhaps a means where I could engage in a collaborative dissemination process, offering a tutorial to other videographic scholars on the volumetric method as I employed it. I view this ‘publishing’ of the method in much the same way that research in hard science might capture their experimental method in step-by-step detail for later researchers to replicate. The difference here is that it is not a replication of the method I am hoping for but rather an extension into other videographic research, which might further illuminate me regarding the path towards a sound-led videographic criticism. It is not my place to seek to define this path, but I hope that through collaboration and engagement with the wider community, certain directions it might take will become apparent. The tutorial as presented here is based on 2 online sessions I ran with 11 videographic scholars. This allowed me to incorporate their experience (and issues) into the final tutorial in a way that I hope makes the process more readily useful to others who might watch it.

At the conclusion of the Sound Stack video essay, I suggest that perhaps the urban film soundscape is as anonymous as the Boston soundscape, which Southworth investigated in his 1967 thesis. During the period in which the work languished (as previously noted), I wondered if I should make another sequel—an iteration on an iteration, so to speak. But earlier this year, a research trip to Maine found me with 3 spare hours in Boston, which was just long enough to walk and record Southworth’s original 1967 soundwalk. The annotated video presented here (which is 88 minutes long) presents the entirety of the soundwalk, as I was able to follow it. I named the soundwalk ‘Taking Delight’ in part to refer to the concept of delight that Southworth sought within the urban soundscape. But here I am also reflecting on the deep sense of delight I felt in being able to practice this particular piece of research in a very real sense. The process of walking, re-tracing, and recording provided me with an opportunity to embody Southworth’s research in a way that I could not do simply by reading the thesis (similar in many ways to Eric Faden’s argument for making his first media stylos; why write about films when you can make films about them [2008]). I find again that Alan O’Leary’s words can help clarify some of my feelings about this soundwalk, with the following from his discussion on material thinking in relation to Catherine Grant’s Touching the Film Object (2011): “Material thinking is a form of critical intimacy rather than critical distance, the performative practice of which has to do with intervening, with making something happen, rather than representing, which implies separation from the object analysed” (2021). This was then my chance to take delight by intervening in Southworth’s research.

Methods 

As noted, the Sound Stack video essay makes use of a volumetric imaging process made possible through the use of the medical imaging program Fiji. My use of this method builds on the previous formative work undertaken by Kevin L Ferguson (2015). Beyond this formal method, the video essay also acknowledges its debut as a conference presentation through the inclusion of a ‘quiz, which was timed in such a way as to allow for participant engagement in the room.

The use of the medical imaging programme Fiji for this video essay was not a fait acompli, but rather an extension of a short annotation experiment that I published on my blog, Deformative Sound Lab, in April 2022. This initial experiment (and indeed the blog as a whole) is an attempt to engage with O’Leary’s “luxury scholarship” (2021), a provocation to videographic practitioners seeking to goad them into experimental work that might lead towards new deformative and parametric methods of practice. Where Sound Stack is inspired by Ferguson’s Volumetric Cinema, this first experiment reflected on his earlier work on Film Visualization, which also used medical imaging software to create a single image, summing together the frames from a film. In my version, I worked with a short clip from The Double, annotating the clip with all the sounds I could hear in it, then summing the frames from the clip together. The resulting summed image retains a ghost of the film soundtrack in these annotations, where the dominant sounds in the scene are rendered in bolder text while more sporadic sounds fade into the image.

Peer Reviews

All reviews refer to the original research statement which has been edited in response to what follows

Review 1: Accept submission for publication with no amendments.

The work in this portfolio revolves around questions of “sonic authenticity within film soundtracks.” “Authenticity” is of course a particularly thorny concept in videographic work, given the long history of technological developments in audiovisual recording, preservation formats, and dissemination. Are we hearing what was “meant” to be heard? Is our viewing/listening environment consonant with the recording environment? How would we even know? And, in Donnelly’s work, how much does the search for videographic “completeness” as a creator run up against the embodied, situated experience of living amid sound?

Donnelly’s portfolio oscillates between such larger philosophical questions and a more specific inquiry into a particular environment, treating both narrative and non-narrative audiovisual texts from the perspective of sound. As he notes, since the visual element tends to be privileged in scholarship, visualizing sound research is a necessary intervention in the field. How might one emphasize the audio in an audiovisual medium that historically has privileged the visual? Donnelly’s work argues in a way that this original “visual prejudice” can be combatted with new visualization techniques that make the audio visual.

This is most apparent in his Sound Stack project, which uses novel visualization techniques to render sound visually concrete and three-dimensional. Or rather: four-dimensional, as the primary qualities his method demonstrates are both on-screen location and duration. While labor-intensive, this technique allows the researcher to quantify an often subjective sense of the soundscape of a film. Donnelly concludes this piece by quoting John Belton’s argument that since “every sound must signify,” film soundtracks operate by a logic of elimination. Donnelly instead pines for the “acousmatic authenticities” of a non-curated sound, or rather a soundscape that does not necessarily serve a narrative or informational function, allowing a different involvement on the part of the auditor.

This interest is quite apparent in “Taking Delight.” It is worth noting that this piece is much closer to a feature-length than most videographic work. For me (and my own impatience with navigating cities), the (mostly) acousmatic duration of the piece draws attention to the act of walking—the unseen means of mobility which is nonetheless captured by dopplering sounds and other audio clues. I confess to initially skipping around a bit in the video on first view, but on a second examination I found myself listening and imagining, but more frequently pausing with anxiety that I had missed some text or important note. This was a sign to me to take Donnelly’s argument more seriously—the goal might be “delight” rather than mastery.

There are, though, interesting contradictions that arise in “Taking Delight.” I noticed in particular a tension between the self-stated attempt to “minimize my own impact on this recording,” such as noting when he adjusts the microphone’s windscreen, and a desire to identify “opportunities to interact with the city,” such as knocking on a mailbox. Donnelly has a bit of a quantum problem here, in that observation of sound phenomenon invariably brings a disturbance. “Intermittent interference is unavoidable,” Donnelly acknowledges, but I am still struck by the researcher’s dilemma of recording vs. participating. Indeed, the digital work is all about participation: rotating volumes, making decisions about text and timing, annotating identified sounds, confirming that a walking passage is indeed closed to the public.

This dilemma lies at the center of the idea of the “naturalness” of sound. This is apparent in “Taking Delight” when Donnelly comments on the lack of a “delightful” listening experience upon recording undesirably loud machinery being operated, or when wondering whether the relative quietness of electric vehicles might restore in some sense more natural sounds. The irony of course is that whole of the experience is being mediated. Wind becomes a signifier of this gap between recording and observing—something heard as it vibrates the microphone, filling space, adding decibels, covering electronic hiss or the raw static sound of any recording device, and even early on in this video becoming “offensively loud” as Donnelly puts it. Like a contemporary Aeolus, Donnelly seeks to control the wind, to which I wonder: is it possible to hear the actual wind or only the sound of wind on a recording device?

Last, as with all good digital humanities work, it is important to recognize that Donnelly is also generous in sharing his methods, offering a clear tutorial that could be adapted by other practitioners. This is valuable not only because it encourages future practitioners to extend upon his work as he extended upon others, but also because it builds a demonstrable practice of knowledge construction.

Review 2: Invite resubmission with major revisions of written statement.

Sound Stack, Soundwalk, Southworth is an engaged and thought-provoking research project/portfolio. It continues in a rich body of recent research that addresses how sound-based research can be expressed in the videographic medium.

“Sound Stack” is an intriguing video essay that addresses methodological concerns (the software Fiji; urban soundscape research), and theoretical issues (authenticity and place), all while being in dialogue with previous research by Donnelly. The accompanying video tutorial directly addresses the technology used in “Sound Stack.” It’s refreshing to see the creator sharing a niche and emerging research method which would otherwise likely feel unattainable by non-experts. The creator notes wanting to engage with a community of practice, and this is supported by the addition of the tutorial. Finally, the soundwalk, “Taking Delight,” adds a diverse piece to this set, being a near 90-minute practice-based intervention into Michael Southworth’s soundscape research.

All three pieces effectively participate in the reflexive research-as-practice theme of the portfolio. Sound Stack, Soundwalk, Southworth is a multi-layered, wide-ranging research project which successfully engages in the creator’s goal of “revision and expansion” and an intentional push back against what he notes as “the self-contained nature of the published video essay…”

I do however think the written statement needs refining. Because the three pieces are all what might be called “research journal entries,” a more explanatory written statement will tie the project together, bringing it focus and clarity. There are some sections in the statement that feel like a digression, for example, the discussion around “commentary, response, and riffing…” in the videographic community. And at the same time, key questions I have about the works are left unaddressed in the statement. While I don’t think an overly explanatory, and thus redundant, written component is desirable, I do think the project would benefit from more direct writing, per the suggestions that follow.

There needs to be more framing given in the methods and outcomes sections for “the annotated volume,” Ferguson’s volumetrics, and the application of Fiji. How is the 3D realm integral to this research? Why couldn’t you just use a 6 second text clip in Resolve, freezing the text on screen? Fiji is a fascinating method and wants to be the star of this project, but it feels underexamined from a critical standpoint.

As well, Southworth’s work serves as a framework and inspiration for “Sound Stack” and the project at large, but there isn’t enough discussion of the particulars of Southworth’s work with which the current research is in dialogue. For example, I sense that Southworth’s ideas on the anonymous urban soundscape inform the questions around authenticity, but that’s not made clear enough in the video or written statement for me. (Authenticity is a thorny concept, and while I don’t think the video necessarily needs more critical analysis of it, the written statement does.)

I enjoyed this engaging research project, and it’s made me think about my own research process, methods, and practice. In short, a more rigorous and focused written statement will make it sing.

All reviews refer to the original research statement which has been edited in response