Contemplations

This work is intended as a spherical 360-degree film, designed for optimal viewing in a headset, cardboard or otherwise. The immersive film is viewed by moving your phone around the space or in a headset. It can also be watched in equi-rectangular format at the following link: https://vimeo.com/194159424.

Research Questions

The research questions for this project are related to the sense of representation of place, especially busy city spaces, through use of immersive production and distribution technologies. This field currently has little research but is significant due to the increasing prevalence and affordability of 360-degree filmmaking tools. Our key research questions follow:

– What relationship can be established between elements of traditional documentary film-making and emerging forms of production?

– If experiential film enables a more immersive experience of place than traditional film, what implications might this have for representations of space and place?

– How can narratives be constructed in the film where an audience is not directed to observe and experience the events of a film in a particular order?

These research questions enable us to simultaneously interrogate how new technologies of production push the boundaries of documentary practice and to provide a tentative framework for how experiential 360-degree film technologies can be used by producers and experiencers to adopt a different relationship to the documentary text.

The starting point for the project was to use 360-degree filmmaking technology to record a space that we were aware of but had not previously visited and hence Contemplations in Chungking was shot on a first visit to Chungking Mansions in 2016. Famous as the setting for Wong Kar-Wai’s Chungking Express (1994), the Mansions is a seventeen-storey commercial and residential complex, described as “a world hub of low-end globalisation” (Mathews 2014: no page). On the ground floor, the building is a complex maze-like market forming the workplace to an estimated 4,000 people and receiving approximately 10,000 visitors per day. Hostels providing some of the cheapest accommodation in Hong Kong are situated on the upper floors of the Mansions.

In his Theory of the Derive, one of Debord’s rules for a dérive was that “the average duration of a dérive is one day, considered as the time between two periods of sleep” (1958:no page). So, our original aim was to ‘drift’ around the location and shoot a 360-degree film quickly and reactively over one day to capture the feel of the place as first-time visitors and consider how we could most effectively communicate this to what we have termed the experiencers, as opposed to an audience or viewers, in a virtual environment.

Once we encountered the location, it quickly became clear that this was a more complex space than envisaged, with complicated ebbs and flows of capital, goods and people as well as visibly sophisticated relationships between the residents and visitors. Therefore, Contemplations in Chungking was produced over two days, rather than the originally intended single day. The resulting film captures the essence and rhythms of the environment by enabling an exploration of the culture and communities that exist within the world of the Mansions.

Through our mode of production this film begins to explore the new forms of experimental/observational documentary that might be possible with immersive filming techniques, specifically 360-degree filming for VR experiences. The Methods section below details this process.

Context

Contemplations in Chungking is an immersive, experiential documentary.

Genna Terranova, former director of the Tribeca Film Festival, defines experiential storytelling as work in which “artists create wildly different adventures that go outside traditional methods” (2016: no page). Considering this in relation to VR filmmaking specifically, the ‘narrative’ is largely determined by a viewer’s interactions with the environment and the direction that they choose to view and engage. This offers considerable new opportunities and challenges for established documentary practice in the sense that such “adventures” may no longer be determined solely by the artist, or producer, of the film but more by the experiencer.

Our film is intended for viewing through a Virtual Reality head-mounted display (HMD) rather than on a computer screen, although watching on a smartphone or scrolling through a spherical view allows a limited sense of immersion. It can also be viewed on the YouTube app with a smartphone placed into “VR” mode and viewed in a HMD such as Google Cardboard. Despite viewing context, there is a directorial influence in what has been filmed, how it has been filmed and the framing that takes place through the modes of speech used. Counter to this there is also a possibility that the viewer either misses the thing that they are ‘meant’ to be looking at, as they may be looking elsewhere, or deliberately chooses to ignore it.

This key difference requires us to introduce a new term to define such viewers: experiencer. A full discussion of audience studies is beyond the scope of this statement but during our research around the viewing experience of 360-degree films it became clear that referring to the ‘audience’ of a film implies a sense of mass, however active, and using the term ‘viewer’ implies an individuated viewing, both of which feel inadequate to describe the experience of immersive VR films. Work in this vein requires a different set of terms as the person experiencing the film, normally through a head-mounted display, is more actively exploring the film space in ways significantly different to traditional methods of viewing. Consequently, an ‘experiencer’ is an individual viewing a shared film space in an exploratory manner.

The filmmakers are working on these notions through an emerging body of 360-degree filmmaking that will include the production of immersive, experiential documentary work and more sophisticated narrative film work along with explorations of the experience of viewing and specifically the addition of multi-sensory stimuli (primarily temperature and smell) at the moment of experiencing the film.

Methods

To interrogate the idea of VR film enabling a more immersive experience of place than traditional film, the production process for this work was very journalistic, observational and factual. It brought together the personal experience of the filmmakers entering a new space with its own cultural practices and interactions with residents inside the Mansions. The immersive process of situating ourselves within the environment, ‘drifting’ through the markets and the hostels while interacting with the visitors passing through and the residents enabled us to get a real sense of the feel of the place that is, hopefully, evident in the final film. This is formalised by the narrator’s voiceover.

The film takes the approach established by Woodhall (2015), especially the notion that the dérive should be an embodied movement and informed aimlessness. Drawing on the ideas of Debord, he refers to it as “walking at its most political and playful”. This exploration of the environment and existing communities allows for representation of, and experiencer reflection on, the complex social and political conditions that face the immigrant refugees living in Chungking as well as the visitors, however superficial this may be.

The film also draws on methodology demonstrated in Patrick Keiller’s films. Kellier’s practice uses the exploration of chance encounters within locations to reflect on wider global themes. During this approach, unstructured interviews with community members gave insights into their life and their struggles which allows for an individual voice to allude to the more complex power relationships outside the closed environments of the Mansions. The realities of being a refugee in Hong Kong; immigration, unemployment, constraints in communities were evident themes and the significance of these chance encounters enabled a form of narrative, however loose and unstructured, that is representative of the environment. The subjects offer a sense of an insider looking out while the narrator’s voiceover provides the exploration of the outsider looking in. A large focus within Keiller’s work is the exploration of contrasts “between the familiarity of old city fabric, the strangeness of the past, and the newness of present-day experience” (Keiler 2003) which is used similarly to stimulate the sense of narrative within Contemplations in Chungking. Through this approach contrasts between the ambiances and activities within private rooms, hostels and shops in the Mansions are open to the experiencer. There are established stable living spaces of the hostels as opposed to fluid, dynamic and ever-changing spaces of the market.

The experiences within Chungking Mansions were chaotic and confusing. There was a lot of noise, a lot of jostling and a lack of sense of time and place. Creating this feeling within an experiential film meant playing with the technology and breaking established filmmaking conventions. This lends itself to a perspective of experimental filmmaking as pushing the boundaries of mainstream filmmaking.

One method of achieving this disorientation was through identifiable stitch lines. This film was made using a 360-degree rig, consisting of two back-to-back cameras. Images from the two cameras were then stitched together to form one larger spherical image. This set up creates a visible stitch line at times, especially when people get closer to the cameras. As in much experimental film work, this foregrounding of production process remained in the final cut to add a sense of unease to the audience rather than being rectified in post-production. At times, you can see people appearing, almost ghost-like, as they move between the camera stitch lines which gives this sense of uneasiness and could play on the experience of being within the environment.

Another technique to create disorientation was our positioning of the camera and people in relation to it. Earlier research (Jones 2016) identifies the invasion of personal space as a concern for audiences within virtual environments. If people get closer than one meter to the camera it feels as though that they are too close, creating a sense of unease. This was then used as a way to enhance the experience. In one scene, a man walks past the cameras, takes a second look at them and then comes back to stand close by, peering down the lens – this proximity to the camera means that he takes up 85% of the field of view. When viewed within a HMD, the intensity of this is heightened, startling the experiencer and perhaps making them feel uneasy as if the man were standing very close to them.

Our initial idea of the dérive was maintained to a small extent by playing with a tracking shot around the Mansions. In a spherical film, this can sometime elicit nausea in the experiencer but we decided it was the best way to capture a sense of exploration, space and place most authentic to capturing this environment. The camera operator appears in the film during this shot, which adds to this sense of an experiencer being carried through the environment in an unusual viewing position. It is an unreal experience of a real place.

Outcomes

To explore the relationships that can be established between elements of traditional documentary filmmaking and emerging forms of production we used 360-degree cameras but added two of key elements of traditional documentary during post-production to see how they functioned together.

Corner identifies the Expositional Mode and Evidential Mode 2: Testimony (1996:27-30) as two classic ‘modalities’ of documentary speech. Contemplations in Chungking uses both in a relatively traditional manner as the ‘voice of God’ of the narrator is supplemented by the testimony of two of the residents of the Mansions. This provides some sense of narrative structure during what is essentially an observational piece. The narrator is not seen, or at least not recognizable in the film and their spoken words do not have direct connection to the visuals. Instead these voices act as more of a reflection and detail the thoughts of the subjects to add colour to the experience. This brings a more experimental filmmaking technique to a traditional narrative approach.

The narration and testimony functions paradoxically by simultaneously expanding our sense of space whilst also limiting it. As voices and images are not synced, we allow the experiencer a sense of freedom to explore. The voices allude to the complexities of the social relationships inside and outside the Mansions but the visuals seem to represent an ‘informed aimlessness’ as suggested by Woodhall (2015). The voiceover limits the viewing experience by structuring the ‘narrative’ but the experiencer can choose to ignore this structure creating a level of agency no possible in traditional documentary. If the experiencer chooses to ignore the provided narrative, they may have a better sense of the space and the physicality but will possibly miss the power relationships alluded to in the narration. By not making the subjects visible we have made them invisible, effectively air-brushing them out of the film in favor of the agency of the experiencer. This film experiments with these ideas and future works might continue to explore how such complexities are presented.

Another outcome of our field research was a realization that, as per more traditional filmmaking, our shot selection was important in highlighting and exaggerating some of the implicit power relationships that exist as outsiders entering the space of the Mansions. For example, the point of view of the tracking shot which elicited a feeling of being carried around the environment for an experiencer. This is an unnatural feeling so breaks the presence that an immersive VR film usually attempts to achieve.

Future Work

This project attempted to understand how traditional filmmaking techniques might be adapted for immersive VR filmmaking and more importantly what new techniques may need to be developed when using emerging technologies of production and distribution. Traditional ideas toward framing and narrative must be challenged to construct a different type of film where an audience is not directed to observe and experience the events of a film in order but is free to experience whatever aspects they choose to engage. This has a profound impact for auterial filmmaking. In this film for example different elements such as narrative structure, shot composition, voiceover, testimony and the final edit – function in an explicitly different manner to traditional documentary form. Essentially, the film disorientates rather than orientates and cannot “speak for itself” in a way that a more linear, structured film would.

This is profoundly exciting for filmmakers and other practitioners working with immersive VR technology. It requires an awareness of filmmaking contexts, histories and practices but also preparation to invent, explore, play and fail at new practices to properly understand the possibilities and limitations of emerging technologies and modes of experiencing. We found for instance that experiencing this film through a HMD can lead to an increased sense of immersivity but also act as a distancing technology, reducing the sense of being in a specific place. So, in further field research related to this film we found that increased levels of immersivity may require other sensory input, such as heat and smell (Jones and Dawkins, forthcoming).

Dissemination

This work was self-funded as a research study to identify new forms of media practice that combine traditional documentary film with emerging technologies.

Research papers based on this work, along with other practice pieces, are currently being submitted to festivals and academic conferences that explore new technologies, practices and ideas about society. It has been accepted at:

The International ARVR conference in Manchester (February 2017)

International Conference on VR in Hong Kong (May 2017)

The work was also screened as part of the RTS Virtual Reality Showcase in London, November 2016.

Additional Information

If anyone is interested in further investigating this emerging area or any of the issues raised in this piece, we would be more than happy to work together on this.

References

Clarke, D (2007) The City of the Future revisited or, the lost world of Patrick Keiller, in Transactions of the Institute of British Geographers, January 2007, Vol.32(1), pp.29-45

Corner, J. (1996) The Art of Record: A Critical Introduction to Documentary. Manchester, Manchester University Press.

Debord, G. (1958) The Theory of the Derive. Accessed at: http://www.bopsecrets.org/SI/2.derive.htm. Last accessed: 27/11/16 at 15.05

Debord G (1996) Theory of the dérive in Andreotti L and Costa X eds. Theory of the dérive and other Situationist writings on the city Museu d’Art Contemporani de Barcelona/ACTAR, Barcelona 22–7

Keiller, P. (2003) The City of the Future City 7 376–86

Matthews, G. (2014) Accessed at: http://travel.cnn.com/hong-kong/life/inside-chungking-mansions-expert-gordon-mathews-098440/. Last accessed: 2/2/16 at 12.10

Terranova, G. (2016) https://tribecafilm.com/stories/tribeca-film-festival-2016-innovation-storyscapes-def-con-mr-robot-virtual-reality, Last accessed 18.11.16

Woodhall, A. (2015) http://reframe.sussex.ac.uk/wp-content/uploads/2015/03/Some-methodologies-philosophy-for-psychogeography-practice-CLHLWR.pdf Last accessed 1.12.16

Peer Reviews

All reviews refer to original research statements which have been edited in response to what follows:

Review 1: Accept work subject to minor revisions of statement
The authors/filmmakers of Contemplation in Chungking bring into question how new technologies, for both media production and viewing, might change the relationship between the viewer and the text. By both experimenting with emerging production techniques for a 360° experience and providing a theoretical framework for a specific spatial experience, the authors do begin to scratch the surface of representation of space and movement of space through immersive 360° video in this important emerging media format.

The film truly is best viewed through an immersive virtual reality headset. At first viewing I did not have an immersive viewer on hand, and I watched and moved around the film in the equirectangular format on both my phone and on a computer screen. I will admit, I watched before I read the supporting Statement and made a list of comments, immediate concerns, and fears for what the film might look and feel like in the immersive viewing format. I then watched the film again in an immersive viewer. It felt like a different film from that viewed on a computer or mobile screen. Many of my initial concerns, but not all, melted away once I was able to experience the film under more immersive conditions in the VR headset. The scale of the space was more easily understood, especially while standing rather than sitting. In addition to the experiential changes felt within the immersive experience, a number of my initial concerns and issues were touched upon and responded to by the authors within their Statement, including the inclusion of stitching seams and the presence of the filmmaker in the shots.

The authors frame this immersive experience as a dérive, as space represented through embodied and aimless wandering through a new place over the course of a day. In the film, the movement from the lower floors, full of frenetic energy, up to the more intimate spaces of the hostel gives the feeling of not only traveling through different spaces of the same building, but also of the passage of time throughout the day in an otherwise atemporal space, only to return again to the main spaces. Though this technique worked well to highlight the unique flow of both indeterminate time and space within the Mansions, the disembodied voices utilized for both journalistic overview and by resident narrators of Chungking Mansions, created a bit of a jarring experience. A viewer, or “experiencer” as the authors believe this new kind of immersive experience viewer should be called, can easily become distracted, looking for where the voice is coming from and what it is describing within the space, only to be met with neither an audio or visual referent. Though the authors do frame this creative decision within the context of trying to best tell stories of a community within the Mansions while providing insight into the farther reaching power relationships and political implications, this highlights a problematic of immersive storytelling and filmmaking – that of giving the viewers agency and freedom to explore whilst these same viewers must learn a new language for viewing. In this case, and though the authors made a conscious decision to use audio and visual in this way, time was spent on my part both looking for the community member speaking and also trying to map within the space what they might be talking about. This, again, is a matter of relying on more traditional techniques of documentary filmmaking, in this case Expositional and Evidential narration, and finding their counterparts in immersive virtual reality filmmaking, which might not mirror one another.

There has been a small explosion in the production of 360° films since 2015. Many of these have been similarly documentary in style, in which a viewer is transported to, and embodied within, a setting distinctly different from their everyday lives. There is both a learning curve for filmmakers and viewers/experiences as to what the language of immersive storytelling is or might be. Contemplation in Chungking is asking the right questions in regards to production techniques and how to tell stories for immersive experience with new technologies, and at the same time are challenging some of the emerging best practices in regards to narration and stitching and editing, towards further experimentation in a nascent field.

Structurally, the Statement throws the reader into its intent and methodology quite quickly, just as the film throws the “experiencer” into ChungKing Mansions. However, whereas the film gives us some nice quotes to ponder just before our journey into a new space, the Statement needs a bit more framing. The Statement would be well served by opening with the larger research questions about the changing nature of the relationship to viewer and text, as well as the implications for representations of space in immersive experiences. Furthermore, an overall restructuring of the argument throughout the Statement is needed to create a better flow of ideas and finding to more seamlessly weave the practice-based research and techniques into the more theoretical research. This could be done simply by expanding a bit on the work of those referenced throughout the paper in more direct relation to the work being investigated in this Statement. Additionally, very few references are made, or provided in the bibliography/filmography to any of the immersive/virtual reality documentaries and how this Screenwork relates to other developments in the rapidly growing field of filmmaking and academic inquiry.

Review 2: Accept work subject to minor revisions of statement
Contemplation in Chunking joins the small but growing group of documentaries that have been filmed in 360 degrees and are intended for use on immersive headsets. Having worked with immersive media myself, I have a sense of its strengths and weaknesses; but one particular challenge with the format is trying to provide the same clarity of narrative and direction that can be achieved with the traditional single-point view film. The audience’s acculturation to the screen sitting directly in front of them (widescreen and IMAX formats notwithstanding) means that they expect the action to unfold as if seen through a window, via the camera lens that directs their gaze towards whatever the director deems to be of most interest. Even the much-vaunted stereoscopic films of recent years only perpetuate this concept because their “3D” illusion only takes place within the constrained space of the forward-facing screen; and the in practice they offer little beyond the established formats of the cinema, which might account for their fading popularity.

360 filming is fundamentally different in that the camera sits at the centre of the scene and takes in everything around it. The advent of digital cameras and on-the-fly processing has enabled multiple viewpoints to be stitched into a single surround image in realtime; and the format has advanced very quickly over the past decade. The resulting 360 film can be screened in specialist viewing setups such as multiscreen projections or dome cinemas, but most audiences will see it through headsets such as the Google Cardboard. Although the resolution and acuity of these displays varies greatly depending on the hardware (from low-end mobile phone screens to the most recent iterations of the Oculus Rift and HTC Vive), the fundamentals are the same: an immersive display that tracks around the scene according to viewer head movements. This can also be delivered via YouTube to a standard computer screen and the viewpoint can be moved using a mouse, but the immersion is of course much reduced in this case,

This preamble is necessary because 360 films are only just beginning to establish their own grammar. At present, many of the 360 videos available on YouTube are from action cameras recording skiing, snowboarding, underwater exploration and similar activities. In these cases the motion of the camera itself provides a basic “narrative direction” and immersive sensation is part of the vicarious thrill of the second-hand adrenaline rush that these videos provide.

In terms of more complex cinematic experiences, documentaries such as Battle for Mosul are filmed on or near the frontlines of contemporary battlefields where frenetic activity surrounds the camera and much of the material is left unfiltered for viewer interpretation. What I mean is that the 360 degree picture enables the viewer to explore more fully in all directions than the film-maker themselves who is likely to be progressing under fire, and with only one direction in mind. Although a narrative direction will inevitably be introduced at the editing stage, these documentaries are about “environmental capture”, to coin a phrase: the capturing of things surrounding the filmmaker that might be peripheral to the documentary’s development but can be explore in more depth later on. In other words, although the viewer is taken where the filmmaker wants the camera to go, they have the freedom to explore other parts of the scene than the filmmaker’s chosen focus, and this absorption in the environment is where 360 documentaries can fully differentiate themselves from single-viewpoint films.

With this in mind, Contemplation in Chungking succeeds as a 360 film because it enables the viewer to absorb the environment in detail; most of its narrative is supplied through voice-over and thereby overcomes the static position of the camera. This also means that the frenetic and unceasing activity around the shops and pathways of Chungking can be better observed and considered: visual narratives emerge in the patterns of people and the recurrence of certain faces in the course of the film; and the culture and ethnicity of those who pass along the hallways. In a place that would be confusing and claustrophobic to the “eye” of a single-point camera, the 360 film opens up the space in a very different way that gives it further meanings.

This certainly fulfils the authors’ intention to capture the “complicated ebbs and flows of capital, goods and people and where sophisticated relationships between the residents and visitors were visible”. Perhaps because the 360 film is naturally inclusive (or at least harder to “frame” in the traditional sense) it is peculiarly well suited to spaces like Chungking and also enables repeat viewings that allow one to focus on different parts of the scene. The concept of a Debordian dérive fits the pacing and the sense of “embodied movement and informed aimlessness.” Of course the experience is still structured in various ways, not least the speeding up of time and the choice of locations; the voiceover also maintains a linear narrative that anchors the flow of people around the viewer. That said, the transitions from site to site are quite abrupt; whilst this is probably deliberate, it does mean that the viewer’s journey is broken up into stages and there isn’t enough time spent in any one place to fully appreciate it.

Where I am less convinced is the authors’ assertion of their connections to Patrick Keiller’s methodology. Keiller’s ruminations on London tie narrative and motion closely together; the observations tend to mark specific locales in the context of a much longer walk through streets that are largely empty. The personal reflections of “Robinson” knit the films together much more strongly than the low-key incidents of the narrators in Contemplation in Chungking and I have to say I find Keiller’s work more compelling in this area. Also he is deeply embedded in the history of the areas he observes and this is an essential part of “psychogeography”, which is nothing if not historically informed. There should be a greater sense of where and what Chunking Mansions is; there should be some attempt to frame it within Hong Kong and give it more sense of location. Though fascinating in itself, the narrator’s comment that “this could be anywhere” is somewhat true (for the wrong reasons) and as it stands it seems too self-sufficient.

Whilst I appreciate the authors’ desire to connect their film to an established voice and method, in this case I don’t think that Keiller provides a successful pattern for a 360 film. His personalised viewpoint demands single-point filming, the world seen through one pair of eyes. The frenetic action of Chungking demands something else.

In conclusion, then, I think this is a fundamentally interesting piece and should be published; but perhaps with more critical reflection on the relative usefulness of the Keiller model; and a recognition that the narrative and structure can disorient the viewer, in ways that don’t allow Chungking to speak for itself. The Research Statement should be further developed to examine the three research questions in the light of the questions raised above:

“If experiential film enables a more immersive experience of place than traditional film, what implications does this have for representations of space and place?

What relationship can be established between elements of traditional documentary film-making and emerging forms of production?

How can narratives be constructed in the film where an audience is not directed to observe and experience the events of a film in a particular order?”

Each of these can be answered by the filmmakers and this reflection will be a valuable addition to the growing genre of 360 documentaries.

All reviews refer to original research statements which have been edited in response.

Back to Volume 7.1

Page updated

Google Sites

Report abuse

Table of Contents

Practice

Research Statement

Criteria

Viewing Conditions