From our forthcoming special issue, Geospatial Memory (Media Theory 2/1), Matthew Flintham describes the immersive, location-based urban experiences as well as the uncanny and unexpected psychological responses of users of geolocative media.
BATTERCTRAX: Observations of Sensory Dissonance, ‘Doubling’ and other Residual Effects of Geolocative Media
MATTHEW FLINTHAM
Kingston University, UK
Abstract
This paper will describe BATTERCTRAX, an experimental geolocative media project undertaken during 2014. It examines the possibility that cinematic media (in this case audio from films) can be repurposed within an immersive, mobile heritage guide. The paper will describe the project in broad terms but will focus on certain unexpected and unusual perceptual effects generated during the test phase of the project. BATTERCTRAX began as an adjunct to a larger academic initiative called Cinematic Geographies of Battersea: Urban Interface and Site-Specific Spatial Knowledge, a collaboration between Liverpool, Cambridge and Edinburgh Universities and English Heritage. Amongst many outcomes, Cinematic Geographies built a comprehensive database of films shot in the London district of Battersea, which created ways to mobilise historic moving images for the analysis of social and material change in cities. As a tangential outcome of this project, BATTERCTRAX was a mobile phone app that played audio content from feature films, documentaries and TV shows at the places where they were originally recorded. The user would walk through clusters of GPS-enabled geofences in the test zone of Battersea Park, triggering a succession of historic cinematic sounds from across the 20th century. However, by anchoring fictional audio content to the places of their origin, BATTERCTRAX appeared to create a sense of dissonance between sensory stimulants, destabilising a sense of perceptual cohesion in the user. These novel effects lead the researchers to speculate that such technology could not only be used to construct highly immersive, location-based urban experiences, but that it could also trigger psychoactive effects in ways that media developers and users had not anticipated. This paper will propose that geolocative media has the ability to tap into unexplored realms of “collective” cultural memory, but can also elicit unexpected psychological and perceptual responses.
Keywords
Cinematic, Database, Film, Geolocative, Urban
Introduction
The following article describes a geolocative research project undertaken in 2014, created to distribute audio media to mobile device users as they trigger GPS-enabled geofences placed within a test zone in central London.[1] BATTERCTRAX, as the research project became known, can be placed within the field of experimental locative media, but also within a nascent digital curatorial practice which exploits “the convergence of computer data and location using portable media” (Reiser, 2005: 2). It describes the genesis of the project, its production and its testing in the field, but focuses in particular on certain unexpected and unusual perceptual and cognitive effects generated by the technology. The article also situates the project within a speculative avenue of digital media studies, exploring the notion of media archaeology (see Huhtamo and Parikka, 2011; Parikka, 2016; Zielinski, 2008, amongst others) as a practical methodology, a praxis by which media is identified, archived, reformatted and redistributed for a new audience. It will also explore the emergence and practical application of database cinema (see for example, Alifragkis & Papakonstantinou, 2017; Keiller, 2007; Manovich 1999, 2001; Penz and Koeck, 2017) as a means by which moving images and associated audio media are analysed, edited into discreet packages and stored on a server, then organised according to a set of predetermined categories such as date, director, actors, filming locations and genre. The media can then be identified and retrieved by category.
Detailing what was intended as a proof-of-concept audio heritage guide, the article begins by contextualising the BATTERCTRAX smartphone app as an experimental adjunct to a larger academic pilot project called Cinematic Geographies of Battersea: Urban interface and site specific spatial knowledge (Liverpool, Cambridge, Edinburgh universities, and English Heritage, 2012-14).[2] The main objective of this 18-month pilot project was to find new ways of mobilising historic moving images (feature films, documentaries and TV shows) as evidence for research in urban studies. Here, I examine the notion of media archaeology and database cinema as a way to demonstrate the methods by which moving images and audio were remediated and redistributed by the Cinematic Geographies project. The article focuses on how BATTERCTRAX emerged from the Cinematic Geographies project as a way of exploring experimental methods of media distribution, particularly from the extensive database of the mother project. Moving on to the development of BATTERCTRAX, the next section describes the technology used to “virtually” place audio media (in this case, audio extracts from feature films, documentaries and field recordings) around Battersea Park using overlapping geofences, and how the user’s experience is dependent on a smartphone app, a continuous GPS link, and connection to a mobile telecom network. While location-aware heritage guides are increasingly common in urban settings, BATTERCTRAX was developed to test new technologies that would help users explore the cinematic history of Battersea Park and the surrounding area, but was defined by experimental curatorial methods, extensive research of film locations, and the serendipitous potential of sonic collage. The resulting experience is a flow of overlapping audio tracks, triggered as the user enters a succession of geofences around the park, creating a collage of “spatialized fictions”. As such, this section of the article will position BATTERCTRAX within the field of sound geographies, soundscapes and digital sonic design. It then describes the observations and unexpected experiences of five test subjects as they use the app to explore the Battersea Park test area. The author’s own extensive testing and observations of using the app in situ will provide the basis for a speculative examination of locative technology and its indefinite influence on perception.
The section Media Memories of Place further explores media archaeology and database cinema as the operative principles for the practical dissemination of audio by BATTERCTRAX, and identifies “collage” as the method by which the app presents media to the user. This section will also seek to contextualize the app within the discourse of digital technology and its relationship to human memory. Here, collage can be understood as the mode by which “place-based memory” is accessed, accompanied by what can only be described as a sense of the uncanny. In the final section, Uncanny Media and Residual Effects, the recent resurgent interest in the concept of the uncanny becomes a useful point of departure for a speculative interpretation of unusual cognitive states and effects generated by locative digital media. Here, hauntology will be discussed as a recent permutation or extension of the uncanny in media discourse, and will also serve to elucidate BATTERCTRAX’s test subject responses to audio media. This section attempts to distinguish between the unusual perceptual and cognitive effects generated within the user experience of BATTERCTRAX in relation to other location-based and virtual reality experiences. It will explore the possibility that such technologies can induce a variety of subtle altered states which may continue even after the user has stopped using the technology.
Before describing these effects in the following text, a clarification of the essay title may be useful: here the key term “dissonance” can be understood as the consequence of a momentary or prolonged de-synchronising of the senses, where the user’s visual reference points do not quite match the sounds they are hearing through their earphones. For example, the user might be using the app and watching a tennis match in the park but the sound of ball-hitting-racket is de-synchronized from the visible action. Another example might be hearing a bicycle pass very close by, but the nearest visible bicycles might, in fact, be some distance away. These unsettling effects, while not always repeatable, are also referred to as “doubling,” inferring a presence which is disconnected from its material source. However, the “residual effects” referred to in the title, are harder to classify: they include a continuation of the doubling effects even after the user has ceased using the app. While clearly the consequence of geolocative technology used in situ, the doubling effects nevertheless provoke questions regarding the future of such technologies and their potential to disrupt the senses in unpredictable and perhaps undesirable ways. In seeking to interpret these effects, the following text will, to borrow from Parikka (2016), explore “new technological, urban environments acting as conduits for altering structures of perception, experiences of temporality and memory” (7).
Because of the constraints of copyright law, BATTERCTRAX is not available for public distribution, and in situ testing of the app was limited to five volunteer subjects via the Test Flight beta testing app. However, their experiences of using the app (and indeed, the author’s own experiences during and after the development phase of the project), were so compelling that they warrant further analysis and interpretation. The purpose of this article then is to evaluate the test subjects’ observations, and the author’s experience of developing and testing the app. With this in mind, the article takes the opportunity to address the unexpected outcomes of BATTERCTRAX, and in doing so aims to contribute to the emerging debates on immersive, geolocative media and social memory.
Cinematic Geographies
The story begins in 2012 during work on Cinematic Geographies of Battersea: Urban interface and site specific spatial knowledge, a pilot collaboration between Liverpool, Cambridge and Edinburgh universities, and English Heritage. The aim of the project was to find ways of mobilising films and moving images as evidence for researching social and material change in cities. We sought to find more effective ways of using film to study the changing “soft city” of communities and social relations, and the evolution of the architectural “hard city” (Raban, 1974). The Cinematic Geographies project can be placed within an emerging field which fuses urban and architectural studies, and film studies. It endeavours to interpret the structural composition of cities and the urban experience through the analysis of film. It also seeks to examine the distribution and impact of moving images within cities as ubiquitous advertising and screen surfaces become a defining feature of contemporary urban life. For a comprehensive genealogy of this emerging field, see Penz and Koeck (2017).
Most relevant for this article is work undertaken by Koeck (2012) and Koeck and Roberts (2011), which relates to the quantitative analysis of film and moving images.[3] Significantly, it was through this work, and the work of Hallam and Roberts (2011), Misek (2012), and Caquard and Taylor (2009) amongst others, that the concept of “cinematic cartographies” emerged – the mapping of shooting locations to study the spatial dimensions of film. As Hallam and Roberts (2011) suggest,
The multilayered and dynamic model of urban space represented by GIS-based [Geographic Information Systems] cinematic cartography allows renewed reflections on the role and place of moving image cultures in the production of everyday social spaces (369).
From this foundation, the Cinematic Geographies project began with the assertion that film, because of its rapidly shifting content and lengthy duration, was a woefully underused resource in geographical and urban analyses. Instead, could the massive wealth of TV shows, feature films and documentaries languishing in archives around the country, be unlocked and used as a tool to reveal social and structural changes in the city? We believed that if we could find ways of taking films apart and storing each scene by geographical location in a database, we could then more effectively compare visual evidence from different times over the course of the 20th century. Drawing on publicly accessible film archives (a number of which had already usefully tagged their entries with filming locations), Cinematic Geographies began by compiling a master database of films shot entirely, or in part, within the municipal district of Battersea in London. Expecting 40 or 50 films, the project actually uncovered nearly 600 films, TV shows, documentaries shot in the area, swamping the researchers with moving image material and metadata. Each film was then watched, and any relevant scenes of Battersea were then identified and extracted, tagged and located on a searchable online map. Then, by clicking on one of numerous place-markers, it was possible to retrieve film scenes which were shot on that particular location. In this way, an historical stratum of visual evidence could be accessed at the click of a button.
This method of organising a database of film scenes by their geographical location constituted a practical form of media archaeology. Here, seemingly disparate content on obsolete formats is retrieved from the deepening strata of cultural production and reordered using metadata attributed to each media file. It is a form of remediation in which images on recording formats such as nitrate film, cellulose acetate, magnetic tape and DVDs find “an afterlife in new contexts, new hands, new screens and machines” (Parikka, 2013: 3). However, here is a form of remediation which is defined by the geographical origins of the media’s subject – the place which the media represents and/or where it was recorded.
The Cinematic Geographies project demonstrated how digital technologies could develop new methods for identifying and mining lost media, and creating new channels for its redistribution. However, throughout the project, we knew that our publicly accessible outputs (apps, websites, conference papers, visualisations etc.) would be significantly constrained by copyright law, which requires permission to use certain media, and fees to be paid to the copyright holders (usually major studios or media corporations). We worked around copyright laws as best we could, using out-of-copyright material and very short scenes within the parameters of “fair use”, and by transposing educational commentaries over moving image material. While these solutions solved our immediate issue with copyright and the public distribution of our outputs, they also highlighted how an effective remediation of moving images would be limited not by technology but by the laws protecting the distribution of images or the prohibitive costs of purchasing the rights to use such media.
By the end of the Cinematic Geographies project, a series of methods had been developed to make the database searchable, mobile and accessible to the general public and to academic researchers alike (Penz and Koeck, 2017). Amongst these methods, a mobile app called Ghost Cinemas was developed which allowed the user to view moving image content at the locations where they were originally filmed: once a geofence had been triggered, the user was presented with several film clips showing the location at different points in time. While the Ghost Cinemas app has been addressed at length elsewhere (Speed, Thomas and Barker, 2017), such technology seemed to promise new avenues of dissemination and pose some intriguing questions regarding geolocative technology and public access to media databases. To explore the full potential of these technologies, we decided to conduct an experimental project that would not be accessible to the public and would, therefore, be unconstrained by the laws prohibiting the use and distribution of media. This experimental project became the BATTERCTRAX mobile app.
BATTERCTRAX: Mobile Access to Urban Audio Archives
Buoyed by the potential of mobile geolocation software and the large media resource made accessible by the Cinematic Geographies database, we then gave ourselves license to experiment with emerging locative technologies and the expedience of the smartphone app. We wanted to see if it was possible to “curate” the distribution of media more precisely across a specific geographical area, and like the Ghost Cinema app described above, we wanted to re-establish the link between the media and the place where it was generated. Working with technologist and academic Alex Butterworth and his media company Amblr, we began to explore novel ways of engaging with place, ones mediated by cinematic sound rather than images, and walking as a mode of engagement. There were a number of reasons for focusing on sound from moving images rather than the images themselves. First, the Cinematic Geographies project had concentrated almost exclusively on the visual dimensions of film and its relation to place and geographical locations, which also meant a concentrated, ongoing engagement with smartphone, tablet and PC screens. Therefore, the project team wanted to explore the possibility of engaging with filmic history and project data without constant reference to screens, and employ a locative technology “where the auditory can finally stand on an equal footing with the visual” (Rieser, 2005: 15). Second, the Cinematic Geographies project group wanted to address the importance of filmic sound and its link to place through location recordings – after all, sound recordings in films offer a unique body of evidence for research into urban change. We began testing mobile technologies that could “return” audio from feature films, TV shows and documentaries to their point of origin, to the place where they were recorded. Not only did this process seem like an innovative way of bringing historical evidence into the public domain, but it also seemed like a novel way of exploring the practical, real-world applications of our database, and perhaps advancing the concept of database cinema itself.
We were encouraged by recent developments in soundscape studies, geographies of sound, and arts practices concerning the “design” of urban soundscapes (Bijsterveld, 2013; Labelle, 2010; Sterne et al, 2012). Other contemporaneous projects which emphasised audio distribution and mobile technologies have since been summarised in Audio Mobilities, a special issue of Wi: The Journal of Mobile Media (2015).[4] We were also encouraged by the notion of “placed sounds” and sonic interaction design within the context of emerging locative mobile technologies where “the distribution of sound in space is pre-curated, and users create their own version or remix of the service by choosing their path through the sounds” (Behrendt, 2012: 286).
Additionally, we were very conscious that our technology used computer code to actualise and define invisible parameters of spaces: like civilian and military airspaces which regulate the air traffic above us, geofences exist as paradoxical code spaces (Kitchin and Dodge, 2007; 2011), which do not, in any material sense of the word, exist at all. And yet, they define movement and mobility around the planet in ways that would have been inconceivable a century ago. The invisible parameters of locative media are code spaces at the local scale, defining navigation, entertainment and commerce for individuals at street level.
Within this context, we asked the following questions: could the BATTERCTRAX app productively explore the cultural history of place through the creative application of sounds from films? Could the app deploy media from the Cinematic Geographies database to create a “cinematic soundscape” of Battersea? To answer these questions, we needed to methodically study the geographical distribution of film locations across the Battersea area. Filmic heritage invisibly saturates the district of Battersea, but to unlock these cultural artefacts first required us to return to the database outlined above and determine which areas of Battersea had high densities of historic filming locations. We identified Battersea Park as a suitable site for an experimental geolocative sound project.
Perhaps because of its proximity to the iconic Battersea Power Station – with its four distinct chimneys defining the south London skyline – Battersea Park has a rich cinematic history. The park has been used throughout the twentieth century as a location for films such as The Day the Earth Caught Fire (1961), Brannigan (1975), All the Right Noises (1971), The Mutations (1974), The File of the Golden Goose (1969), Plenty (1985), Journey into the Unknown (1969) and many other feature films, news reels and documentaries. Within each of these films, it was possible to isolate scenes shot in Battersea Park and identify their exact location. For instance, in All the Right Noises, the two principal characters briefly walk by the boating lake in the park, intimately discussing the future of their doomed love affair. Similarly, in the Day the Earth Caught Fire, the two main characters are stood in the park on the edge of the River Thames witnessing the sudden appearance of a thick bank of fog as it engulfs the river and parts of the city. After identifying dozens of such scenes set in the park and plotting the parameters of their locations, it was then possible to separate the audio (dialogue, music, background sounds etc.) from each media file. We then used authoring software provided by Butterworth and Amblr to fix radial geofences to the locations where each scene took place. Next, we attributed media sound files to each geofence, so that audio could be unlocked on the user’s smartphone app whenever a user walks into any of the invisible GPS trigger zones. After several days of iterative testing, we were able to position over one hundred geofences and audio files around the park, refining their sizes and distribution as we worked. Even in these early stages of the research process, it became clear that something unusual was happening. A walk in the park while using the app became infused with fictional encounters: invisible characters whispered to each other as you pass where the scene originally took place, crowds of children cry in delight at the site of a funfair (since raised to the ground over four decades ago), and earnest British actors solemnly discuss the shifting of the earth’s axis and the end of human civilisation. As the user traverses multiple, layered geofences, streams of dislocated sounds blend into each other, a collage of fictional fragments lingering at the sites of their derivation. In this way the author, Butterworth and Amblr developed a form of “sonic interaction design,” creating an interactive loop between audio content, environment and user (Berendt, 2015). We believe we succeeded in creating a proof-of concept “cinematic soundscape” of Battersea Park triggered by the user’s movement around the park, drawing on a location-centric database of moving image media. If the act of researching and identifying material for the database constituted a practical form of media archaeology, then the act of deploying that media could potentially reveal a unique cultural history of the area.
The priority for BATTERCTRAX was to create a proof-of-concept app which develops new methods for distributing media from a database to users in specific locations. While there was never any expectation to test a beta version with a larger group of test subjects, the early accomplishments of the projects and the occurrence of certain unexpected perceptual effects generated by the app, prompted the researchers to convene a test user group drawn from the extended Cinematic Geographies project. Five test subjects were given mobile phones pre-loaded with the BATTERCTRAX app and asked to simply walk around the park and return at a pre-arranged time. The test subjects were then interviewed in situ about their initial observations and experiences. While each subject took a different route around the park (and therefore triggered a different sequence of audio files), all reported that they were successfully hearing a mixture of audio from films, pre-recorded field recordings of the park, and ambient sound beds filling the gaps between “placed sounds.” Some users expressed surprise and delight at hearing fictional scenarios from films and TV shows which somehow seemed to be linked to the place where they were standing. While this apparent paradox can be understood by those with an understanding of the technology employed to create the effect, it did seem to provoke a startled response in some users. However, while these observations might constitute a successful outcome for the test, all test users described certain unusual and unexpected effects when using the app: some claimed at times to find it difficult to distinguish between real sounds and pre-recorded “placed” sounds. This led to some users momentarily removing their headphones to determine the authenticity or origin of particular sounds. This temporary inability to distinguish between real and pre-recorded sounds, or real and fictional scenarios, was a puzzling outcome, and one of a number of unexpected effects encountered during the development stage of the app. Encouraged and puzzled by the test user observations, the author continued to refine BATTERCTRAX in situ, using the authoring software to place new sounds and move others, and embarked on a further two days testing of the app. The remaining text is a collection of observations and analyses of testing alone “in the field.”
As characters, genres and meanings merge into one another, the effect is disarming – a mélange of cultural fragments linked to real places, and some that no longer exist. As with much smartphone activity, the user becomes a nexus of “socio-spatio-technical relations” (Leszczynski, 2015: 732), a point of convergence for geolocation satellite transmissions (GPS), developer software and mass-produced hardware (smartphone), within real world spaces. However, with BATTERCTRAX, the user re-establishes links between narrative audio fragments and real-world places, and is generating meanings from those connections. The collage effect is reinforced by the different genres of film audio, from detective and horror scenarios, to upbeat British newsreels and kitchen sink dramas. However, it became clear that the effectiveness of the experience was sometimes undermined by large gaps of silence between clusters of geofences. To preserve a sense of continuity, it was therefore necessary to create more beds of sound to bridge the short distances of silence. The content of the sound beds took two forms: ambient, subtly shifting tones which continue the audio experience without distracting the user; and field recordings of the park taken at various point where the users might be walking. In fact, the latter was necessary to preserve the everyday sounds of the park, which would otherwise be eliminated by the noise cancellation earbuds and headphones predominantly worn by smartphone users. With these additional sounds in place, it was possible to walk seamlessly between clusters of film audio. However, the combined effect of audio from films, ambient sound beds and field recording of the park continued to create a series of unusual and disquieting effects which demand further examination. The following sections will attempt to interpret these and other “uncanny” side-effects of geolocative technology, as fragments of film culture and real places converge and coalesce within the user.
Media Memories of Place
Increasingly, individuals gather and store digital traces of their daily activities, as photos, training routes, recordings or purchase data. Indeed, according to Frith and Kalin, “[m]any individuals are using mobile media to mobilize place and memory together to create new forms of digital network memory from which they may begin to remember their pasts and to write their histories” (2015: 5). With this in mind, it seems clear that just as books and analogue recording devices were used to record “memories,” now digital platforms are being used to record, store and retrieve the traces of our everyday activities. However, at the risk of stating the obvious, these digital accounts of daily events (exteriorized as packets of digital media), are not in any true sense “memories” (which, as we know, are mental recollections generated by cognitive functions of the brain). And yet these digital processes mirror the functioning of the brain (record, store, retrieve) on a vastly larger scale, incorporating huge networks of individuals. Frith and Kalin’s study of “place-based memory” refers to personal digital archiving within the context of social media, automatic email and cloud-based data storage, and how such data can be recalled and reflexively “reconstruct [the user’s] ongoing present sense of identity” (44). By contrast, BATTERCTRAX was an attempt to mobilise cultural artefacts created by studios and film crews before the digital age, but which were subsequently digitised and stored by the Cinematic Geographies project. With BATTERCTRAX, the ambulant user experiences a park augmented by fragments of fictional conversations and cultural echoes from past films and TV shows; media retrieved from obscurity in public archives and activated by their proximity to a geographic coordinate, the place of their origin. These subtle interjections of sound into the spaces of everyday life, detached as they are from their moving image source, create an uncanny and unnerving presence according to some test users, effects which belie the systematic methodology of the Cinematic Geographies project and its attempt to quantify moving images by geographical coordinates in a database. This discrepancy seems to suggest that database cinema could have unpredictable outcomes, as with recent examples of geolocative mobile software leading users into unexpected places and sometimes compromising their safety.[5] The logic by which BATTERCTRAX summons and distributes audio packets across space is determined not by chance or aesthetics, but by the locational imperatives of the database: the sounds are simply placed where they were originally recorded. The effect of this logical distribution, in fact, resembles the technique of collage, where stylistically diverse components are brought together in a single assemblage. Collage, however, carries the subversive potential to slice and reorder meaning: as Manovich (2001) says of Vertov’s “database imagination,” it is an attempt to “discover the hidden order of the world…a struggle to reveal (social) structure among the multitude of observed phenomena” (207, 208).
Moving around the park, test users described feelings of recognition and uncertainty as they attempt to identify characters and sounds within the collage of audio fragments, and the eras in which they were set. However, some test users, including the author, recognised that some fictionalised dialogue and documentary sounds are uncannily and convincingly present in “real space,” as if they were generated by events happening around them, in the present. To offer a hypothesis for this unusual effect, much of the historic audio shares the same spatio-acoustic presence of the actual places. This means that that more often than not, the audio was originally recorded live in situ, with the actors performing in roughly the same location as the app user, only years earlier. The actors have long gone but their performances somehow remain, perhaps only as memories of place.
While Frith and Kalin’s (2015) notion of “place-based memory” reflects on the personal value of user-generated media and data, BATTERCTRAX draws from a database of pre-made historical media artefacts triggered and released by the user’s mobility. The fact that these fictional interjections resemble memories is disquieting: there is an uncanny sense that the user is experiencing memories, but not their own. The place itself is triggering audio emissions, representations of events from its own past. This conceit is echoed by Ravetto-Biagioli, who suggests that,
Technology is now standing in for memory, consciousness and experience, feeding it back to us in the form of an uncanny effect. Technologies exteriorize our memories (in the form of a database or archive), offering us ‘tertiary memory’ (a secondary or non-lived memory) and image consciousness (a consciousness that does not derive from embodied perception) (Ravetto-Biagioli, 2016: 14).
However, while users felt able to distinguish between experiences augmented by technology and those generated by their own consciousness, the uncanny effects suggest an uncertainty which strips “the body of its positioning as a locus of sensation, perception and recollection, which in turn strips thought of its presuppositions” (Ravetto-Biagioli, 2016: 20). The following section will examine these and other destabilising effects in detail, including those which continue even when the user stops using the app.
Uncanny Media and Residual Effects
BATERCTRAX began as a project to explore new avenues of media distribution by attempting to attach audio from films, TV show and documentaries to the locations where they were originally recorded. In achieving this aim, albeit within the parameters of an experimental pilot project, the researchers became increasingly aware of certain unexpected effects generated during the user experience. This section will attempt to distinguish between these effects and interpret them within the context of other locative sound art projects, and research within audio geographies and sound studies.
As described in the previous section, while wearing earphones and using BATTERCTRAX around Battersea Park, users sometimes found it difficult to distinguish between sounds triggered by geofences and those generated by real events taking place around the user. This is particularly true of field recordings made in situ by the author and Butterworth to bridge the gap between archival media files. Test users claim that they sometimes had to remove their earbuds in order to tell whether a conversation or voice was real or not, and in some cases users leapt out of the way to avoid phantom bicycles and cars, when in fact they were hearing “placed sounds.” This effect has also been observed by Michael Gallagher during the testing of the sound work, Kilmahew Audio Drift No. 1, in which he describes a sense of “doubling”, in which:
we may hear, for example, what sounds like a woodland in springtime through the headphones, but other cues – both sounds from outside the headphones and other sensory perceptions – tell us that we are in a woodland in winter. This doubling creates an excess of meaning, less a representation of springtime than an uncanny sense of the two seasons merging (Gallagher, 2014: 479).
Kittler (1997) reminds us that the notion of “the double” is a recurring motif in romantic literature and film, and is a key characteristic of the uncanny. However, he also suggests that it emerges from the medium itself, as a writing-effect: “[T]he double turns up at the writing desk…it can hardly be more clearly stated that the Double of classical Romanticism essentially emerges from books” (88-89). By extension, the medium of film “is in its essence a world of doubles” (Royle, 2003: 78). If we accept that each new medium generated its own doubling motifs, its own uncanny effects, as unexpected vents for the unconscious, then it follows that audio locative technologies may similarly duplicate or re-present versions of the world, stimulating users in surprising and unpredictable ways. Gallagher is one of the few researchers to observe and engage with the psychological and potentially traumatic effect of locative sound art, where the possibilities for “dream-like, hallucinatory and uncanny affects” are very real and potentially dangerous, leading users into uncertain territories (2014: 480). Like Kilmahew Audio Drift No. 1, BATTERCTRAX has the semblance of certain other cultural projects gathered under the rubric of hauntology which repurposes media from the past or imitates the stylistic tropes of 1960s and ’70s culture to interrogate the apparent failures of the present. The collage of audio activated by BATTERCTRAX may well induce unexpected effects, including a nostalgia for “lost futures” (Fisher, 2014: 41), as sounds from the 1951 Festival of Britain collide with sci-fi films such as Mutations and The Day the Earth Caught fire, but such responses are entirely subjective. Indeed, Gallagher remarks that perhaps the value of such technology lies in its ability to resist such forms of evaluation, by “loosening the grip of fixed representations, and counteracting the stultifying politics of detached objectivity that still surrounds much academic research” (2014: 480). In referencing doubling, the uncanny and hauntology, there is an attempt here to understand certain unusual effects through the lens of accepted cultural tropes. However, it would seem almost impossible to qualify the spectrum of uncanny, spectral and affective states generated by such new technologies without recourse to more rigorous testing and evaluation in the field. Observations of doubling and dissonance in BATTERCTRAX, between real and unreal events, support Rieser’s (2005) observation that the “active participant [of locative media and spatial narratives] appears in a liminal state between worlds, whose attention moves between absorption in diegisis, the intrusive ‘real’ and the ambient physicality of the environment” (2).
More curiously, when the users removed their earbuds and ended the BATTERCTRAX trial, some retained a residual sense of still being in the experience, where they continued to question what sounds were “real” and what were not. The experience had finished and yet there was a heightened sense of “unreality,” an altered state or a side effect in which real sounds and conversations seemed to be part of a synthetic or augmented experience. This curious and compelling effect was also observed by the author on multiple occasions during solo testing in Battersea Park. Snatches of everyday conversations heard in the bustle of the park, cries of circling gulls and distant traffic, all seemed to be part of a structured collage of sounds, juxtaposed in a premeditated way – a continuation of a curated experience that had since ended. Could such technology indirectly act as a form of psychoactive stimulation, altering the perception and cognitive state of the user?
Without further trials or tests, it is very difficult to assess the degree to which geolocative sonic experiences can affect the user’s ontological state or psychological disposition. However, manufacturers of virtual reality headsets such as Oculus VR, acknowledge that using their technology can produce a variety of alarming “post-use symptoms”, including nausea, seizures, loss of awareness and disorientation.[6] While the numerous negative effects listed in the small print of the Oculus Rift Health and Safety Manual may be litigious base-covering, they do suggest that VR and augmented reality could stimulate a myriad of subtle physiological conditions which may continue after use. The innumerable studies of so-called “cybersickness” and Virtual Reality-Induced Symptoms and Effects (VRISE) (see for example Rebenitsch and Owen, 2016: Lavelle, 2016), and the apparent benefits of such technologies in the clinical and therapeutic disciplines (Aiken and Berry, 2015; Riva and Raspelli et al, 2010), do not address geolocative media technologies which tend to be less immersive. Perhaps for this reason, there seems to be far less research on the physiological and psychoactive impact of location-responsive audio technologies.
The geolocative audio projects described above, including BATTTERCTRAX, are not visually and physically immersive in the manner of VR, but they nevertheless influence and affect the individual user in subtle but unexpected ways. Only further testing of such technologies will determine if these post-use, residual effects are significant and if they do indeed compromise the user’s ability to distinguish between real and synthetic phenomena.
Conclusion
BATTERCTRAX was a small trial of locative media technology, where the smartphone user could walk amongst clusters of geofences in Battersea Park, activating audio files and hearing them through earbuds or headphones. In this way, the app had the ambition of distributing or returning audio from films, TV shows and documentaries to the places where they were originally recorded. It was an attempt to explore technologies that could establish links between people and the communities, and the cinematic heritage of their communities, and to understand the impact of place and space on film. Fictionalised dialogue, ambient tracks and field recordings blended into a collage of unrelated sounds (united only by their association with the location), which the user could activate by freely moving around the park. In this way, the project also acted as a way of activating a cultural memory of place, as a site of historic cultural production and dynamic social forces. However, the act of re-establishing the link between film and place led to unexpected and uncanny effects. At times, members of the test user group could not distinguish between real sounds and those that were generated by the app. Furthermore, for a few minutes after using the app, some users claimed to be experiencing a continued sense of unreality in which some real sounds presented themselves as auditory hallucinations. While these kinds of unexpected residual effects are being observed in others forms of commercial media applications, the psychological implications are little understood. We may choose to interpret such dissonance or doubling effects through the lens of the uncanny, but we have yet to understand or anticipate how locative technologies will destabilise our sense of being in the world. We increasingly inhabit spaces augmented by location-responsive media, but we do so without acknowledging the ontological ambiguities that such spaces engender. Media that subtly appeals to our specific place in the world could undermine the integrity of that place, offering hallucinations, altered states and uncertainty in place of reality.
The invisible parameters of location-responsive media (and the unpredictable effects they may trigger) are still somewhere beyond our critical understanding. Without further independent empirical studies, we may become victims of ontological ambiguities and spatial paradoxes, where memories and sensory effects are instilled in us by places and products, and our sense of individual agency slowly slips away from us.
If the Cinematic Geographies of Battersea project developed new methods for mobilising films and moving images for future research, then BATTERCTRAX inadvertently posed a number of disquieting questions about our future interface with space and technology.
References
Adams, B., Dorai, C., and Venkatesh, S. (2000) “Towards automatic extraction of expressive elements from motion pictures: Tempo,” IEEE International Conference on Multimedia and Expo, Vol. II, pp. 641-645.
All the Right Noises, 1971 [film]. Directed by Gerry O’Hara. UK: Trigon Films.
Aiken, M. P., and Berry, M. J. (2015) “Posttraumatic stress disorder: possibilities for olfaction and virtual reality exposure therapy,” Virtual Reality, Vol. 19, No. 2, pp. 95-109.
Alifragkis, S., and Papakonstantinou, G. (2017) “Urban Cinematic Palimpsests,” in F. Penz and R. Koeck, (Eds.), Cinematic Urban Geographies, New York: Palgrave Macmillan.
Audio Mobilities, a special issue of Wi: The Journal of Mobile Media, (2015): Vol. 9, No. 2.
Behrendt, F. (2012) “The Sound of Locative Media,” in Convergence, Vol. 18, No. 233.
Behrendt, F. (2014) “Creative Sonification of Mobility and Sonic Interaction with Urban Space: An Ethnographic Case Study of a GPS Sound Walk,” in S. Gopinath and J. Stanyek, (Eds.), The Oxford Handbook of Mobile Music Studies. Volume 2, pp. 190-211.
Behrendt, F. (2015) “Locative Media as Sonic Interaction Design: Walking through Placed Sounds,” Wi: The Journal of Mobile Media, Vol. 9, No. 2.
Berardi, F. (2011) After the Future, AK Press.
Bijsterveld, K. (2013) Soundsapes of Urban Past. Bielefeld: Transcript Verlag.
Brannigan, 1975 [film]. Directed by Douglas Hickox. UK, USA: Wellborne, Levy-Gardner-Laven.
Burrough, W.S., Gysin, B. (1978) The Third Mind, New York: Viking Press.
Caquard, S., Taylor, D. (2009) “What is Cinematic Cartography?” in The Cartographic Journal, Vol. 46, No.1, pp. 5-8.
Cinematic Geographies of Battersea project website: http://cinematicbattersea.blogspot.co.uk
Fisher, M. (2004) k-punk website: http://k-punk.abstractdynamics.org/archives/001324.html
Fisher, M. (2014) Ghosts of My Life: Writings on Depression, Hauntology and Lost Futures, Zero Books.
Frith, J., and Kalin, J. (2016) “Here I used to Be: Mobile media and Practices of Placed-Based Digital Memory,” Space and Culture, Vol. 19, No.1, pp. 43-55.
Gallagher, M. (2015). “Sounding ruins: Reflections on the production of an ‘audio drift.’” Cultural Geographies, Vol. 22, No. 3.
Hallam, J., and Roberts, L. (2011) “Mapping, memory and the city: Archives, databases and film historiography.” European Journal of Cultural Studies, Vol. 14, No. 3, pp. 355-372.
Hemment, D. (2006) “Locative Arts,” Leonardo, Vol. 39, No. 4, pp. 348-355.
Huhtamo, E., and Parikka, J. (2011) Media Archaeology, University of California Press.
Keiller, P. (2007) The City of the Future, multimedia installation, BFI Southbank.
Keiller, P. (2014) The View from the Train: Cities and Other Landscapes, London: Verso.
Kittler, F. (1997) Romanticism – Psychoanalysis – Film: A History of the Double, in J. Johnson, ed., Literature, Media, Information Systems. Amsterdam: G+B Arts.
Koeck, R. (2012) Cine-Scapes: Cinematic Spaces in Architecture and Cities, Oxford: Routledge.
Koeck, R., and Roberts, L. (2011) The City and the Moving Image: Urban Projections, Basingstoke: Palgrave Macmillan.
Labelle, B. (2010) Acoustic Territories: Sound Culture and Everyday Life. New York: Continuum.
Lavalle, S. M. (2016). Evaluating VR Systems and Experiences, in Virtual Reality. http://vr.cs.uiuc.edu/
Leszczynski, A. (2015) Spatial media/tion, in Progress in Human Geography, Vol. 39, No. 6, pp. 729-751.
Manovich, L. (2001) The Language of New Media, Cambridge, MS: MIT Press
Manovich, L. (1999) “Database as a Symbolic Form,” Millennium Film Journal, No. 34.
Manovic, L. (2013) Visualizing Vertov. Russian Journal of Communication, 5(1), pp. 44-44.
Marker, C. (1990) Zapping Zone, Proposals for an imaginary Television, multimedia installation at Centre George Pompidou.
Misek, R. (2012) Mapping Rohmer: Cinematic cartography in post-war Paris, in L. Roberts (Ed.), Mapping Cultures: Place, Practice, Performance, New York: Palgrave Macmillan.
Oculus Rift, Health and Safety Manual, Oculus VR: https://static.oculus.com/documents/310-30023-01_Rift_HealthSafety_English.pdf
Parikka, J. (2016) What is Media Archaeology (3rd ed.), Cambridge: Polity.
Penz, F., Koeck, R. (eds). (2017) Cinematic Urban Geographies, New Uork: Palgrave Macmillan.
Plenty, 1985 [film]. Directed by Fred Schepisi. UK, USA: Pressman Productions, RKO Pictures.
Raban, J. (1974) Soft City, Glasgow: Fontana.
Ravetto-Biagioli, K. (2016) “The Digital Uncanny and Ghost Effects,” Screen, UC Davis, Vol. 57, No. 1.
Rebenitsch, L., & Owen, C. (2016) “Review on Cybersickness in Applications and Visual Displays,” Virtual Reality, Vol. 20, No. 2, pp. 101-125.
Rieser, M. (2005) “Locative Media and Spatial Narrative,” REFRESH: BANFF 2005. First International Conference on the Media Arts, Sciences and Technologies, Banff Center Sept 29-Oct 4.
Riva, G., Raspelli, S., Algeri, D., Pallavicini, F., Gorini, A., Wiederhold, B. K., and Gaggioli, A. (2010). “Interreality in Practice: Bridging Virtual and Real Worlds in the Treatment of Posttraumatic Stress Disorders,” Cyberpsychology, Behavior, and Social Networking, Vol. 13, No. 1, pp. 55-65.
Roberts, L. (2012) “Cinematic cartography: Projecting place through film,” in L. Roberts (Ed.), Mapping Cultures: Place, Practice, Performance, New York: Palgrave Macmillan, pp. 68-84.
Royle, N. (2003) The Uncanny, Manchester: Manchester University Press.
Salt, B. (1976) “Film Style and Technology in the Thirties,” Film Quarterly, Vol. 30, No.1, pp. 19-32.
Salt, B. (2016) “The exact remake: a statistical style analysis of six Hollywood films,” New Review of Film and Television Studies, Vol. 14, No. 4, pp. 467-486.
Speed, C., Thomas, M., and Barker, C. (2017) Ghost Cinema App: Temporal Ubiquity and the Condition of Being in Everytime. In F. Penz and R. Koeck, eds., Cinematic Urban Geographies. New York: Palgrave Macmillan.
Sterne, J. (2012) The Sound Studies Reader. Oxford: Routledge.
The Day the Earth Caught Fire,1961[film]. Directed by Val Guest. UK: Pax Films.
The File of the Golden Goose, 1969 [film]. Directed by Sam Wanamaker. UK, USA: UK, USA: Caralan Productions, Edward Small Productions.
The Mutations, 1974 [film]. Directed By Jack Cardiff. UK, USA: Cyclone, Getty Picture Corp.
Trigg, D. (2017) Uncanny Bodies and Altered States, in A. Popa and F. Flueraş, eds., Black Hyperbox, Bucharest: Punch, pp. 308-323.
Tsivian, Y. (2005) Cinemetric, http://www.cinemetrics.lv/index.php.
Zielinski, S. (2008) Deep Time of the Media: Toward an Archaeology of Hearing and Seeing by Technical Means. Cambridge: MIT Press.
Notes
[1] A geofence is a virtual boundary which defines a real-world geographical area. Such boundaries can either be radial – emanating from a single geographic coordinate, or polygonal – defined by multiple geographical coordinates. The limits of such boundaries can be programmed into an app on a mobile device with GPS capabilities, which then alerts the user when they enter the pre-defined zone. Geofencing is now commonplace in many location-aware mobile apps, alerting users to the presence of relevant services or activities nearby.
[2] Cinematic Geographies of Battersea: Urban interface and site specific spatial knowledge, was an 18 month-long (including 6 month follow-on funding) pilot project funded by the British Arts and Humanities Research Council (AHRC). It was led by Prof François Penz (Cambridge University, Department of Architecture and Moving Image), with co-investigators Prof Richard Koeck (Liverpool University, School of Architecture), Prof Chris Speed (Edinburgh University, Design Infomatics), Aileen Reid (English Heritage, Survey of London), with Research Associates Chris Barker (Edinburgh University), Dr Matthew Flintham (Liverpool University, School of Architecture), and Maureen Thomas (Cambridge University, Department of Architecture). AHRC project reference: AH/I022252/1. See Research Council UK website for more information: http://gtr.rcuk.ac.uk/projects?ref=AH%2FI022252%2F1
[3] As in the emerging field of cinemetrics, which measures and categorizes narrative or aesthetic elements of films into units of time, a quantitative analysis of film might include a statistical breakdown of narrative, aesthetic, chromatic or geographic elements of moving image media. Such analyses might require the diligent and time consuming measurement of shot lengths, (for example, Salt, 1976; 2016), and others propose computational or automated methods for extracting data from films (for example, Adams, Dorai and Venkatesh, 2000; Manovich, 2013; Tsivian, 2005).
[4] See also Drew Hemment’s essay Locative Arts for a comprehensive summary of ‘early’ locative and sonic arts projects:
http://locative.articule.net/wp-content/uploads/2013/06/Hemment_Locativearts.pdf
[5] There are innumerable examples of SATNAV software leading drivers into impassable and dangerous locations, and many examples of Pokémon Go compromising public/user safety:
[6] ‘Just as with the symptoms people can experience after they disembark a cruise ship, symptoms of virtual reality exposure can persist and become more apparent hours after use. These post-use symptoms can include the symptoms above [seizures; loss of awareness; eye strain; eye or muscle twitching; involuntary movements; altered, blurred, or double vision or other visual abnormalities; dizziness; disorientation; impaired balance; impaired hand-eye coordination; excessive sweating; increased salivation; nausea; lightheadedness; discomfort or pain in the head or eyes; drowsiness; fatigue; or any symptoms similar to motion sickness], as well as excessive drowsiness and decreased ability to multi-task. These symptoms may put you at an increased risk of injury when engaging in normal activities in the real world (Oculus Rift, Health and Safety Warnings booklet).
Matthew Flintham is an Early Career Research Fellow at Kingston University, London, specialising in cinematic representation and new media, and issues of militarisation, security and surveillance. He holds a BA (Hons) in Fine Art from Central Saint Martins, an MA in Cultural Studies from the London Consortium, and a PhD in Visual Communications from the Royal College of Art. His work intersects academic and arts practices, exploring speculative relationships between film, architecture, power and place. His most recent work can be found in the edited volume, In the Ruins of the Cold War Bunker: Affect, Materiality and Meaning Making from Rowman & Littlefield, and in Critical Topographies, forthcoming from McGill-Queens University Press.
Email: matthewflintham@hotmail.com
The ‘version of record’ of this article is available as part of the special issue on Geospatial Memory (Media Theory 2/1) here: http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/36
The full issue on Geospatial Memory (MT 2/1), edited by Joshua Synenko, is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/issue/view/2
Leave a Reply