Abstract

Affordances provide a useful frame for understanding how users interact with devices, but applications of the term to digital devices must take into account that human agents operate within a material environment that is distinct from the digital environment in which these devices operate. A more restrictive approach to affordances would focus on the agency of digital devices distinct from the agency of human users. Location-aware mobile devices provide a particularly compelling example of the complex interplay of agents and agencies, and how “augmented affordances” give rise to a lived space of information for human users.

Keywords

actor-network theory, affordances, digital agents, mobile computing, sense of place, smartphones

 

The Affordances of Place: Digital Agency and the Lived Spaces of Information

MARK NUNES

Appalachian State University, USA

 

The near-ubiquity of ubiquitous computing and mobile devices foregrounds the degree to which our everyday lives now involve data. This foregrounding shifts our relationship to our lived environment by embedding experience within a framework of digital interaction – not only making the world “clickable,” as Wise (2014) suggests, but likewise making information space inhabitable. We no longer need to imagine our computers as vehicles designed to transport us to and through cyberspace; today, cyberspace is all around us – an information overlay mapped onto our everyday experience of place. As William Gibson (2011) himself has noted, reflecting on the term he coined in 1982: “cyberspace is everywhere now, having everted and colonized the world. It starts to sound kind of ridiculous to speak of cyberspace as being somewhere else.” As Jones (2014) notes, “the timeline of eversion” may well begin with the first appearance of smartphones in 2006 and 2007, which took computing off of our desktops and placed it into our hands (22).^[1] Once these devices become location-aware, the process of eversion is complete; cyberspace is indeed everywhere.

The potential for location-aware mobile devices to bring the computational and networking powers of computers off the desktop and on the move simultaneously alters our sense of place and our experience of the place of information in everyday life. As such, these devices provide a particularly literal instance of understanding our environmental relation to both information and information technologies, to the extent that these devices locate us in a space that is simultaneously embodied and informatic. From McLuhan (1994) onward, an ecological approach to media has asserted that the medium mattered – and more so, that its materiality mattered. This ecological approach to media acknowledges that “the overall human environment includes and incorporates technological extensions, and these are never merely add-ons. They alter our sensibilities and capacities, our notions of self and other, our notions of privacy and propriety, and our orientations in space and time” (Anton, 2016: 131). It is within this ecological context that an examination of digital affordances can help us better understand the impact of mobile devices on our lived relation with information, as well as our embodied experience of space and place in an age of mobile computing.

J.J. Gibson (1979) developed the concept of affordance to help explain an organism’s embeddedness within its environment, arguing that what an animal perceives depends upon a kind of coupling between organism and environment based upon that particular animal’s potential for action within that particular environment. This ecological approach to perception offers an understanding of how agents make use of their environments, and the sorts of interactions that give rise to ways of not only using the environment, but also embodying space through use. As such, affordances are “equally a fact of the environment and a fact of behavior” (Gibson, 1979: 129). Affordances are invariant to the extent that it is not the will or desire of the organism that brings forth the affordance, but rather this structural coupling between an organism’s potential for action and its environment (Gibson, 1979: 138-139). These invariants communicate a specific set of relations between actor and world such that, in Gibson’s (1979) words, “to perceive the world is to coperceive oneself” (141).

Gibson’s concept of affordances has been applied in a wide range of contexts; some recent reviews of the literature have attempted to reframe the concept within a more narrow band of applications, for example by modifying exactly which affordances these applications attempt to address (Parchoma, 2014; Osiurak, Rosetti and Badets, 2017; Bucher and Helmond, 2017). In their comprehensive discussion of social media affordances, Bucher and Helmond note: “While all conceptualizations of affordances take Gibson’s original framing of the term as a starting point, they differ in terms of where and when they see affordances materializing (i.e. features, artefacts, social structures) and what affordances are supposed to activate or limit (i.e. particular communicative practices, sociality, publics, perception)” (240). In thinking through human interactions with location-aware devices, which in turn interact with and determine a user’s sense of place, we might want to begin by following Bucher and Helmond in their distinction between “high-level affordances” which “locat[e] affordances in the relation between a body and its environment,” and “low-level affordances” that “locat[e] affordances in the technical features of the user interface” (240).

According to this perspective, Norman’s (2013) discussion of affordances would have to be categorized as “low-level,” and, at least from the standpoint of object design, it is certainly accurate to observe that Norman’s definition offers an adequate explanatory framework for describing users’ bodily interactions with location-aware mobile devices. However, it is worth noting the degree to which Norman maintains an ecological understanding of human interaction with objects, insisting on a “relational definition” of affordances as “jointly determined by the qualities of the object and the abilities of the agent that is interacting” (11). For Norman, an affordance is “a relationship between the properties of an object and the capabilities of the agent that determine just how the object could possibly be used” (2013: 11). This relational definition sets up a kind of “discoverable,” communicative system between object and agent, a “mapping” that first must be realized before it can be actualized (20-23). And while Bucher and Helmond are correct in noting Norman’s greater attention to user interfaces, Norman also introduces an important distinction between affordances, which “determine what actions are possible,” and signifiers,^[2] which “communicate where the action should take place” (14). A lever affords pushing. Visual cues in the form of arrows signify that the agent can push the lever in two directions. Mapping discovers and communicates a relation between the spatial orientation of those two directions and the motion of the object it controls – for example, a projection screen in a lecture hall.^[3] Or, to apply this idea to a familiar interaction on a location-aware mobile device: the screen on my smartphone affords touch, and each app icon serves as a signifier in both Norman’s sense and in a semiotic sense, identifying each application while at the same time indicating where on the screen to touch. When I tap on the Google Maps icon, for example, the app ‘zooms’ out from its particular location on my home screen to fill the entire screen, providing me with a map of my current location – but also providing me with a conceptual mapping of my interaction with the device: touching = opening.

But as we move from describing my interactions with the object in my hand to my experience of the information that this object provides, use of the term ‘affordance’ becomes more complicated, to the extent that embodied interaction with these devices occurs at a level that is fundamentally distinct from the level at which algorithmic and the computational actions occur. As Bucher and Helmond suggest, our interactions with digital devices, software platforms, and networks of information require a “multi-layered approach” that can adequately address our material and social interactions with technology, and the relation between the two (2017: 242). As Bucher and Helmond note: “the term ‘technology affordances’ establishes material qualities of technology as (partly) constitutive of sociality and communicative actions” (237). Along similar lines, Hutchby (2001) highlights the importance of focusing on “the material substratum which underpins the very possibility of different courses of action in relation to an artefact” (450). How, though, do we account for a material substratum that is, in fact, digital – and with which human agents do not have any direct interaction?

While mobile devices have properties (as objects) that afford human users a range of potential actions, they likewise possess a range of digital capabilities (as agents) that act upon a digital environment. In contrast to user interaction, which remains embedded within a material environment, the device itself acts upon a digital environment through a distinct set of affordances, including: the execution of protocols, database queries and retrieval, data processing, input/output functions, and network transmission. This digital substratum impacts the different ways in which a user can engage a device, yet the data-device coupling that gives rise to algorithmic and computational action does not afford human users the capacity to act directly upon a digital environment. As the device acts upon and within a digital environment, however, it materializes and actualizes opportunities for user (inter)action that do indeed allow users to coperceive themselves within an inhabitable space of information. The material environment and what it affords does not change, but my conceptual mapping of what the environment affords changes by way of this information overlay, as does my sense of how I might actualize a potential set of actions. If we understand affordances, as Hutchby (2001) defines the term, as “functional and relational aspects which frame, while not determining, the possibilities of agentic action in relation to an object” (444), we might consider the device itself as the agent that engages the digital, which in turn materializes on its screen a new set of functional and relational framings for human agents. The human-digital interface, then, would map a doubly-mediated coupling of affordances, a relationship that draws forth the possibilities of the digital into a range of possible human uses and actions.

I am thus proposing an environment-specific, actor-oriented account of “low-level affordances,” an approach in line with Osiurak, Rosetti and Badets (2017) who, in their study of tool use, attempt to restrict as much as possible the definition of the term to Gibson’s “animal-relative properties of the environment” (406). In contrasting an allocentric (or tool-centric) account of tool-object coupling with an egocentric (or hand-centric) account of hand-tool coupling, they conclude that the term affordance applies only to the potential for action mapped by way of a hand-tool relation. In short: “an affordance exists because of the existence of a potential physical interaction between an animal and the environment. They correspond to action possibilities resulting from animal-relative, biomechanical properties” (Osiurak, Rosetti and Badets, 2017: 409). From this perspective, affordances cannot be contextually determined through object-centered relationships. To use their example: a shoe and a hammer offer similar affordances to a human agent capable of grasping and striking a nail; affordances thus describe a relation between agent and object, even though hammers and nails exist within a contextual and designed relationship that shoes and nails do not share (Osiurak, Rosetti and Badets, 2017: 411). But unlike a hammer, the smartphone is indeed a “tool” with its own agency and potential for action within a digital environment. In this context, then, we may well need to consider a device-centric application of the term digital affordance, one defined not in terms of biomechanical relations, but rather as an algorithmic, protocological relationship to a digital environment that maps an intention-independent framework for potential action, namely: data input, output, storage, retrieval, and manipulation/computation.

In this regard, Latour’s (2005) actor network theory (ANT) provides a useful starting framework for understanding the complexity of agent-specific affordances,^[4] actions, and interactions within these ecological systems, as well as how human and non-human agents map affordances of place. As Parchoma (2014) notes, “within an ANT frame, technological affordances can be examined for their enabling, restricting, and regulatory roles, emerging from the networked effects of temporal relations among physical-social, material-cultural, and human-technical phenomena” (367). This framework would suggest not only a complex relationship between human and non-human agents, but also an ecological understanding of how digital devices serve as an extension of the human, and how the human likewise functions as a material extension for the digital device. This framework also allows for what Best (2009) describes as “relational affordances,” in which device and human alike “inscribe” each other as agents within a set of interactive dispositions, within a complex, interactive system (402). Drawing upon Latour’s concept of devices as “technical actors” (Johnson, 1988), Best describes how extension/embodiment is experienced by the user as a change in potential action, one that “enables [the user] to act on the world – do something to it – rather than just live in it” (405). We may likewise describe location-aware mobile devices as digital agents oriented toward acting upon a “world” of data that is “coupled” with embodied space by way of “place,” much in the same way that human agency gains access to an augmented sense of place by way of the information overlay materialized by these devices. This process is akin to what Latour (writing as Jim Johnson) refers to as a translation of “scripts” between actors and their delegates, human and nonhuman alike (Johnson [Latour], 1988: 308). At this point of double coupling and double articulation, material and digital agents alike embed their actions within this scene of material-informatic translation.

Mobile devices offer this sort of “double coupling” between user and environment most notably, perhaps, through their location-aware properties – a double articulation that materializes and makes visible place-specific information as a frame for human agency. For location-aware mobile devices and their human users, “place” provides a particularly rich nexus for this exchange. The body-centric and data-centric affordances that describe my doubly articulated relation with digital and material environments make clear how affordances operate within a network of agents, and the transformative impact of this complex media ecology on the spaces of everyday life. Our sense of place – and the affordances of place – change because we have more access to information about that place. Physical space, and what it affords, remains unchanged; at the same time, I now find myself embodied within an inhabitable information space. I open Google Maps and a blue dot appears on the screen, pulsing at a rate of about 20 beats per minute – somewhat faster than a resting respiration rate, and about half as fast as a resting heart rate. The pulsing signifies a “live” signal, providing user feedback on the status of the system^[5] as well as their own status as a data point located within the data set that is communicating within a GPS network: a real-time presence, both on the screen and in the world. A shaded area, which pulses at the same rate, signifies directionality – my heading. Moving the device creates a conceptual mapping between my embodied directionality and my orientation on the map. My body provides two different sets of interactions with the device. By touching the screen, I can alter my relation to the image, but my touch does not alter the location of the blue dot indicating my position. Only by altering my body’s location and orientation in lived space can I change the location of the blue dot on the screen. In effect, I am bi-located: co-perceiving myself simultaneously in the information space on the screen and in the embodied space of the material world.

Place, then, offers a mapping of potential action onto environment for two distinct agents – one human and one digital. Place-as-information affords algorithmic processing for the location-aware device-as-agent, but in doing so, the device creates a material environment for human agency within a lived space of information. When using a map application such as Google Maps, the map positions me in a set of relations that are both informatic and materially embodied. I see myself in a spatial relation to the world around me, and I understand the world around me by way of this information overlay – the names of streets, the location of rivers, public monuments and private businesses, etc. Likewise, my wayfinding is dictated by my body’s interaction with the world and the corresponding translation of these actions as data input for my device, which is then acted upon by the device to produce a representation of position on a dynamically changing map upon the screen of my smartphone. My body couples with both device and physical space, creating a complex mapping that materializes affordances of place that did not exist for me in the physical world, but which are now articulated through the actions of a digital agent. The device alters our wayfinding – and therefore our sense of place – through this actualization of information, producing not an “augmented reality,” but rather a set of augmented affordances by way of this double coupling of agents and agencies.

Affordances as relational couplings, then, mark points of articulation in a material-digital assemblage. The potential for action thus points in two directions: the algorithmic, acting upon data that is derived from material, embodied action; and the embodied, acting upon a material environment that is, in turn, informed by algorithmic processing. Following Rutz (2016), we might think of this point of double coupling as a site of “mutual incursion” – a Deleuzean assemblage marked by “exchange and assimilation processes between human and machine” (74). “Algorithmic agency,” then, would operate in two (or more) directions – as the artist (as in Rutz’s example) engages in “trajectories” of data processes derived by algorithmic action at the same time that the algorithm engages in material processes derived by the embodied action of the artist and/or audience (Rutz, 2016: 76-80). This emphasis on the interrelationality of artist/experimenter and algorithm in machinic assemblages calls attention to how “both sides engage in boundary operations that are best described as reconfiguration, operations where many elements and relations, representations and concepts remain intact but a few critically change” (Rutz, 2016: 82).

In this sense, each agent (material or digital) operates at a point of incursion in a material-digital assemblage. Each acts upon an environment within its own relational structure of potential actions, but each also marks “boundary operations” in the form of a doubly articulated agency. This same analytical framework would hold for both “low-level” as well as “high-level” understandings of affordances, which must still come to terms with this boundary event between two or more agents engaged in relational couplings across material and digital environments. Van Dijck (2013) in part addresses this issue through a discussion of “user agency,” which maps a complex relation between algorithms, protocols, interfaces, and human interaction. This complexity plays out, he notes, in the blurred boundary, for example, between “human connectedness” and “automated connectivity” of how user agency engages “the social” in social media (11-12). In a similar move, Bucher (2012) discusses Facebook’s “algorithmic friendship” as “a relation between multiple actors, not only human individuals, but also nonhuman software actors” (480). Combining Deleuze and Guattari’s (1987) concept of assemblage with Latour’s (2005) actor network theory, Bucher argues that “friendship” on Facebook is expressed as an assemblage of both human and non-human actors, articulated in moments such as when the platform assists the end user in importing contact lists into Facebook (482). Likewise, since the algorithms that rank News Feed content determine which friend relations are, in effect, rewarded with attention and hence reinforced, these digital acts function in a way that determines future friendship interactions (484). Bucher concludes: “Friendships online thus need to be understood as a socio-technical hybrid, a gathering of heterogeneous elements that include both humans and non-humans … Thinking of friendship as an assemblage – a relational process of composition – offer[s] a way to critically scrutinize how software participates in creating initiating, maintaining, shaping, and ordering the nature of connections between users and their networks” (Bucher, 2012: 489).

We can extend Bucher’s discussion of “programmed sociality” to an account of programmed spatiality, in which the production of space involves both human and nonhuman actors. Place-as-data/data-as-place marks a double articulation, a boundary operation in two directions: the digital action of digital agents is dependent upon materially embodied human agents, who in turn act upon an embodied environment in ways that are equally dependent upon the digital action of digital agents. As Gibson himself notes in a discussion of “places and hiding places,” what an environment affords to an actor is not without the influence of “place learning” and “social perception” (136). In this instance, however, place learning and social perception occur through a “relational process of composition” across two environments (digital and embodied) and a multitude of actors. Embodied actors engage a programmed and material environment articulated not only by the algorithmic agency of a data-driven, human-device coupling, but also by way of incursions in the other direction, as multiple human agents translate their lived experience of place into data that will, in turn, make information space inhabitable.

While Bucher and Helmond (2017) question the degree to which Gibson’s concept of invariant affordances would apply to the “increasingly dynamic and malleable nature of [social media] platforms,” an environment-specific, actor-centric account of affordances and agent-enabled action would, in fact, acknowledge how agent-action coupling plays itself out in invariant ways within material and digital environments for different sets of interdependent agents, distinct from variable features of interface design (248). By way of contrast: Karahanna et al. (2018) distinguish features from affordances by claiming that features “enable” an application’s affordances.^[6] Thus, they claim: “social media offer the affordance to connect with others, enabled by, for example, features such as ‘friending’ on Facebook and ‘following’ on Twitter” (Karahanna et al., 2018: 739). In their attempt to generate a comprehensive taxonomy of social media affordances, Karahanna et al. identify three egocentric affordances – self-presentation, content sharing, and interactivity – along with four allocentric affordances – presence signaling, relationship formation, group management, and browsing others’ content (744-745). In an environment-specific, actor-centric account of affordances, however, “self-presentation” and “content sharing” of a human agent would collapse into a single affordance: the potential to respond to a prompt. Signifiers for these prompts vary by feature – they are indicated by way of icons and photographic images, but also by text marked in bold, underlined and/or alternate color. Features signify, and thereby structure by design the user’s conceptual mapping of affordance to action. Anyone who has double-tapped on a Facebook photo in an attempt to “like” the image has experienced the degree to which features of response vary across applications, but this error is at the same time indicative of the invariant affordance that both Facebook and Instagram present: the potential to respond to output on the screen. As a potential for action, “providing input” offers a set of affordances that is distinct from “response,” one that again maps in feature-specific ways (Instagram’s “+” icon at the bottom of the screen vs. Facebook’s prompt “What’s on your mind?” at the top of the screen). Note that while embodied actions of the user may appear similar in both instances (tapping on a screen), the environment differs considerably: in one instance, I am responding to the device’s output of content, while in the other, I am responding to a request for input from the application itself. In addition to “responding to a prompt” and “providing input,” we might add to this tentative list of affordances “tracking,” or what Schrock (2015) identifies as locatability,^[7] to the extent that the device provides me with a representation of my own location as both input and output, prompting me to various actions and engagements. While not meant to provide an exhaustive list, these examples of an actor-centric approach to affordances suggest a way of mapping invariant potential action within material and digital environments for human and digital agents alike, as well as accounting for complex interactions between agents and agencies.

If features map affordances for human agents acting on and within an inhabitable information space, they likewise provide a mapping for digital agents acting upon the human user as a material extension of their potential for digital action. This insight aligns with Bucher and Helmond’s (2017) insistence on “the multi-directionality of agency and connectivity” and “a socio-technical sensibility towards the distributed agency of humans and nonhumans” (249).Vaast et al. (2017), for example, argue that “connective affordances” emerge through social media use as a result of “mutually dependent yet distinct patterns of feature use among emerging roles … What is afforded to one role depends upon how other roles use the technology” (1199). I would suggest that these same features-mapped connective affordances are likewise providing a mapping for digital agents as they make use of human agents as networked human actors, extending their digital agency into an embodied space, much as mobile devices extend human action into a digital realm. When human agents create data through embodied action, those data sets then provide a basis for algorithmic action by a digital agent. The materialization of this action as output on a smartphone screen provides an articulation from one layer to another, giving rise to a boundary operation between immaterial information and the embodied environment of the human agent about to act upon this information.

This account of environment-specific action and boundary operations aligns well with Don Ihde’s post-phenomenological account of agency (Ihde, 2002; 2009; 2011). Ihde (2011) suggests an “inter-relational ontology” to understand how “human-technology relations” can be understood as a “mutual co-constitutional process” (18). Ihde (2002) notes that “in this interconnection of embodied being and environing world, what happens in the interface is what is important” (87). This “area of interaction or performance” marks a “symbiosis of humans plus their artefacts in actional situations” (Ihde, 2002: 92-93). This “hybrid agency” occurs in multiple human-technology relations, but is particularly notable when humans find their embodied actions coupled with computational environments (Kang, 2011: 111). As Kang notes, “embodiment serves as an interpretive framework through which computable information and its impact on human perception are understood as a continuous, co-constitutive relation rather than as separate, independent processes” (2011: 112). Within such a framework, affordances would operate as relational, bidirectional, ecological expressions of action, in which environment becomes a complex assemblage of potential interactions that are co-constitutive of agents and action.

In distinguishing his post-phenomenological account of human-technology relations from both assemblage theory and actor network theory, Ihde (2002) asserts: “There is, indeed, a limited set of senses by which the nonhumans are actants, at least in the ways in which in interactions with them, humans and situations are transformed and translated” (100). Critical to Ihde’s (2002) account, however, is an understanding that this relation is neither “innocent” nor “neutral” – and more often than not asymmetrical in translation and subsequent transformation (100). As Klinger and Svensson (2018) note, even if we are to understand algorithms as “actions,” those performances, of course, will still reflect “the norms and processes of media production, distribution, and usage, as well as how programmers and users perceive these norms and processes that go into the design/programming process” (4658-4659). They note, “to argue that algorithms have agency on their own, agency that is independent of human activity … occludes the power inscribed in the algorithm as structure” (4667). “Algorithmic agency” bears the marks of a mutual incursion prior to the use of any end-user to the extent that algorithms, programs, and protocols bear the agency of the programmers that script these calculative parameters, but which is likewise distinct from a digital actor’s algorithmic, data-oriented action, which “is less human and more shaped by the big/thick/trace data that they filter, sort, weigh, rank and reassemble into some sort of outcome” (4659). By focusing on “ideals” and “commercial imperatives” as well as “technological affordances,” Klinger and Svennsson call attention to how social and economic forces, programmed into these platforms, shape how these embedded acts translate elements within a digital environment: an incursion that converts “humans actively and intentionally spending time on these communication platforms” into “traces that are subsequently (and algorithmically) mined in order to surveil users with commercial intent, to target advertisements and so forth” (4662-4663).

Clearly, on a corporate-designed and commercially supported mobile application such as Yelp, for example, dominant consumption-driven ideals and commercial imperatives embedded within algorithmic agency exert a powerful influence on how human actors engage with the affordances of place. At the same time, the application also foregrounds how human agency is critical to digital agency, to the extent that users are responsible for creating the database of reviews that the application allows other users to access. In this regard, the relationship between data input and data manipulation as a programmed form of “place learning” is quite complex. When I open the Yelp app, signifiers guide me toward points of interaction, highlighting various features that enable its location-aware search functions. A search bar at the top of the screen affords input; beneath that, the app affords the potential to respond to a prompt by selecting a search category. My engagement with this environment is driven by design, by the prompts of data actors aimed at leading me into a set of protocologically and ideologically delimited relations with both the device in my hand and the lived space in which I find myself embodied and present. If, for example, I respond to the prompt to “filter” a search by category, I can “select” amongst several signifiers, but it is the application that acts upon the database; hence, it is the device, operating on the scripts derived from the application, that is agent in this digital action. At the same time, the affordances of place provide the digital agent with a means of calling out to users to provide data on their current location, perhaps even at the moment they are sitting at a restaurant waiting for their bill. In effect, the digital device is constantly acting upon me through both active prompting and passive tracking; as a digital agent, it is taking advantage of this double coupling to translate my embodied experience within a physical environment to generate data that can be acted upon within a digital environment in a variety of ways, a number of which are captured to serve commercial imperatives well beyond the reach of end users.

As Wendy Hui Kyong Chun (2016b) notes, “capture systems” operate at an interface between data and action: “In a capture system the base unit is an action, or a change of state, rather than an entire person” (60). Here, we see how interfaces draw upon the human actor as environment, and, in this boundary moment, elicit data, but at the same time elicit embodied, habitual activity: “capture systems are all about habitual actions. They seek to create new, more optimal habits; they record habitual actions in order to change them” (Chun, 2016b: 61). Chun (2016b) develops the degree to which habit structures human action: a “productive nonconscious” distinct from any conceptualization of a rational-subject-as-sovereign-actor (7). Critical to this argument is a notion that habit occurs relationally between an individual and an environment that is both social and non-social: habit as habitus (7). For Chun (2016b), habitual action is equivalent to inhabiting a set of practices that position users “within” socio-technological environments. In this regard, her discussion helps to complicate how we understand affordances as relational interfaces between agent and environment to the extent that “habit is ideology in action” (Chun, 2016b: 9). Yelp, for example, captures my action as it prompts me to engage. Regardless of what I am doing with Yelp, I am functioning not so much as a data subject, but rather as a set of relations within a data environment for a digital agent. As Chun (2016a; 2016b) notes, digital agents act in part to transform the actions of individuals into nodes and edges within a set of data correlations: a translation from a singular “me” into a correlational “YOU.”  For better and for worse, “singularity is fundamentally plural” (Chun 2016a: 378). Chun (2016a) argues that “what matters are relations not between things that happen repeatedly or successively to one individual, but rather correlations between actions by different ‘neighbors’ over time and space” (374). If habit is ideology in action, then my engagement within a programmed sociality by responding to a prompt amounts to an Althusserian “hailing” into a set of relations as both embodied actor and constellation of data within material and digital environments (Chun, 2016b: 120-122).

My smartphone maps territories in which I am constantly reminded that others have already been here, be that through restaurant reviews on Yelp, or travel time estimates on Google Maps. Our sense of place is always haunted by data, an overlay that is both here and not here – data that declares others have been here as well. To play on words, mobility offers a kind of digital echo-location. I am located within a territory by the echoes of others who have already come this way. The echoes of others prompt me to add my own voice, or the absence of any voice likewise calls on me to input data corresponding to my location. In effect, then, Yelp does more than map commercial imperatives and the ideals embedded in revealing hidden local gems for tourists and travelers: it reconceptualizes my relation to place and locale, and does so by transforming the place of information within the spaces of everyday life through my dual role as human agent and human extension of a digital agent. In design, the human actor operates as source for captured data that is then articulated within data sets and represented back to users. Human actors engage in the interface with the understanding that their relations are expressed as data acts. The user is not, then, a “data subject” to themselves; rather, their production of node identities and edge relations through intentional and captured acts allows the user to orient toward a becoming-data relationship, much as the social graph allows digital actors to orient toward a becoming-human of data.

Hidden deep within the features of Yelp is “Monocle,” which pushes this embeddedness furthest by using augmented reality (AR) features.^[8] With Monocle activated, my camera now shows me not only what I am seeing through my lens, but also waypoints for nearby restaurants and businesses, geo-located and overlaid on my screen. The AR features of Yelp, while relatively buried, do bring to the fore this moment of double articulation, and the degree to which the device operates as a screen of another sort, one placed “between” the digital and the material. “Looking through” the screen of my phone, with my phone’s camera as a “monocle” onto a digital world that is not directly accessible to me, makes this act of double articulation all the more visible. At the same time, the experience of an information overlay is present for users even without the AR experience, to the extent that augmented affordances of place create for the human user an inhabitable information space in which information materializes for human action through the hybrid agency of a digital actor. Rather than slipping into a facile critique of how screens and devices take us “out of” the here-and-now, I would suggest instead we consider how information becomes both habitual and inhabitable through our engagement with location-aware devices – information that is at the same time engaging the user in human and social interactions within their material environment.^[9]

While Yelp may bury its Monocle feature deep within its menu structure, other apps strongly foreground the ability of the device to allow users to “look through” a now-materialized lens of information presented on our screens. Farman (2012; 2014) details a number of examples of how AR on location-aware devices has been deployed to create narrative overlays for walking tours and cultural heritage sites in cities, yet this is equally true for apps that provide information overlays on natural landscapes, such as stargazing and trailfinding applications. PeakFinder, for example, is a location-aware app that positions users within a topographic map showing the names of mountain peaks, the path of the sun from sunrise to sunset, and the user’s current longitude, latitude, and compass heading. Unlike a mapping app, PeakFinder assumes that you are looking “through” your screen and pointing the mobile device in the direction of a peak, hence it only positions you on one side of the screen. PeakFinder bills itself as an AR application, although again I would suggest that what it provides, more precisely, is augmented affordances by way of the potential actions of a digital agent to materialize information. I now have access to a database of geographic and cartographic information (the location and elevation of peaks), but I can now also see the human written on the landscape (the names of peaks). I experience the affordances of place differently in that I now have information (literally) written over a landscape, a topography in its most literal sense as a writing of place. The information overlay alters not only my orientation toward the landscape; it also provides me with a potential set of interactions that I would not otherwise have at my disposal. For example: the app shows its user the sun’s path mapped out across the sky, as well as its time and location for sunrise and sunset. While Peakfinder defaults to locating me in the here-and-now, a settings feature allows me to alter the date, revealing to me the changing course of the sun – and making visible (for example) the exact two days when the sun will rise directly over a particular peak. As such, how I perceive the world offers an opportunity to co-perceive myself within a new sense of place, with an altered range of current and future action potentials.

Over a decade ago, I addressed the importance of understanding the “cyberspaces of everyday life” as virtual topographies:^[10] performative speech acts that “write” space through material, conceptual, and experiential processes. In a similar move, Ihde (2009) describes what he calls a “material hermeneutics” – what is otherwise non-perceivable is “translated by … instruments into bodily perceivable images,” a “technological transformation of a phenomenon into a readable image” (56). Much as Ihde (2009) suggests, I am arguing that the boundary operation of a human-device coupling offers a “constructed and an intervening process that is deliberate and designed [that] brings into presence previously unknown phenomena … by translating what is detected into images that can be seen and read by embodied observers” (61). Applications such as PeakFinder reveal the degree to which information can quite literally overlay our embodied experience of space and place, reordering our topographies to such an extent that we do not merely access information; we now find ourselves embedded in it. It strikes me that the point of interaction between embodied and digital agents is indeed reciprocal, though not necessarily symmetrical, neutral, or innocent (to use Ihde’s terms). It is not just that PeakFinder provides a data overlay; rather, one’s physical location equally provides a material overlay for the data mappings of a digital agent. My double-orientation toward the device and the place where I find myself serves as a doubled point of articulation in the production of space. Users act upon a transformed material environment through a range of augmented affordances – the product of digital action by digital agents in a digital environment – drawing out data that we ourselves provide actively (through responses to prompts) or passively (through capture of motion or habitual action) as material extension of digital devices through our bodily orientations and dispositions.

Affordance as a concept provides us with a vocabulary for discussing the relational coupling between user and device that is critical to ecological understandings of the role and place of media in everyday life. Likewise, it allows us to acknowledge the gap between two environments, one digital and the other embodied. As a material being, I have no direct coupling with the digital, other than by way of the device. Yet, if we are to accept Ihde’s (2009) post-phenomenological “interrelational ontology,” we must also acknowledge that “the human experiencer is to be found ontologically related to an environment or a world, but the interrelation is such that both are transformed within this relationality” (23). This interrelational ontology implies that “there is a co-constitution of humans and their technologies. Technologies transform our experience of our world and our perceptions and interpretations of our world, and we in turn become transformed in this process” (44). From the perspective of embodied experience, the result is that “our sense of ‘body’ is embodied outward, directionally and referentially, and the technology becomes part of our ordinary experience” of the environment in which we act and interact (42). So, too, is our sense of place mediated through this complex, yet ordinary experience of location-aware mobile devices.

As we attempt to understand this complex ecology of interacting agents and environments, we are led to consider: what, then, serves as the interface, and for whom? Is it the screen on my smartphone, or is it my corporeal presence, smartphone in hand and eyes darting back and forth between the device and the world in which I find myself? This embodied disposition marks a point of material, conceptual, and experiential orientation toward an inhabitable space of information. Cyberspace is indeed everywhere; yet it would perhaps be more accurate to note that our mapping of place now assumes access to information, a digital overlay that is outside of our environmental coupling, but discoverable through our interaction with mobile, digital devices.

 

References

Anton, C. (2016) ‘On the roots of media ecology: A micro-history and philosophical clarification’, Philosophies 1(2): 126-132.

Best, K. (2009) ‘When mobiles go media: Relational affordances and present-to-hand digital devices’, Canadian Journal of Communication 34: 397-414.

Bucher, T. (2012) ‘The friendship assemblage. Investigating programmed sociality on Facebook’, Television & New Media 14(6): 479-493.

Bucher, T. and Helmond, A. (2017) ‘The affordances of social media platforms, in: J. Burgess, A. Marwick & T Poell (Eds.), The SAGE Handbook of Social Media. New York: SAGE Publications. 233-253.

Chen, B. X. (2009) ‘Yelp sneaks augmented reality into iPhone app’, Wired 27 August. Available at: https://www.wired.com/2009/08/yelp-ar/ [Accessed 27 September 2018].

Chun, W. (2016a) ‘Big data as drama.’ ELH 83(2): 363-382.

— (2016b) Updating to Remain the Same. Cambridge, MA: MIT Press.

Deleuze, G. and Guattari, F. (1987) A Thousand Plateaus: Capitalism and Schizophrenia. Minneapolis: University of Minnesota Press.

Didur, J and Fan L. T. (2018) ‘Between landscape and the screen: Locative media, transitive reading, and environmental storytelling’, Media Theory 2(1): 79-107. Available at: http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/37 [Accessed 13 June 2019].

Farman, J. (2012) Mobile Interface Theory. New York: Routledge.

—. Ed. (2014) The Mobile Story: Narrative Practices with Locative Technologies. New York: Routledge.

Gibson, J.J. (1979) The Ecological Approach to Visual Perception. Hillsdale, NJ: Lawrence Erlbaum.

Gibson, W. (2011) ‘William Gibson, the art of fiction no. 211’ (Interview by D. Wallace-Wells), The Paris Review 197. Available at: https://www.theparisreview.org/interviews/6089/william-gibson-the-art-of-fiction-no-211-william-gibson [Accessed 27 September 2018].

Google, LLC (2018). Google Maps. Available at: https://apps.apple.com/app/id585027354 [Accessed 13 June 2019].

Hutchby, I. (2001) ‘Technologies, texts and affordances’, Sociology 35(2): 441-456.

Ihde, D. (2002) Bodies in Technology. Minneapolis: U Minnesota P.

—. 2009. Postphenomenology and Technoscience: The Peking University Lectures. Albany, NY: SUNY Press.

—. 2011. ‘Smart? Amsterdam urinals and autonomic computing,’ in: Mireille Hildebrandt and Antoinette Rouvroy (Eds.), Law, Agency, and Autonomic Computing. New York: Routledge. 12-26.

Jones, S. E. (2014) ‘Eversion’, in: The Emergence of the Digital Humanities. New York, NY: Routledge, pp. 18-38.

Johnson, J. [Latour B.] (1988) ‘Mixing humans and nonhumans together: The sociology of a door-closer’, Social Problems 35(3): 298-310.

Kang, H. (2011) ‘Autonomic computing, genomic data, and human agency: The case for embodiment.’ in: Mireille Hildebrandt and Antoinette Rouvroy (Eds.), Law, Agency, and Autonomic Computing. New York: Routledge. 104-118.

Karahanna, E. et al (2018) ‘The needs-affordances-features perspective for the use of social media’, MIS Quarterly 42(3): 737-756.

Klinger, U. and Svensson, J. (2018) ‘The end of media logics? On algorithms and agency,’ New Media & Society 20(12): 4653-4670.

Latour, B. (2005) Reassembling the Social: An Introduction to Actor-Network Theory. Oxford: Oxford University Press.

McLuhan, M. (1994) Understanding Media. Cambridge, MA: MIT Press.

Miller, J. H. (1995) Topographies. Stanford: Stanford University Press.

Norman, D. (2013) The Design of Everyday Things. Revised and expanded edition. New York: Basic Books.

Nunes, M. (2006) Cyberspaces of Everyday Life. Minneapolis: University of Minnesota Press.

Osiurak, F., Rosetti, Y. and Badets, A. (2017) ‘What is an affordance? 40 years later’, Neuroscience and Biobehavioral Reviews 77: 403-417.

Parchoma, G. (2014) ‘The contested ontology of affordances: Implications for researching technological affordances for collaborative knowledge production’, Computers in Human Behavior. 37: 360-368.

PeakFinder Ltd. (2018) PeakFinder AR. Available at: https://apps.apple.com/us/app/peakfinder-ar/id357421934 [Accessed 13 June 2019].

Rutz, H. (2016) ‘Agency and algorithms,’ CITAR Journal 8(1): 73-83.

Schrock, A. R. (2015) ‘Communicative affordances of mobile media: Portability, availability, locatability, and multimediality’, International Journal of Communication 9: 1229-1246.

Vaast, E. et al. (2017) ‘Social media affordances for connective action: An examination of microblogging use during the Gulf of Mexico oil spill,’ MIS Quarterly 41:1179-1206.

Van Dijck, J. (2016) The Culture of Connectivity. Oxford: Oxford UP.

Wise, J. M. (2014) ‘Hole in the hand: Assemblages of attention and mobile screens’, in: A. Herman, J. Hadlaw and T. Swiss (eds.) Theories of the Mobile Internet. New York, NY: Routledge, pp 212-31.

Yelp, Inc. (2018) Yelp. Available at: https://apps.apple.com/us/app/yelp-food-services-around-me/id284910350 [Accessed 13 June 2019].

 

Notes

[1] For a related reading of this cultural shift in computing as eversion, in the context of a discussion of the digital humanities, see Jones (2014). See also Farman (2012: 3-12).

[2] As Norman (2013) himself notes, his use of the term “signifier” in this context is quite distinct from how the term appears in semiotics (14).

[3] This would be an example of what Norman (2013) calls natural mapping, although effective mapping can also arise from arbitrary pairing between action and interaction, as long as the model is both discoverable and memorable. See Norman (2013: 22-23).

[4] As Bucher and Helmond (2018) note, Latour himself acknowledges Gibson’s influence in developing his understanding of actor networks (16-17; Latour, 2005: 72).

[5] For discussion of the role of feedback in sustaining conceptual models and mappings, see Norman (2013: 23-25).

[6] Karahanna et al. (2018) are by no means the only researchers who have attempted to catalog new media affordances, but they provide a useful example, both in how recent their research is, and in their attempt to align their taxonomy with the work of other scholars. See Karahanna et al. (2018: 744-747) for a comparative alignment of their taxonomy with previous attempts to catalog social media affordances.

[7] Schrock (2015) identifies “locatability” as a communicative affordance (alongside portability, availability and multimediality,) with a direct impact on communicative practices that includes “coordination,” “surveillance,” and “locational identity” (1235).

[8] Monocle has always been a “hidden” feature, originally accessible only through a body-enabled “Easter egg” unlocked by shaking one’s smartphone three times (Chen, 2009). Yelp introduced Monocle in 2009, around the same time that Jones (2014) cites a rise in everyday experiences of “mixed reality” as well as representations of these experiences in works of literature and film; as such, we may add this feature to the list of “a variety of changes in technology and culture [that] converged and culminated in a new consensual imagination of the role of the network in relation to the physical and social world” that he associates with the eversion of cyberspace (25).

[9] I am by no means alone in making this observation. For a detailed discussion, see Farman (2012: 35-55). See also, for a more recent contribution to this conversation, J. Didur and L. T. Fan (2018).

[10] For more discussion, see Nunes (2006): 11-19. See also Miller (1995: 3-5) for a more general reading of topography as performative speech act.

 

Mark Nunes is Professor of Interdisciplinary Studies and Chair for the Department of Interdisciplinary Studies at Appalachian State University. He is author of Cyberspaces of Everyday Life (Minnesota, 2006) and editor for a collection of essays entitled Error: Glitch, Noise, and Jam in New Media Cultures (Continuum, 2011.)

Email: nunesm@appstate.edu

 

This article is from the special issue, Rethinking Affordance (Media Theory 3.1), edited by Ashley Scarlett and Martin Zeilinger.

The official version of record is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/85

The full issue is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/issue/view/4