Ashley Scarlett & Martin Zeilinger introduce their special issue of Media Theory on Rethinking Affordance.
The official version of record is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/78
The full issue is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/issue/view/4
Rethinking Affordance
ASHLEY SCARLETT
Alberta University of the Arts, Canada
MARTIN ZEILINGER
Abertay University, UK
Introduction
Jol Thomson’s Deep Time Machine Learning (2017-2019) is a single and multi-channel video installation that captures, among other things, the playful investigation of a very old mechanical device by way of a very new technological apparatus (Fig. 1). The role of the old is filled by the first fully functional 4-stage hand-cranked calculator – conceptualized and built by the German pastor, astronomer and inventor Phillipp-Matthäus Hahn in the 1770s, the calculator is a wondrously intricate mechanical device capable of addition, subtraction, multiplication, and division; Phillipp-Matthäus was amongst the first to build a functional machine capable of all four basic arithmetical operations, initiating the precision industry in Württemberg (Klemme & Kuehn, 2016). The video installation captures Hahn’s device as it is scrutinized by an equally wondrous next-generation six-axis robotic arm. Designed by Bosch GmBH engineers specialising in ‘robot-human collaboration,’ the APAS robotic arm is a deceptively simple-looking machine equipped with a wide range of advanced imaging optics, and sheathed in a proximity-sensing “skin” that allows the robotic arm to operate at high speeds in very close proximity to humans.[1] In Thomson’s video, the APAS robot subjects the mechanical calculator to a variety of different sensor-based and computational ‘ways of seeing’ that range from regular video capture to laser-guided 3D-measurement and the recording of optical data that is invisible to the human eye. This allows the device to observe the object before it through a perceptual apparatus that far surpasses what human agents generally mean by ‘seeing.’ Thomson reveals this to the viewer by pairing video documentation of the interactive environment in its entirety alongside visualizations of the different forms of visual and non-visual data captured during the project (Fig. 2). The work is punctuated with textual excerpts that are drawn from a European Parliament report on Civil Law Rules on Robotics (2017) and that call for a consideration of the ‘subjectivity,’ rights, and liabilities of intelligent machines.[2]
Deep Time Machine Learning explores the speculative interfacing of the historical with the futuristic, of the familiar with the unknown, and in doing so thematizes the perceptual limits of what is humanly knowable about mathematics, computational logic, machine vision, and interactions between technological devices. To this end, the arrangement of old and new in Thomson’s work, as well as the new modalities of non-human perception that are forefronted, press viewers to attend to the shifting affordances of technological tools and intelligent systems, as well as of machine-human and machine-machine interactions. From a human perspective, the interactions depicted in the video still (Fig. 2), which rely on advanced stereoscopic vision and light detection 3D measuring (essentially the same LiDAR machine vision technology used in self-driving vehicles and other mobile, semi-autonomous devices), become a meditation on the purposes and affordances of emerging technologies. While the APAS arm has been praised primarily for its proximity- and touch-sensitive leather ‘skin,’ an innovation that engineers at Bosch imagine will significantly alter human-robot relations as well as labour and industry practices (Thomson, 2017-2019), it also triangulates a diverse range of data that enables it to navigate its surroundings in a manner that both surpasses and marks the uncanny limits of human capability. Thomson’s visualization of this information exposes this discrepancy and points to alternative and augmentative means through which one might perceive, relate to, and make use of the environment. Like the APAS robotic arm, the old mechanical calculator was also once a cutting-edge, futuristic technology, equipped with powers that allowed it to undertake calculations beyond normal human cognitive ability. The mechanical calculator ‘divines’ complex mathematical truths; the robotic arm ‘feels’ its human co-workers – both technologies produce and deploy ‘invisible’ realities that otherwise are not immediately available to human agents.
Thomson’s work was produced as part of the “Wimmel Research-Fellowship,” situated on the main campus of the Robert Bosch GmBH’s engineering arm in Southern Germany near Stuttgart, and co-organized with nearby Akademie Schloss Solitude, a public foundation that hosts artist and research residencies. Such connections between experimental art and technology research centres continue a long tradition, which includes illustrious examples such as residencies hosted at Bell Labs, Xerox PARC, or, more recently, the Pier 9 residency program at AutoDesk. Like the Wimmelforschung residency, these predecessors sought to leverage experimentation in art-making to help push the functional, commercial and discursive parameters of existing and emerging technologies. Artists, in exchange for access to new tools and technical expertise, are regularly invited to collaborate across disciplinary or medium-specific boundaries in an effort to elicit the imagination, identification and communication of new and unforeseen affordances (Noll, 2016; Scarlett, 2018). This applies particularly to emergent technologies, where conceptual frameworks or contexts for practical implementations may not yet have been determined or rendered habitual.
Deep Time Machine Learning captures the intersecting practices and pressures that initiated this special issue. Thomson’s work explores the horizons of possibility associated with the uses and functions that a given technology may afford. Not only does it employ devices that have stood at the forefront of technological innovation, expanding the potential for human action and interaction with the environment, but it also captures a generational shift in how and where and to what extent computational machines are interfacing with and making use of their surroundings. Critical in this case is the sense that these operations unfold largely beyond the limits of human perception and would therefore have remained invisible had Thomson not provided a visual representation of them, at least where the APAS robotic arm is concerned. Furthermore, Deep Time Machine Learning was developed in a corporate research environment that was designed to facilitate a reimagining of technological potential and use by pushing participants (artists, engineers, researchers, etc.) beyond familiar frames of reference, and into challenging new constellations of cross-disciplinary collaboration. At stake in each of these instances appears to be a renegotiation and reconceptualization of ‘affordance.’
‘Affordance,’ which we introduce and survey in greater detail below, features centrally across a growing number of scholarly disciplines, including: psychology; design; human-computer interaction (HCI); communication studies; media studies; organizational studies; and education. As is widely acknowledged in these fields, the term was coined by cognitive and ecological psychologist, J.J. Gibson. In transforming the verb ‘to afford’ into a noun, Gibson sought to account for the fundamental means through which agents (human or otherwise) navigate, conceptualize and more generally relate to their environment. “The affordances of the environment,” he explained “are what it offers the animal, what it provides or furnishes, either for good or ill” (Gibson 2015: 119). For Gibson, then, agents’ perception and implementation of what the environment offers, provides or furnishes – ultimately what behaviours it enables – is the primary way in which they make sense of and become enmeshed with their surroundings. Drawing upon Gibson’s work, if only for inspiration, researchers within the domains of design (e.g., Norman, 1988) and HCI (e.g., Haugeland, 1993; Smith, 1996) were quick to amend, apply and popularize the term. Most prominent amongst these scholars and practitioners was Don Norman, who argued that a designer’s task was to make the intended uses of an object or environment – treated here as nearly synonymous with ‘affordances’ – readily apparent to and easily enacted by an imagined user (Stone et al., 2005). As we discuss further below, in this configuration of affordance, Gibson’s focus on the concept’s relationality gave way to the assumption that the concept circumscribes clearly delimited uses that can be determined and rendered explicit by the designer in order to direct (and constrain) use and prescribe action. In the work of Norman and others, this has increasingly included an application of affordance theories to digital artefacts and environments.
The APAS robotic arm and hand-cranked calculator that feature centrally in Deep Time Machine Learning offer human agents a series of affordances; for example, when embedded in their ‘natural’ environs, both devices have enabled humans to interactively overcome particular limits where labour and reliable calculability are concerned. Both devices also stand as exemplars of humans’ drive to expand their field of action through the innovation and design of new technologies and, by extension, novel affordances. This being said, human agents are not featured in Thomson’s multi-channel video. Instead, the APAS robotic arm surveys the machine; its multidimensional and triangulated perception of the interaction drives (and therefore enables) the arm’s subsequent behaviours. What becomes apparent is not only that the robotic agent has the capacity to autonomously identify and enact environmental affordances, something that would be required within the industrial context in which it is intended to operate, but also that these actions mark the culmination of cascading operations that unfold below the perceptible surfaces of mediation. Thomson’s work suggests that the range of affordances at play here might not simply be those that humans can perceive in the surrounding environment, but also those that exist and are enacted within the algorithmic underbelly of digital computation.
The allusion here marks a significant departure from canonical accounts of affordance. Despite being common parlance within contemporary design and HCI discourse (Nagy & Neff, 2015), the original theoretical apparatus out of which the concept of affordance emerged has yet to undergo a critical re-examination in light of the term’s ‘digitization;’ scholars concern themselves increasingly with the identification of affordances associated with particular digital tools and artefacts, but rarely is the affordance concept revisited in order to better account for the complex computational and algorithmic grounds through which these objects of analysis are constituted. Consequently, ‘affordance,’ rather than contending explicitly with the computational or algorithmic, continues to operate within a conceptual framework of objects and environments that are defined by their physicality, phenomenological accessibility, and liveliness (e.g., Wells, 2002; Morineau et al., 2009). Many of the defining characteristics of affordance, as it had been previously conceptualized, therefore conflict with what have been described as the evasive realities of digital media (Lovink, 2014; Parisi, 2013; Zielinski, 2008). Furthermore, a grounding in the physical and phenomenologically apparent also overlooks the emerging sense that through the sensorial collection, aggregation and enactment of data, algorithmic systems are arguably learning to recognize and respond to virtual affordances that lie outside of the realm of human consciousness (Gabrys, 2016; Hansen, 2015; Massumi, 2015; see also Nunes, this issue). Developments like these, addressed in detail in the latter sections of this introduction, call into question the extent to which the concept of affordance in its original formulation is still useful, relevant, and meaningful, particularly in theoretical analyses of and practical engagement with the digital.
Our aim with this special issue is, therefore, to undertake a critical and creative re-examination of ‘affordance’ for the digital age. This means to explore the critical, historical and contemporary valences of the concept in a manner that productively engages with the dynamic malleability of the digital, highlighting the critical potentials that this dynamism embodies. The contributions collected here pursue this goal by proceeding along three vectors: historical (e.g., renegotiating the continuities and tensions between different perspectives on the affordance concept), theoretical (i.e., theorizing the uses and meanings of the concept in critical dialogue between digitally-oriented practitioners, researchers, and other stakeholders), and artistic (i.e., exploring how media artists have engaged with, reimagined and conceptualized technological affordances).
The remainder of this introduction will build out a conceptual framework for the contributions to this issue. We will begin by offering comprehensive overviews of the two earliest, and most prominent critical perspectives on ‘affordance,’ J.J. Gibson and Don Norman. After reviewing various alignments and contrasts in their positions, as well as their significance for a wide range of fields of research, the subsequent sections transition from the original context of affordance theory – the relationship between objects, environments, and their users – to consider recent efforts to identify and begin accounting for the specific affordances attributed to particular media technologies. Central to this discussion will be a consideration of the ‘novel’ affordances made possible by contemporary communication technologies, as well as a realization that these affordances emerge from and unfold in concert with auxiliary layers of affordance that are corresponded with the material grounds and digital operations of computational systems. This range of affordances is actualized despite being technically imperceptible, marking both a departure from canonical accounts of affordance as well as a call to ‘rethink’ the affordance concept in response to the particularities of its computational and algorithmic realization. The final sections identify and unpack three areas of analysis through which we might begin to answer this call. First, building on an account of the material and formal grounds of computation, we begin to parse and conceptualize the imperceptible configuration and operations of computational affordances. Second, we undertake a practical and theoretical analysis of recent efforts to instrumentalize and automate the concept and execution of affordance through algorithmic means. Finally, we move to a sustained discussion of how the concept of affordance figures in and resonates with contemporary digital art. The essay will conclude with a brief introduction to each of the contributions to this special issue.
Framing affordance: Gibson and Norman
The cognitive and ecological psychologist J.J. Gibson first coined the concept of affordance in his book The Senses Considered as Perceptual Systems (1966). Gibson continued to refine and expand the concept in “Affordances and Behaviour” (1975), and finally offered his most sustained discussion of the concept in The Ecological Approach to Visual Perception (1979).[3] Focusing his discussion on interactions between live agents (both humans and animals) and their environments, Gibson used the term affordance to explore the actionable properties of environments and, by extension, of physical objects. Doorknobs afford the opening of doors; steps afford climbing or descending between floors; cliffs afford falling off. With a view to Don Norman’s reconceptualization of affordance (see below), it is noteworthy that, for Gibson, objects as such only represent a subset of the more general environments with which humans can interact. The affordances of an object or an environment are thus assumed to describe the phenomenological qualities it embodies, by projecting potential uses, delimiting possible actions, and signalling possible functions for the object or environment in question.
In Gibson’s original conception, affordance is a decidedly environmental (or ecological) phenomenon. On the one hand, an affordance may exist independently of whether or not an agent who could act upon it actually recognizes it; at the same time, any affordance is only actualized when it is acted upon. Additionally, one and the same object (or environment) can embody different context-specific affordances (a shoe, for example, could protect a foot while walking, but it can also be used to hammer in a nail, or open a bottle of wine). Affordances thus exist independently of human intention but can nevertheless not materialize without them. These characteristics have also been discussed as “relational” (e.g., Hutchby, 2001) and “interactional” (e.g., Nagy & Neff, 2015), at times with reference to the “situativity” of human-environment interactions (see Greeno, 1994). As Gibson (1979) puts it, “An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behaviour. It is both physical and psychical, yet, neither. An affordance points both ways, to the environment and to the observer” (129).
According to Gibson, any interaction between human agents and their environment could be described as geared towards the manipulation of this environment, for the purpose of shaping affordances that are more amenable to the intended uses. The oft-invoked example of the teapot emphasizes this: the functions and uses of this object are generally assumed to be embodied in the object’s physical characteristics – its handle is the only spot that allows a human user to comfortably hold the teapot without burning their fingers; the wide opening on top lends itself ideally for the action of filling the object with liquid, while the narrow neck and mouth are ideal for controlled pouring-out of the liquid. Often, such potential uses (but also their limits!) may be graspable even to someone who hasn’t previously seen or used the object in question. Nevertheless, a teapot’s affordances materialize only in and through the actual interaction. As Gibson notes, an object’s affordances may be grounded within its material form, but are, ultimately, realized through processes of identification and purposeful implementation through an agent. As such, affordance is an inherently relational concept which, for Gibson, accounts for the “middle ground wherein the perceiver and the perceived actually meet” (Letich & Lisack, 2009: 62); i.e., where uses and functions are actualized through an interaction between the user and the object/environment in question. Importantly, this focus on relationality also indicates that affordances should not generally be considered as fixed and stable; they are, as Gibson states, “relative to” and thus also “unique for” the agent in question (see Gibson, 1979, Chapter 8). Because the agent must recognize an affordance in order to realize it, Gibson’s affordance theory is intimately tied to theories of learning and socialization – human agents learn to recognize uses, functions, and limits of objects and environments, and consequently also strive to alter them as needed. In an important contrast to Norman’s perspective, Gibson thus considers an environment’s affordances to exist independently from its potential users’ ability to recognize them.
Over the last four decades, the meanings associated with the term affordance have begun to significantly diverge from the original definitions Gibson offered. The most noteworthy and dominant departures from the Gibsonian affordance concept are represented by the work of cognitive scientist and design theorist Don Norman, whose perspective is now very widely adopted in the field of design (from product design to user experience and interface design), frequently to the point of eclipsing Gibson’s perspective. While Norman built on Gibson’s foundational work, in part he also negates or contradicts it. Primarily working in design contexts, Norman has developed a perspective which, in comparison to Gibson’s, is much less focused on the multifarious interactions between agent and environment/object (as well as the dynamic nature of these interactions); instead, Norman foregrounds specific uses and functions and the assumption that they can be built into an object ‘by design.’ As McGrenere and Ho (2000) have put it, for Norman “an affordance is the design aspect of an object which suggests how the object should be used” (n.p.). Related to this, a key aspect underlying Norman’s work on affordance is the idea of the “conceptual model” (e.g., Norman, 1999), which he conceives of as explanations that delineate for users how something works, such that they are able to construct mental, interactive models of it (2013: 25-26); design, in other words, is supposed to project conceptual models based on which users can perceive an object’s or environment’s affordances.
The departure from Gibson’s model is significant; as Martin Oliver (2005) has observed, “Indeed, so little of Gibson’s intended sense of the word remains that the appropriateness of its use must be questioned” (407). Where Gibson’s perspective was meant to open up our understanding of the relational ontology of objects and environments, of their uses and purposes, and the limits thereof in relation to human agents, Norman’s view narrows this ever-widening and potentially open-ended horizon: “Affordances provide strong clues to the operations of things. Plates are for pushing. Knobs are for turning. … When affordances are taken advantage of, the user knows what to do just by looking: no picture, label or instruction is required” (Norman, 1999: 9).
Norman thus replaces Gibson’ interactional, relational focus with a user-centred focus: for him, an affordance is something with which a designer imbues an object in order to guide and channel (some might say to control and limit) that which a user perceives as the object’s uses and functions and, consequently, the uses which a user can imagine to be possible. The important keyword for Norman is ‘to perceive.’ In his focus on users (and, by extension, on by-design usability), Norman foregrounds “perceived affordances” above all else, a designation by which he means properties of an object that are actually perceived by a user and which can therefore be acted upon. This is in clear contradistinction to Gibson, for whom, as noted, ‘affordance’ referred to an interactional possibility that exists independent of an actor’s ability to perceive this possibility. Norman here differentiates between his ‘perceived affordances’ and what he calls ‘real affordances,’ which he describes as affordances that may exist, but which a user cannot act upon if they cannot be perceived. This distinction is so important for Norman that he states, “all affordances are ‘perceived affordances’” (Norman, 1999: 39).
Widely adopted in design and engineering contexts, Norman’s view now frequently dominates discussion about and understanding of the concept of affordance, to the point where elaboration on Norman’s perspective often takes precedence over Gibson’s originary discussion of the term (see, for example, The Glossary of Human Computer Interaction). While Norman adopts from Gibson the perspective that affordances are embodied in objects and thus circumscribe an object’s phenomenological characteristics, Norman perceives these affordances to be rather a lot more fixed than Gibson. Where Gibson foregrounds how affordances emerge – necessarily and inevitably – in relational constellations of environment and agent, Norman proposes that affordances can be designed and subsist, in an object as abstractable as a button, independently from environments (and technologies) that might mediate between object and user. In other words, Norman’s object-centric and user-focused approach insists that affordances are designed, and that, if they are well-designed, they are more or less fixed.
As an indication of the significance of this departure from Gibson’s thinking on the subject, it may be useful to highlight that Norman’s most well-known book on the topic was initially published as The Psychology of Everyday Things (1988), but later re-released as The Design of Everything Things. The changed title is programmatic for Normans’ perspective: it marks a shift away from focusing on the way in which affordances emerge necessarily in the interactional link between object/environment, on the one hand, and user, on the other, and towards a focus on the object itself, which, Norman argues, projects a fixed set and quality of affordances that remain stable across interactional configurations and events. Where Gibson would certainly have discounted such a view, for Norman the ideal goal of design is to lock affordances into place, aiming for them to become ‘invisible’ and ‘one’ with the object to which they become attached. Norman’s account of the intersection between ‘affordance’ and ‘design’ renders the ideological underpinnings of affordance explicit. While Gibson’s relational account of affordance unfolds at the ideological intersection between mind and matter, Norman’s account hinges on the designer’s instrumentalization of the affordance concept and, therefore, on ideologically-laden interventions that tend to close down, rather than broaden, the interactional horizon of an object or environment.
A call to ‘digitize’ the affordance concept
Norman’s theorization of affordance spurred its popularization. Not only did he align the concept with the field of design, introducing and reformulating it in a manner that appealed to scholars and practitioners working in a number of corresponding subdisciplines (see, for example: Gaver, 1991; Haugeland, 1993; Flach and Dominguez, 1995; Smith, 1996), but his work also identified a critical set of connections between the concept of affordance and the burgeoning terms of computational media. While the concept of affordance in its initial formulations accounted for physically robust artefacts and phenomena, through Norman it was increasingly applied to digital artefacts and environments, particularly within the fields of interaction design (e.g., Hartson, 2003), software development (Pressman and Maxim, 2014), information science and information architecture (e.g, Bernhard et al., 2013; Pozzi et al., 2014), interface design (Drucker, 2014; Ruecker et al., 2011), and user experience design (e.g., Pucillo and Cascini, 2014). Within the corresponding context of Communication and Media Studies, the concept of affordance was adopted as a means of making sense of the operational potential of devices and platforms (e.g., Gillespie, 2010; Neff et al., 2012), as well as in an effort to identify the emergent terms through which media might, indeed, be deemed ‘new’ (see Manovich, 2001; Lister, 2009).
Yet, despite the cross-disciplinary adoption of the ‘affordance’ concept, as noted in the introduction, very few scholars have sought to significantly update Gibson’s or Norman’s theoretical frameworks in order to more thoroughly account for the realities of digital, rather than physical, systems; not only does much of the contemporary research on affordances involve a straight-forward review and adoption of both scholars’ theoretical frameworks, a tendency that Evan et al. (2017) have associated with a lack of ‘theory-building’ where contemporary accounts of ‘affordance’ are concerned (36), but there has yet to be a critical examination of the increasingly prominent intersection between ‘affordance’ and ‘algorithm’ (Ettlinger, 2018).[4] This is a significant oversight. As Nancy Ettlinger (2018) has articulated, “affordances as a field of possibilities are considerably more complex in algorithmic life than in a Gibsonian environment-actor relation…” (3). One of the reasons for this, she explains, is that digitally mediated environments encompass an expansive and diverse assemblage of “animate and inanimate actors in addition to public and private-sector actors connected to them” (ibid). While the same might be said for any environment-actor relation that is embedded within a larger assemblage of actors, objects, and environments, Ettlinger reminds her reader that the algorithmic field of possibilities is comprised of increasingly complex constellations of intersecting feedback loops, driven in large part by the solicitation, aggregation, operationalization, and calculated implementation of data (Nunes, this issue, addresses similar issues). Within this context, human agents are increasingly encountering ‘smart’ and networked technologies, whose potential and real affordances stretch beyond their interactive surfaces, into the imperceptible yet affective undercurrents of their coded operations, networked infrastructures, and socio-cultural apparatuses. Not only does this point to the ‘nested’ and ‘cascading’ layers that comprise computational technologies, but it also calls attention to the different modes and means of affordance that these technologies have the capacity to enact. It is primarily in response to this complex situation that Ettlinger (for whom this scenario is a feature of ‘algorithmic life’ more broadly) determines the inadequacy of canonical conceptualizations of affordance. Furthermore, she argues that the algorithm, as both a process and object, is a new phenomenon that requires a phenomenon-specific theorization of affordance.
Despite the significant theoretical gap that Ettlinger identifies, numerous scholars and practitioners have indeed begun to develop technologically oriented accounts of affordance, with an increasing interest in contending with the forensic grounds, algorithmic infrastructures, and digital artefacts that comprise contemporary computation (see for example: Best, 2009; McVeigh-Schlutz & Baym, 2015; Davis & Chouinard, 2017; Hurley, 2019). While much research has considered the affordances that emerge from and correspond to the use of specific technologies (see for example: Graves, 2007; Sutcliffe et al., 2011; Moloney et al., 2018), other scholars have worked to reorient the theoretical parameters of affordance to begin grasping the particularities of computational processing, if only in metaphorical terms (see for example: Leonardi, 2011; Nagy & Neff, 2015). In the case of the former, it is often the physical and phenomenologically apparent surfaces of technology (and responding practices) that are considered; in the case of the latter, attempts are made to account for the hidden, or ‘imagined,’ dimensions of algorithmic mediation and digital artefacts. Despite an interest in mapping the specific grounds of computational affordances, very few accounts move from a treatment of ‘the digital’ as a sweeping cultural phenomenon to an examination of the specific technical and algorithmic means through which the digital operates and is materialized (boyd, 2010). Even efforts by some to grasp at the imperceptible dimensions of computational processing forego the specific grounds of the digital in favour of allusive language.
Following these trajectories of research, we will now turn to a selective review of scholarly responses to the technological and increasingly digital dimensions of affordance. In addition to providing an overview of key texts, the discussion in the sections below aims to contribute to scholarly accounts of the invisible and imperceptible affordances associated with digital systems. Our objective is to begin mapping out a theoretical scaffolding capable of accounting for the materially complex and nested grounds of digital affordances, as well as the increasing instrumentalization, implementation, identification and actualization of affordances through algorithmic means.
Identifying the affordances of contemporary media technologies
From cognitive psychology and design to social media studies, education and law, the affordance concept has been taken up, with increasing intensity it would seem, across an expanding array of contemporary disciplines as scholars work to make sense of the effects that 21st century media technologies are having within their respective fields of study (see for example: Alvarez, this issue; Diver, 2018; Costa, 2018; Carah & Angus, 2018; Heemsbergen, 2019; Hurley, 2019). Within these contexts, analysis of ‘affordances’ is often advanced as a ‘third way’ (Hutchby, 2001: 444) of approaching media criticism; an affordance-based approach stands between and draws together discourses of technological determinism and ‘enframement,’ on the one hand (Finn, 2017: 118; Mitchell & Hansen, 2010), and social constructivism, on the other. As McVeigh-Schlutz & Baym (2015) explain, analyses that depart from a consideration of ‘affordance’ typically address “how people make emergent meaning through interactions with technology, while also accounting for the ways that material qualities of those technologies constrain or enable particular practices” (1). In this vein, they recognize that while media technologies are comprised of “a set of practices that cannot be defined a priori, and [that] are not predetermined outside of their situated everyday actions and habits of usage” (Costa, 2018: 3642), their material and structural constitution “request, demand, allow, encourage, discourage, and refuse” (Davis & Chouinard, 2017: 242) particular kinds of use.
The practices that emerge at the intersection of these differing pressures have been conceptualized as both broad indicators of the communicative, social, and political affordances of contemporary media technologies, as well as medium-specific affordances (see for example: Heemsbergen, 2019; Schrock, 2015; Sutcliffe et al., 2011). Andrew Schrock (2015), for example, has reviewed over a decade of research into the effects of mobile media on communication practices, and argues that what is central, but often overlooked, within this scholarship is an understanding of mobile media’s ‘communicative affordances’ (1234). According to Schrock, the term ‘communicative affordances’ comprises an overarching class of affordances and describes instances in which the relational intersection between “subjective perception of [technological] utility and objective qualities of a technology … results in altered communication and subsequent patterns of behaviour” (1239). Under the banner of ‘communicative affordances’ falls a collection of medium-specific affordances as well, each of which affects configurations and practices of communication. For example, reflecting on mobile media, Schrock identifies device portability, user availability and ‘locatability,’ as well as the convergence of an assortment of mediums and platforms (1235) as key affordances that have significantly altered communication practices. Similarly, Treem & Leonardi (2012) and Evans et al. (2017) chart a series of communicative affordances that are specific to social media technologies (such as blogs, wikis, and social networking sites), focusing on visibility, editability, persistence and association (Treem & Leonardi, 2012; Evans et al., 2017) as well as anonymity (Evans, 2017: 41).
Of the prominent recent accounts of affordance, boyd (2010) provides one of the very few (if not also the most robust) considerations of how the digital and algorithmic grounds of contemporary media technologies contribute to the affordances that they help to realize. In “Social Network Sites as Networked Publics: Affordances, Dynamics, and Implications,” boyd explores how the technologies that constitute and structure so-called ‘networked publics’[5] afford particular kinds of social engagement. Rather than suggesting that social behaviours are determined through the technological media that enable them, boyd turns to affordance theory to recognize how the technologies that comprise ‘networked publics’ are ‘actualized’ through the very practices that they enable and shape. While boyd is primarily concerned with parsing the social and communicative affordances of networked publics, she begins by differentiating between the material grounds of physical and digital technologies. This enables her, by extension, to differentiate between the particularities of physical and digital affordances. Perhaps obviously, boyd accomplishes this by explaining that the physical is materially delimited by the atom while the digital is comprised of bits. “The underlying properties of bits and atoms,” she explains, “fundamentally distinguish these two types of environments, define what types of interactions are possible, and shape how people engage in these spaces” (41). Unlike atoms, bits are easily “duplicated, compressed, and transmitted through wires” (ibid.). They are also “easier to store, distribute, and search than atoms” (46). The affordances of networked publics are, by extension, “shaped by the properties of bits, the connections between bits, and the way that bits and networks link people in new ways” (41).
boyd maps her claims concerning the properties of bits across a close examination of the defining features and practices that comprise social network sites (such as profiles, friends lists, and tools for public communication). She identifies four affordances that “emerge out of the properties of bits,” and in turn “play a significant role in configuring networked publics” (46). These affordances include persistence, replicability, scalability, and searchability (46), each of which introduce[s] new dynamics that participants in ‘networked publics’ must contend with (48). According to boyd, these dynamics include the emergence of questions concerning visibility and anonymity; a collapse of distinctions between the public and private sphere; and a decontextualization of social and communicative exchanges. In this vein, boyd’s work offers both a consideration of how the affordances of networked publics are transforming the practices that comprise communication and everyday life as well as an account of how the material specificity of bits gives rise to a series of affordances that are fundamentally different from those associated with physical objects and environments.
From the perceptible to the imperceptible: the grounds of computational affordances
boyd’s work identifies and begins to account for the ways in which the underlying building blocks of digital systems affect their corresponding affordances. This being said, she does not consider the affordances of the algorithmic means through which they work and are put to use. Her project aims instead to assess the affordances that are materialized through the use of social media platforms and in relation to ‘networked publics,’ connecting it more thoroughly to work that identifies the specific affordances of contemporary media technologies. This being said, boyd’s turn to bits as the definitive grounds of digital affordances identifies how the different-yet-intersecting layers of materiality that comprise the digital challenge the applicability of traditional conceptualizations of affordance; each layer initiates a different sense of where and how the affordances of digital systems arise and operate. A contemporary account of affordance must therefore encompass the different modes of materiality through which the digital operates, and in relation to which the affordances of digital systems are realized. Indeed, Ian Hutchby (2001) has argued that agents’ conceptualization and use of technological artefacts are fundamentally shaped by “the ranges of affordances that particular artefacts, by virtue of their materiality, possess” (193, emphasis added). It is important to understand, as a result, that as the materiality of the digital shifts, so too do its potential affordances. Bloomfield et al. (2010) use the term “‘cascades’ of affordances” (ibid.) to describe this phenomenon (420), highlighting the co-articulatory (Latour, 1999) and processual emergence of digital affordances, as they unfold across time and in response to shifting layers of materiality. As Nagy & Neff (2015) suggest with their affordance-oriented assertion that “communication theory deserves a richer theory of the materiality of media” (1), in order to better understand the multilayered affordances of the digital, it is critical to develop a clear understanding of digital materialism.
In Mechanisms: New Media and the Forensic Animation, Matthew Kirschenbaum (2012) provides a dialectical account of digital materiality, comprised of the iterative synthesis of forensic and formal materialism. Grounded within the “richness of a physically robust world” (9), ‘forensic materiality’ here refers to the physical and embodied dimensions of the apparatuses, environments and practices that comprise digital technologies. For Kirschenbaum, this includes the physical “residue of digital inscription” (10), “surfaces, substrates, sealants, and other material that have been used … as computational storage media” (10), as well as the “labour practices that attend [to] computation” (ibid.). Rather than treating the physical underpinnings of computation as the exclusive grounds of digital materiality, Kirschenbaum introduces ‘formal materialism’ to account for “the multiple behaviours and states of digital objects and the relational attitudes by which some are naturalized as a result of the procedural friction, or torque … imposed by different software environments” (132-133). Formal materialism, then, arises through the “simulation or modelling of materiality via programmed software processes” (9). These processes impose “a specific formal regimen on a given set of data” (13), lending it, and ‘the digital’ more broadly, an aesthetic and material sense of cohesion and durability. Formalization does not only provide a perceptible and seemingly stable surface through which to identify and enact the affordances of the digital, but formalized image objects and environments also offer a selective glimpse into the undercurrents of mediation as they are seen to index the processual intersection between hardware, software and code (Hand & Scarlett, 2019).
Following this line of reasoning, the materiality of the digital emerges through the ‘sustained duality’ of forensic and formal modes of materialism (Kirschenbaum, 2012; Drucker, 2009); digital objects and environments are understood in this case to be forensically grounded, processually executed and formally durable. The intersection and coincidence of these modes of materiality help to differentiate between the multilayered or ‘nested’ (Gaver, 1991) affordances actualized through agents’ interactions with digital technologies, objects, and environments. The affordances of the digital are not only shaped by the ‘forensic’ materials that undergird digital technologies, rendering them operable, graspable, and interactable, but Kirschenbaum’s conceptualization of digital materiality also helps to account for affordances that are grounded within the iterative, ephemeral and seemingly ‘dematerialized’ structures of formal regimens. While afforded, to an extent, by the forensic, in rendering the imperceptible layers and processes of computation perceptible and seemingly material, the formal dimensions of digital materialism help to establish the conditions of possibility for recognizing the affordances of the digital whatsoever.
Formal regimens do not only render the affordances of the digital apparent, they also actively frame digital affordances in a manner that forefronts the seemingly ‘immaterial’ qualities of formal materialism. At stake in this case is both a consideration of the means through which digital objects and environments are ‘enformed,’ as well as the ideological pressures that inform these processes (Chun, 2006; Galloway, 2006). Analyses of the former might call for a close consideration of the role that code plays in the semiotic delineation, generative execution, formalization and perceptual stabilization of particular affordances. As we discuss in greater detail below (with regards to software studies), this line of inquiry necessitates both a consideration of the affordances associated with coded language, and ‘actionable’ signifiers more broadly, as well as an analysis of how the formalized qualities that delimit digital objects and environments inform and contribute to our sense of what the digital affords as well as what affordances are particular to the digital as such. For example, we might consider how it is that the formalized qualities of digital objects and environments contribute to their perceived manipulability, scalability, deletability and undoability (Lialina, this issue), regardless of whether this is actually the case, or not.
Deletability is particularly illustrative of this notion, as users’ sense of immediate deletability is often a function and affordance of the formal, rather than the forensic, level of computation. As Kirschenbaum details, when users “delete a file from their trash or recycle bin it is not immediately expunged from their hard drive. What happens instead is that the file’s entry in the disk’s master index … is flagged as space for reuse” (50). As such, “the original information may yet persist for some time before the operating system gets around to overwriting it” (50-51). As we discuss in greater detail below, for Gaver (1991) this would likely suggest that deleability is in fact a ‘false affordance.’ Similarly, moving beyond the isolated hard drive to consider the forensic grounds of ‘deletability’ in networked environments, Treem & Leonardi (2012) and Evans et al. (2017) advance the opposite affordance, highlighting the nagging ‘persistence’ of digital information rather than its erasability. This being said, as Lialina details in this issue, at the formal level, ‘deletability’ is not only a perceived affordance, but it is also central to the ways in which digital tools and platforms are put to use within creative practice. In this sense, it is an affordance particular to the formal (if not also – eventually – the forensic) dimensions and operations of digital mediation.
Discerning the imperceptible dimensions of computational affordances
Forensic and formal materialism render the affordances of digital technologies perceptible. This being said, they also afford an awareness of the hidden (Gaver, 1991), and therefore imperceptible, dimensions of computation. While some forensic materials and processes are graspable, their blackboxed components and micro-temporal operations render the bulk of their material grounds and operations inaccessible to the human senses despite the perceptibility of their resulting outputs. Similarly, while formalized materials are, by definition, apparent to the senses, their mutable qualities and flexible materiality call attention to the imperceptible procedures and processes that make this mode of materialism possible. The affordances of hardware and software are cascading below the surfaces of computation, whether users perceive them directly or not. Despite Gibson’s and Norman’s focus on the perceptible, the imperceptibility of computation’s backend does not stop users from identifying and according ‘action possibilities,’ and therefore affordances, to it. While boyd (2010) is largely concerned with the affordances of networked publics, her work alludes to a series of “action possibilities” that are specific to bits. Similarly, Adrienne Shaw (2017) calls attention to the ways in which users ‘decode’ affordances associated with aspects of mediated experience that remain invisible to them; she uses algorithms as an illustrative example, connecting the drive to decode their encoded affordances and implications with the sense that they “affect what users can and cannot do in online space, but operate out of view” (600). Indeed, Eslami et al. (2015) have also demonstrated that whether or not users are able to decode or understand algorithms correctly, their “perceived knowledge” of underlying computational processes affects how they interact with devices as well as how they behave more generally (153). A growing awareness of the presence and cultural implications of algorithmic technologies’ submedial undercurrents (Groys, 2012), paired alongside a willingness to accord affordances to their invisible operations, has coincided with scholarly efforts to theorize and excavate the terms of imperceptible affordances more broadly. Central to these lines of inquiry are efforts to make sense of how users ‘imagine,’ construct, and project the affordances of computational (algorithmic) technologies. This is not only a matter of theorizing the imperceptible, but also points to how imperceptibility, or invisibility, might be conceived of as an affordance in and of itself.
William Gaver provided one of the first efforts to theorize the different layers of perceptible and imperceptible affordances that unfold through the operations and use of computational systems. In “Technology Affordances” (1991), Gaver expands upon Gibson’s claim that “people perceive the environment directly in terms of its potential for action, without significant intermediate stages involving memory or interferences” (ibid., emphasis added), to advance a more fully delineated account of “perceptible affordances,” “hidden affordances,” and “false affordances” (80). According to Gaver, perceptible affordances are those affordances that are recognizable when “the attributes of the object relevant for action are available for perception” (81). Hidden affordances, by contrast, are those for which “there is no information available” and that must therefore “be inferred from other evidence” (80). False affordances arise when “information suggests a nonexistent affordance,” leading people to “mistakenly try to act” (ibid.). Gaver corresponds his delineation of perceptible and hidden affordances to computational interfaces and undercurrents, respectively. Interfaces, he explains, remediate a set of underlying and otherwise hidden affordances, rendering the relevant and actionable properties of computational processes and objects perceptible. Through this formulation, Gaver does not only attest to the existence of computational affordances that remain hidden below the threshold of perceptibility, but he also calls attention to the complexity of interfaced affordances as they comprise both the perceptible features of the interface, such as the physical parameters of a device or the button and scrollbar that appear on a screen (81), as well as a cross-section of the operational undercurrents that make these computational objects and environments work. Not only does this make it difficult to discern between perceptible affordance and ‘evidence’ of a hidden affordance, but it also suggests that the perceptible affordances of computation do not necessarily belong to the system, per se. They may instead be affordances that are proper to the interface, its physical design and representational mediations, as well as the broader socio-technical and ecological context in which the encounter unfolds. This illustrates precisely the kind of complexity that complicates the easy application of Gibson’s and Norman’s theories of affordance to digital technologies and processes.
While Gaver’s interface renders hidden affordances perceptible, he does not push aside, flatten or erase the existence of these affordances through their mediated signification; hidden affordances unfold and come into existence through the processual operations of computation, whether they are experienced and perceived directly or not. Furthermore, rather than associating the perceptible affordances of the interface exclusively with the physical design and hardware that comprise the interface (as is often the case), Gaver advances an account of the coherent ‘image object,’ which he explains emerges iteratively through and ‘indexes’ the procedural operations of computation; the processual image object does not only visualize and render imperceptible processes actionable, but in so doing, it also marks the intersection between the potential affordances of the interface and the actualized affordances realized through backend operations. As this suggests, Gaver’s work does not only begin to contend with varied modes of digital materiality (as discussed above), but it also identifies and preserves the existence of affordances that remain hidden from view.
Prominent amongst accounts of affordance is Peter Nagy and Gina Neff’s (2015) conceptualization of ‘imagined affordances’ – a concept developed to better account for the role that users’ expectations and beliefs play in the identification of affordances, as well as users’ capacity to imagine the affordances of technologies and technological operations that remain hidden from view.[6] Nagy & Neff argue that what people believe and expect technologies to be able to do shapes “how they approach them and what actions they think are suggested” (4). These beliefs and expectations are not, from their perspective, restricted exclusively to that which is directly communicated or immediately perceptible, but often correspond to what people are able to imagine a particular technology might be used for (5). They explain:
Users may have certain expectations about their communication technologies, data, and media that, in effect and practice, shape how they approach them and what actions they think are suggested…This is what we define as imagined affordance… (ibid.).
While Nagy & Neff connect imagined affordances to any and all instances in which individuals attempt to identify the uses that a tool or medium might make available to them, they are particularly interested in identifying the role that imagined affordances play in shaping the relationships that comprise “complex socio-technical systems such as machine-learning algorithms, pervasive computing, the Internet of Things, and other such ‘smart’ innovation” (1). Rather than contending with the actual affordances of computational hardware or algorithmic scripts, the authors parse the ways in which users imagine and attribute affordances to these socio-technical systems, rightly or wrongly. For example, they consider how users have imagined their social media news feeds as offering objective access to their friends’ posts (and vice versa), despite the fact that this information is algorithmically mediated and therefore structurally constrained. While the objective form of communication that social media platforms are imagined to afford indicates a false understanding of what is technically happening, Nagy & Neff suggest that the affordances that are imagined lead to particular uses and actions regardless of whether or not they are, in fact, misunderstandings, misperceptions, and/or misinterpretations (5). For Nagy & Neff this is significant insofar as it suggests that reflexive engagement with imagined affordances might enable us to better make sense of and engage critically with the otherwise imperceptible dimensions of computational devices, as well as the broader socio-technical systems through which they operate.
A line of questioning that begins to emerge in response to Nagy & Neff’s work concerns the means and pressures through which particular affordances come to be ‘imagined.’ Paul Leonardi (2011) offers one possible explanation. Working within the context of organizational studies, Leonardi undertakes a critical examination of the relationship between humans and techno-material agencies in the workplace; his text aims to make sense of how the terms surrounding this relationship have the capacity to change the routines of work and/or the predominant technologies of the workplace (151). Leonardi deploys the concept of ‘affordance’ to capture the means through which humans and techno-material agencies relate and become ‘imbricated’ in one another. Drawing upon Hutchby (2001), he explains that while technological affordances are grounded within the materials and material practices that delimit a particular technology, individuals “actively construct perceptual affordances and constraints” (153, emphasis added) as they interpret and attempt to reconcile the material parameters of a particular technology with their broader “goals for action” (ibid.). Despite recognizing that many of the technologies that we encounter have been thoroughly ‘blackboxed,’ Leonardi does not differentiate between affordances that are constructed in response to that which is perceptible versus that which is imperceptible. Anticipating Nagy & Neff’s later theorization of ‘imagined affordance,’ Leonardi argues instead that the relationship between humans and techno-material agencies takes shape as individuals imagine how a particular device or tool might afford them the ability to accomplish a particular goal. This suggests a kind of ‘reverse-engineering’ of affordance, as potential uses do not emanate from an artefact or environment but are instead projected onto an artefact or environment in response to a desired result. Leonardi’s account does not only help to make sense of the pressures that might influence how affordances are imagined, but in focusing on goals for action, it also offers a way of making sense of the imperceptible dimensions of techno-material artefacts; if a device or tool enables an individual to undertake a particular action and/or achieve a desired goal, then this can be identified as one of its affordances, regardless of whether or not the individual is able to explicitly connect the affordance with perceptible qualities or characteristics of the artefact itself. This, again, provides an entry point for analyzing and critiquing the phenomenologically evasive grounds of digital mediation, albeit indirectly.
Nagy & Neff and Leonardi grasp at the imperceptible dimensions of computational affordance in a manner that ultimately allows the imperceptible to remain imperceptible. There is an understanding here that the processual operations of computation, which mark the iterative coming-together of an expansive technological apparatus (hardware and software, socio-material discourse and practice), can never be perceived in their entirety and are rarely perceived directly – there is always some component or process that remains out of reach. Before proceeding to a further consideration of the affordances of coded signifiers and representational artefacts, as well as the algorithmic means through which these affordances are increasingly identified and enacted, it is worth pausing for a moment to consider how the unavoidable imperceptibility of digital processing, identified above, has come to the fore as one of the key affordances of computational technologies. As we noted above, imperceptibility is one of the inescapable qualities of computation; not only is invisibility a material fact of the electronic and algorithmic operations that drive computation, but it also facilitates many of the purposes that computation serves within contemporary culture. Indeed, Jussi Parikka (2015) has charted the “invisible infrastructural layers that sustain what is visible” (216), highlighting how the invisibility of algorithmic logic and processing works to produce particular configurations of the social and visual. Of particular interest to Parikka are the invisible means through which algorithms produce (and in turn visualize) “financial, urban and security regimes … as one fold in the topological continuum between spatial architecture and informational ones” (213). Here, ‘invisibility’ describes both a quality of the interstitial space within which algorithmic operations unfold and configure the relationship between informational (i.e. digital) and spatial (i.e. physical) realities, as well as an affordance that is leveraged in order to secure regimented control over how this relationship is structured and rendered manifest. Following this line of reasoning, Hoelzl & Marie (2015) have detailed how invisibility is in many ways that which facilitates the collection, surveillance and commoditization of user data (101). Santos & Faure (2018) have undertaken an analysis of WhatsApp to argue that invisibility, framed as the ability to encrypt and render data imperceptible, has become a critical affordance and corresponding ‘sales tactic,’ in the post-Snowden era (9). Echoing these sentiments, Parikka follows up his consideration of invisibility by identifying how “invisibility is, in increasing ways, something that has to do with the proprietary logic of closed platforms (software) and devices (hardware), putting a special emphasis on critically tuning technological skills to investigate such ‘nothing to see’ logic…” (Parikka, 2015: 216).
As these examples suggest, the invisible dimensions of algorithmic technologies, their capacity to remain below the threshold of perceptibility by humans, are leveraged by individuals, organizations and governments to better collect, survey, and close off data. Gregoire Chamayou (2015) identifies and elaborates on this point in his theorization of the drone. Calling upon Adorno, he suggests that moments of seeming transparency and structural invisibility are in fact indicative of “a great deal of subjective activity, involving huge efforts and enormous energy, designed to cover one’s tracks, efface evidence, and wipe out any trace of a subject involved in action” (207). An affordance associated with invisibility and imperceptibility might, as a result, be understood as the capacity to erase the perception of subjective presence and interference. As Özgün Eylül İşcen (featured in this issue) argues in her critical re-examination of the affordance concept in response to the racialized, racializing and ultimately dehumanizing technologies of drone warfare, the invisibility of computational operations and algorithmic processing has the capacity, on the one hand, to afford a degree of privacy, while on the other hand also affording the obfuscation and foreclosing of responsibility and solid grounds for critique. İşcen asks, as a result, affordances for whom? From this perspective, accounting for imperceptible, but real, affordances might provide a means of triangulating, stabilizing, and in turn engaging critically with the ever-receding yet increasingly influential grounds and subjects of computation. This approach originates from the relational experience of humans; as such, it reasserts the presence and significance of the human within, or in relation to, the imperceptible dimensions of computation.
Non-human perception and algorithmic affordances
While much of the canonical scholarship on affordance is grounded within the ecological and material, it typically assumes that the realization and actualization of affordances hinge upon interactive relations established by (or at least in relation to) humans. This being said, researchers working on the development of ‘smart’ computational and robotic systems are increasingly instrumentalizing the affordance concept – understood as the ability to identify and make use of opportunities for action within a given environment (whether real or virtual) – in an effort to build technologies that are efficient, autonomous and responsive to “complex, unstable, and real-time environments” (Horton et al., 2012: 70). According to Horton et al., by formalising and instrumentalising the relationship between the agent and its environment, rather than the environment as such (70), agents are freed from:
… the need to maintain complex representations of the world. The agent can instead interact with the world as it is, allowing for more flexible and timelier responses in a dynamic environment, with the agent able to learn the affordances of its surroundings through first-hand exploration (79).
This affordance-based approach has been adopted by researchers developing a wide array of automated technologies, including autonomous driving vehicles (Chen et al., 2015); hand-like attachments for autonomous robotic systems (Saponaro et al., 2018); “artificial agents” capable of identifying “actionable” properties of an image (Chuang et al., 2017); and algorithms for determining “actions afforded by a scene” (Wang et al., 2017). Central to each of these projects are the predictive and probabilistic affordances of an ‘affordance-based’ approach. As Saponaro et al. (2018) explain with regards to robots working alongside humans: “A crucial ability needed by these robots to succeed in such environment (sic) is to be able to predict the effects of their own actions, or to give a reasonable estimate when they interact with objects that were never seen before” (1). For Chuang et al. (2017), an affordance-based approach does not only allow for a computer’s more seamless negotiation of the image-scape, but also enables the prediction of relationships between objects in the image (and, by extension, objects in the world). Similarly, for Wang et al. (2017), an analysis of figure placement within a scene (their data set is comprised of over 10 million stills from Sitcoms) sheds light on both the relationship between objects and environment and on the probability that the perceived relationship (affordance) be realized. In addition to augmenting systems’ ability to automatically negotiate complex environments, Pirk et al. (2017) hypothesize that the delineation of affordances might also provide indirect “insight into the semantic identity of the object,” again contributing to the development of increasingly ‘smart’ technologies.
As Thomson’s Deep Time Machine Learning suggests, machines can be equipped with a variety of sensorial and algorithmic means through which to discern and enact environmental affordances, without the intervention of human agents. This being said, at the forefront of much of this research (as the preceding examples of recent innovations suggest) are efforts to leverage technologies of machine vision as well as machine learning algorithms to automate the identification and actualization of affordances associated with visual data. In each of the cases discussed above, autonomous computational agents are being developed and trained to identify and respond to image-based affordances. This is to say that while the identification of affordances is corresponded (by the authors) to the actual environment, depicted in the images, the environment is in fact the pixelated and patterned landscape of the image file itself. As N. Katherine Hayles has detailed, a slippage occurs here between reality and abstraction, where an abstraction (the image-object) first stands-in for and is then mistaken for actual reality (Hayles, 1999). While the systems’ responsive actions may appear to attest to the affordances of the actual environment, the apparent coincidence is an indicator of the accuracy of the image-model, rather than being an indicator of the actual affordances of the environment (or the system’s ability to recognize them without mediation). What these examples reveal, therefore, relates to the affordances of pixel artefacts (a formalized material, in Kirschenbaum’s terms) and encoded pixel data, as well as of the machine vision algorithms that discern and attach relationally derived meaning to these coded artefacts.
According to Hoelzl & Marie (2015), digitization has resulted in a significant shift in the “photographic paradigm of the image” (100), from a representational landscape replete with signifying pictures to one comprised of algorithmically operationalized, collated, and (at times) visualized data sets. Following Harun Farocki, Hoelzl & Marie explain that digital images are no longer “visual entities, aimed at a human mind, but visual patterns recognized and interpreted by a computer” (101). As the authors articulate, what computational technologies recognize, read and aggregate are the sampled and quantized bits of information (datasets) that comprise and render digital images operative and actionable. When these technologies identify the affordances of an environment through the means of a digital image, what is realized is an algorithmically discerned pairing between patterns within the pixel data and encoded (or learned) criteria delimiting ‘opportunities for action.’ On the one hand, these ‘opportunities for action’ are rendered, as in the examples reviewed above, into actual behaviours, identifying a shift in the traditional ‘agent’ of affordance. No longer an organic being, algorithmic means of perception and discernment are increasingly adept at identifying and enacting environmental and object-oriented affordances. On the other hand, this situation also calls attention – once again – to the imperceptible unfolding of algorithmic, code- and bit-based affordances. While boyd (2010) provides an account of the material grounds and affordances of bits, these affordances cannot be computationally realized without the coinciding affordances of code and algorithm. How then, might we begin to parse the affordances of algorithms and, correspondingly, code?
The field of software studies has sought to expose the programmed undercurrents that enable and constrain computational processes, implicitly suggesting that the affordances of computation can be explained (at least in part) through a close reading of computer code and algorithms. Where ‘code’ here refers to the basic representational building blocks that comprise and structure programming languages, algorithms are the instructional means through which code is harnessed, “focalized and instantiated in a particular program, interface, or user experience” (Finn, 2018: 35). Reflecting on the resulting intersection between algorithm and affordance, Shintaro Miyazaki (featured in this issue) argues that “algorithms, when stored and not-yet-unfolded, have affordances, since they are made of instructions to structure and move hard-, soft- and wetware…” (n.p.). Whether executed or not, algorithms bear the capacity to “put things forth, forward or further” (n.p.); they possess the potential to enact a cascading series of relational actions. Similarly, David Gauthier (2018) has explained that algorithmic commands “request and constrain action to fulfil the promise of its execution which, in turn, should shed expected effects” (74). He explains that “the command itself does not act per se, but rather prescribes an action that it, in turn, assesses or judges (‘correct value’)” (ibid.). Both Miyazaki and Gauthier point here to the instrumentalization and subsequent representation of affordances within the algorithms that drive contemporary computation. To unearth and analyze these textual undercurrents is, then, to identify and begin parsing the structural affordances that are embedded within and enacted by computational systems. There are two critical implications here.
First, there is an understanding that much of this activity unfolds within purely computational environments, without experiential output.[7] Instead, algorithms often operate in recursive and inter-algorithmic feedback loops, executing and establishing connections between component parts of computation (e.g., bits of data, code, and programs) as well as between computational processes. Even outside of its specific formulation and operation, there is a manner in which this might be understood as one of the fundamental affordances of the algorithm, namely its actualization of connections between components and layers of computation. While “the command itself does not act,” it does enable and initiate – and therefore afford – relation.
Second, this sentiment reifies some of the basic principles of software studies, inevitably necessitating a critical examination of the broader apparatuses within which algorithmically encoded affordances are developed and deployed. As Norman promised, the action possibilities of contemporary computation have been severely restricted through the algorithmic encoding of predetermined affordances, many of which are designed in response to what programmers are able to recognize (or imagine) as possible uses for the system and broader apparatus, as well as in response to what is socio-politically and economically desirable. (This is discussed in greater detail by a number of authors featured in this issue, including Maximillian Alvarez, Özgün Eylül İşcen, and Vendela Grundell.)
Algorithms operate through coded means, and therefore leverage the affordances of code. As noted, computer code abstracts and ascribes representational signs – language – to the messy realities of hardware and software operations. While code helps to establish the relational means through which users interact with and attempt to harness the capabilities of computing, it is also that which fundamentally delimits the expansive potential of computation. “By isolating, stratifying, discretising, categorizing and foreclosing the spatiotemporal continuum the process of execution articulates” (Gauthier, 2018: 81), code erases users’ perception of the messiness of electronic processing and slippages between what the code and its symbolic extensions stipulate and what actually occurs (72).
While Chun (2011) has critiqued the code-enabled desire to erase execution, a gesture that coincides with the earlier identification of invisibility and imperceptibility as affordances unto themselves, we might also understand the capacity for code to interface with and translate between the electronic and textual operations of computation as one of the critical affordances that code enables. It is fundamentally through the execution of coded signifiers that the ‘action possibilities’ of computation, from the level of machine language to the flickering signifiers appearing on our screens (Hayles, 1999) and back again, are realized. This is not to say that code and its execution, or that code and activated hardware, are synonymous; as articulated above, code provides limited insight into the actual material operations of computer hardware. Nor is this a repeated call to access and read code in an effort to identify the particular affordances that are embedded within the language that drives computational systems. Instead, at stake in this case appears to be an understanding of the affordances of actionable signs – the affordances of executable language and representational artefacts. Ed Finn (2018) has begun to map an account of actionable signs in his consideration of the intersection between code and magic. He argues that the execution of code actualizes long-held cultural beliefs concerning the “mythic power of language and the incantatory magic of words” (196). Reflecting on computation, he explains, code is comprised of symbols that can be manipulated and executed in a manner that does not simply abstract, represent and produce meaning about the physical world, but that also has a ‘real’ impact on the physical world. This magical enactment of actionable signs does not only suggest the culmination and closure of the perceived gap between representation and reality, insofar as language is no longer restricted exclusively to the realm of representation, but it necessitates a close examination of the role of affordance in how computers, at the lowest levels, navigate the bi-directional gap between electronic instantiation and abstraction.
Art and affordance
The complexities and emerging nuances of affordance theory in digital and algorithmic contexts find expression not only in recent theoretical work, as elaborated in the preceding sections, but also, as we identified in the introduction, in the historical and contemporary work of media artists. Artists’ access to emerging technologies has always informed the development of industrial, scientific, and commercial applications of these technologies. While significant scholarship has demonstrated how early access to computational technologies and industry shaped the foundations of many contemporary art movements, such as Performance and Conceptual Art (e.g., Cook, 2016; Shanken, 2015), such accounts often overlook the corresponding contributions that artists have made to the perceived affordances and discursive constitution of emerging technologies (e.g., Noll, 2016; Patterson, 2015; Kane, 2014). Several of the contributions featured in this issue explore and elaborate precisely such connections and seek to emphasize the importance of affordance for current discourses on the organisation and control of artists’ access to, use of, and experimentation with emerging digital technologies (see for example: Guillermet, Marcinkowski, and Lotti, this issue).
As these entries suggest, many contemporary artists continue to probe emerging technologies in experimental work that helps recognize, expand, and ultimately rethink the affordances linked to these technologies. Often, this kind of work also takes place outside of institutional contexts and follows approaches that might be more aligned with hacker ethics (see Cox, 2010; 2012), critical engineering (Oliver et al., 2019), or other alternative attitudes towards the appropriative use of emerging technologies. As such, many works of media art can be read as critically engaging, directly or obliquely, with Gibson’s and Norman’s perspectives on the affordance concept, and as significantly problematising and expanding these perspectives along some of the conceptual and theoretical vectors outlined above.
Representative examples of artworks that highlight some of the critical positions we outline in this essay were included in the group exhibition that stood at the beginning of the ‘Rethinking Affordance’ project (Fig. 3).[8] Aside from Jol Thomson’s Deep Time Machine Learning, discussed above, additional works shown in the exhibition included, for example, _white paper (2018) by FRAUD (aka Francisco Gallardo and Audrey Samson), and Ways of Sitting (2018) by Foci+Loci (aka Chris Burke and Tamara Yadao). In Ways of Sitting, the New York City-based duo Foci+Loci place digitally rendered Duchampian ‘readymades’ in the responsive environment of the Sony-produced video game Little Big Planet 3, where players’ interactions trigger the emergence of new sets of affordances of this commercial, proprietary software (see Fig. 4).[9] Rather than ‘hacking’ the game, the artists take advantage of functionality that has been designed by the game developers, but which was never meant to take shape in the form of critical, experimental, or performative media art work. Foci+Loci, in other words, realize algorithmic affordances that may arguably have remained imperceptible to the game developers, even though they were purposefully built into a popular ‘participative’ game that relies heavily on the player’s provision of user-generated content. In this and other works by Foci+Loci, it becomes apparent that even purpose-built, rule-driven digital artefacts such as video games – which tend to strictly limit users’ powers while offering them a simulated sense of interactive freedom – afford wide-ranging critical, alternative, and creative uses that are not predetermined in Norman’s sense, but which correspond to Nagy and Neff’s framework of ‘imagined affordances’ (2015).
FRAUD’s _white paper (see Fig. 5), consisting of a series of seemingly white posters and print-outs (in fact, white ink was used on white paper stock), is an extension of a cryptocurrency art project (Indulgence Coin) which the artists developed in collaboration with Guido Rudolphi. In preparing for launching the ICO (Initial Coin Offering) for that project, the artists had begun to question the affordances of the white paper as a specific type of information document, while simultaneously starting to explore the affordances of white paper as a medium through which their critique of this particular document type could be articulated. The white paper is today very prominently used within the speculative domain of emerging crypto-economies – a domain which often relies on ideologically problematic ways on the kinds of invisibility and imperceptibility discussed above. As a document type, the white paper is here meant to emblematize rigour, transparency, and extensive descriptions of business plans, technical platforms, or other details related to a crypto venture. However, as has become clear in the countless crypto-scams that continue to populate the Internet, the white paper can also function as a facade that is meant to point to a larger and deeper ‘truth’ beyond itself, which the reader is never granted full access to. Here, the important affordances of invisibility, as discussed above with reference to Parikka (2015), Hoelzl & Marie (2015), and others, come into play. The tendency to hide functional, economic, or ideological issues of a project in a type of document that is by definition meant to fully lay bare the system to which it speaks has gone so far that boilerplate web pages advertising new crypto initiatives now sometimes only announce white papers, rather than actually making them available.[10] As such, the white paper can, in fact, function as a kind of blackbox. Expanding on this, FRAUD rethink the affordances of the medium of the white paper, and of written text more generally, whether in analogue or digital form. In the form in which the work was exhibited at the ‘Rethinking Affordance’ exhibition, it maps the affordances of the white paper across a wide range of contexts that reach back from current crypto contexts to earlier forms such as government declarations and public announcements of cultural, economic, and other types of official policy. The resulting sculptural interventions call attention to the ‘invisible’ ideological undergirding established through these seemingly innocuous documents that exist to announce or introduce, under the guise of transparency, preliminary positions while also projecting surety and finality. This, again, offers interesting conceptual counterpoints to both Gibson’s and Norman’s perspectives on affordance, in considering how the infrastructural code layers of the white paper, approached here as both document type and medium, can be recast for critical purposes.[11]
After this brief consideration of how contemporary media artists engage with and rework the concept of affordance, we will now conclude with a brief summary of the contributions to this special issue, many of which go into considerably more detail in their critical exploration of how artists continue to recuperate and expand the affordances of the media substrates, technical specificities, and ideological implications of the technological environments they inhabit.
Overview of the special issue contributions
As noted, the lines of inquiry developed in this special issue seek to revisit the discontinuities of affordance theory and to recuperate ‘affordance’ in ways that can productively engage the dynamic malleability of the digital. Since the concept of affordance is by definition located between design and implementation, between environment and user, we are particularly interested in approaches that bridge or combine theoretical and practical approaches. In developing the larger project that led to this special issue, it was our observation that the (dis-)continuities between established discourses on affordance and the ways in which the concept is currently deployed are poorly understood and require critical attention. The contributions to this special issue begin to fill this gap in contemporary media theoretical criticism.
Olia Lialina’s contribution, which is based on the author’s keynote lecture at the 2018 Rethinking Affordance symposium, offers a comprehensive survey of the tensions between Gibson’s and Norman’s perspectives on the concept of affordance, and formulates an incisive critique of how Norman reconfigured Gibson’s initial theory. Triangulating her inquiry in a critical dialogue between design practitioners, affordance theory, and a critical reading of design pedagogy, and the revisiting of her own practice as a pioneering net artist and digital folklore researcher, Lialina’s contribution moves from early internet design practices through human-computer interaction and user experience design towards a speculative consideration of the affordances of human-robotic interaction.
Leveraging the terms of critical pedagogy, Maximillian Alvarez critiques and disassembles the supposed affordances that digital technologies lend to the learning environment and advances – in their place – an account of ‘critical media pedagogy.’ For Alvarez, this is a project with ontological implications. Following the work of Bernard Stiegler, Alvarez explains that the epiphylogenesis of the human is inescapably imbricated with the technological. Contemporary neoliberal pressures treat learning technologies as tools that constrain and compel particular behaviours based on what developers determine appropriate learning and teaching to be. This does not only limit the potential for learning in accordance with critical pedagogy, but it also obfuscates individuals’ capacity to form a critical understanding of the grounds for contemporary technical life and, more fundamentally, the technological conditions of possibility through which the human comes into being. Alvarez’s critical media pedagogy works to undo this tendency by exploring what digital media can do for – might afford – critical pedagogy and vice versa.
In a comparative reading of the affordance concept across a wide range of critical theorists – from Gibson to Foucault, Deleuze, Galloway, Debord, and beyond – Torsten Andreasen rethinks key terms of media theory (the medium, the interface, the dispositif) and applies his insights to the close analysis of an interactive media artwork. The author’s discussion of Transmute Collective’s Intimate Transactions (2005) thus problematizes established assumptions that “where the medium affords [certain uses] because of its physical design, the dispositif determines and limits a set of possible actions.”
Aline Guillermet’s contribution traces German Informel painter K. O. Götz’s efforts to identify and implement the affordances of television in his endeavour to realize the historical promise of ‘kinetic painting’ – which he believed would help bridge the demands of painterly modernism with the encroaching rise of information theory and corresponding electronic technologies. Götz did not have access to an actual television set and was therefore left to imagine its presumed affordances. Guillermet analyzes, as a result, both the effects that the imagined affordances of television had on Götz’s creative activity and historical milieu as well as the possibility that technological affordances be conceived of as offering a ‘flexible paradigm,’ grounded within the terms of interpretation and subjective meaning.
Charting a trajectory from J.J. Gibson’s initial theoretical writing on affordance to Manuel DeLanda’s theory of assemblages, Michael Marcinkowski explores the concept of digital ‘ambient literature’ projects in relation to the social assemblages that can be established by new media art installations and the interactional affordances they project. In doing so, the author calls for a reconfiguration of ontological assumptions regarding the function of the affordance concept in digital contexts of experimental literary production.
Vendela Grundell undertakes an analysis of works and practices associated with the Blind Photography movement to expose the tactical means through which visually impaired photographers press up against and push beyond the presumed limits of technological affordances. Grundell argues photographers aligned with the movement identify and implement a series of tactical affordances through their creative practice and within their images. Not only do the photographers surveyed identify counter-intuitive and unexpected uses for visually-oriented technologies, but their images visualize alternative ways of seeing the world and photography, alluding to the manner in which technologies normalize particular ways of engaging with and thinking about reality.
In Mark Nunes’ contribution, the affordances of digital technologies – specifically the location-awareness of mobile apps – is explored in order to account for what the author considers as fundamentally different agencies at play in the interactions that many mobile apps facilitate. While digital technologies often serve to facilitate a “relational coupling” between user and device, a user’s presence and activities are themselves indicative of the emergence of new affordances. Drawing on actor-network theory as a main conceptual framework, Nunes argues that technologies such as GPS, and the large scale data analysis processes carried out by always-on apps, require a new perspective on digital affordances, one in which human users themselves become ‘interfaces’ that mediate between algorithmic processes and the physical environments they navigate.
Özgün Eylül Iscen leverages a historical and theoretical examination of the racialized, racializing and ultimately dehumanizing technologies of drone warfare to call for a critical reconsideration of the affordance concept. Iscen works to expose the political pressures and privileges that often lurk behind the professed affordances of particular technologies, while also charting the particular ways in which this is made manifest through drones’ affording particular players ‘the right to look.’ Iscen illustrates these principles and points towards strategies of critique and resistance through an introduction to the work of artist-collective Forensic Architecture.
Following a speculative philosophical approach, Shintaro Miyazaki’s essay critiques the blackboxing of many algorithmic processes, which the author perceives as resulting in a kind of ‘unaffordability’ of algorithms. Engaging with current theoretical debates on ‘commonism,’ Miyazaki offers a speculative formulation of commonistic affordance and, taking into consideration issues of access and open source, explores steps towards a ‘making affordable’ of algorithms that emphasizes commoning rather than corporate propertization.
Exploring some new affordances of the complex algorithmic systems that form what is now commonly described as ‘financial technologies,’ Laura Lotti focuses on the recent phenomenon of ‘tokenization’ within the cryptosphere; i.e., the issuance of new crypto assets to self-fund decentralized projects. Integrating ongoing critical debates in the field with a Simondonian reading of decentralized computation, Lotti discusses the rampant financialization of creative practices presently observed in blockchain contexts. Through the examples of two blockchain-based art projects (terra0 and 0xΩ), Lotti analyses new forms of value generation and distribution and argues that various instrumentalisations of blockchain affordances open up ways of reimagining and reprogramming financial and social relations in contexts of decentralized computation.
Acknowledgments
We would like to extend our sincerest thanks to the reviewers and contributors for the generosity of spirit and hard work that they poured into the production of this issue. We would also like to thank Akademie Schloss Solitude and Simon Dawes for their patience and support in realizing this multifaceted project.
References
Bernhard, E. et al. (2013) “Understanding the Actualization of Affordances: A Study in the Process Modeling Context.” Proceedings for International Conference on Information Systems (ICIS 2013), 15-18 December 2013, Università Bocconi, Milan. Access: http://eprints.qut.edu.au/63052/
Best, K. (2009) “Invalid Command,” Information, Communication & Society. Vol. 12(7): 1015 – 1040.
Bloomfield, B. et al. (2010) “Bodies, Technologies and Action Possibilities: When is an Affordance?” Sociology. Vol. 44(3): 415 – 433.
boyd, d. (2011) “Social Network Sites as Networked Publics: Affordances, Dynamics, and Implications,” in Networked Self: Identity, Community, and Culture on Social Network Sites. Ed. Zizi Papacharissi. Routledge: 39-58.
Carah, N. & Angus, D. (2018) “Algorithmic Brand Culture: Participatory Labour, Machine Learning and Branding on Social Media,” Media, Culture & Society. Vol. 40(2): 178 – 194.
Chamayou, G. (2015) A Theory of a Drone. New York: The New Press.
Chen, C.et al. (2015) “DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving,” Proceedings of 15th IEEE International Conference on Computer Vision. Access: http://deepdriving.cs.princeton.edu/
Chuang, C.Y. et al. (2017) “Learning to Act Properly: Predicting and Explaining Affordances from Images,” arXiv. Access: https://arxiv.org/abs/1712.07576
Chun, W. H. K. (2016) Habitual Media: Updating to Remain the Same, Cambridge, MA: MIT Press.
Chun, W.H.K. (2006) Control and Freedom: Power and Paranoia in the Age of Fiber Optics. Cambridge, MA: MIT Press.
Cook, L. (2016) Information. Cambridge: MIT Press.
Costa, E. (2018) “Affordances-in-practices; An ethnographic critique of social media logic and context collapse,” New Media & Society. Vol. 20(10): 3641 – 3656.
Cox, G. (2010) Antithesis: The Dialectics of Software Art. Aarhus: Digital Aesthetics Research Center, Aarhus University. http://www.anti-thesis.net/wp-content/uploads/2010/01/antithesis.pdf
Cox, G. (2012) Speaking Code: Coding as Aesthetic and Political Expression. Cambridge, MA: MIT Press.
Davis, J.L. & Chouinard, J.B. (2017) “Theorizing Affordances: From Request to Refuse,” Bulletin of Science, Technology & Science. Vol. 36(4): 241 – 248.
Diver, L. (2018) “Law as a User: Design, Affordance, and the Technological Mediation of Norms,” scripted. Vol. 15(1): 4 – 48.
Deleuze, G. (1992) “Postscript on the Societies of Control,” October. Vol. 59: 3 – 7.
Drucker, J. (2014) Graphesis: Visual Forms of Knowledge Production. Cambridge: Harvard University Press.
Evans et al. (2017) “Explicating Affordances: A Conceptual Framework for Understanding Affordances in Communication Research,” Journal of Computer-Media Communication. Vol. 22: 35 – 52.
Eslami, M. et al. (2015) “I always assumed that I wasn’t really that close to [her]”: Reasoning about Invisible Algorithms in News Feeds.” Proceedings of the 2015 Annual Conference on Human Factors in Computing Systems. Seoul, Korea:153 – 162.
Faraj, S., & Azad, B. (2013) “The Materiality of Technology: An Affordance Perspective,” in Materiality and Organizing. P. M. Leonardi, B. A. Nardi, & J. Kallinikos (Eds.), Oxford University Press: 237–258.
Finn, E. (2017) What Algorithms Want: Imagination in the Age of Computing. Cambridge: MIT Press.
Flach, J.M. & Dominguez, C.O. (1995) “USE – Centered Design: Integrating the User, Instrument, and Goal,” Ergonomics in Design: The Quarterly of Human Factors Applications. Vol. 3(1): 19 – 24.
Gabrys, J. (2016) Program Earth: Environmental Sensing Technology and the Making of a Computational Planet Minneapolis: University of Minnesota Press.
Galloway, A. (2011) “Are Some Things Unrepresentable?” Theory, Culture & Society. Vol. 28(7-8): 85-102.
Galloway, A. (2006) Gaming: Essays on Algorithmic Culture, Minneapolis: University of Minnesota Press.
Gaver, W. (1991) “Technology Affordances,” in Proceedings of CHI’91: 79-84.
Gauthier, D. (2018) “On Commands and Executions: Tyrants, Spectres and Vagabonds,” in Executing Practices. Eds. H. Pritchard et al. Open Humanities Press: 69-84.
Gero, J.S. & Kannengiesser, U. (2012) “Representational Affordances in Design, with Examples from Analogy Making and Optimization,” Research in Engineering Design. Vol. 23: 235 – 249.
Gibson, J.J. (1966) The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin.
Gibson, J.J. (1975) “Affordances and Behavior” in E. S. Reed & R. Jones (eds.), Reasons for Realism: Selected Essays of James J. Gibson, pp. 410-411. Hillsdale, NJ: Lawrence Erlbaum.
Gibson, J.J. (1979) The Ecological Approach to Visual Perception. London: Routledge.
Gillespie T. (2010) “The Politics of “Platforms,” New Media & Society. Vol. 12: 347–364.
Graves, L. (2007) “The Affordances of Blogging: A Case Study in Culture and Technological Effects,” Journal of Communication Inquiry. Vol.31(4): 331 – 346.
Greeno, J.G. (1994) “Gibson’s Affordances” Psychological Review Vol. 101(2): 336-342.
Hand, M. & Scarlett, A. (2019) “Habitual Photography: Time, Rhythm and Temporalization in Contemporary Personal Photography,” in The Routledge Companion to Photography Theory. Eds. Tormey, J. & Durden, M. London: Routledge.
Hansen, M. (2015) Feed-Forward: On the Future of Twenty-First Century Media. Chicago: University of Chicago Press.
Hartson, R. (2003) “Cognitive, Physical, Sensory, and Functional Affordances in Interaction Design,” Behaviour & Information Technology. Vol. 22(5): 315-338.
Haugeland, J. (1993) “Mind Embodied and Embedded,” Mind and Cognition: 1993 International Symposium. Eds. Yu-Houng H. Houng & J. Ho. Academica Sinica: 233 – 267.
Hayles, N.K. (1999) How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago: University of Chicago Press.
Heemsbergen, L. (2019) “Killing Secrets from Panama to Paradise: Understanding the ICIJ through Bifurcating Communicative and Political Affordances,” New Media & Society. Vol. 21(3):693–711.
Hoelzl, I. & R. Marie (2015) Softimage: Towards a New Theory of the Digital Image. Chicago, Ill.: Univ. of Chicago Press.
Horton, T.E. et al. (2012) “Affordances for Robots: A Brief Survey,” AVANT. Vol. 3, No. 2:70 – 84.
Hurley, Z. (2019) “Imagined Affordances of Instagram and the Fantastical Authenticity of Female Gulf-Arab Social Media Influencers,” Social Media + Society. January/March: 1 – 16.
Hutchby, I. (2001) Conversation and Technology: From the Telephone to the Internet. Cambridge: Polity Press.
Kane, C. (2014) Chromatic Algorithms: Synthetic Color, Computer Art, and Aesthetics After Code. Chicago: University of Chicago Press.
Kirschenbaum, M. (2012) Mechanisms: New Media and the Forensic Imagination. Cambridge: MIT Press.
Klemme, H.F. & Kuehn, M. (2016) The Bloomsbury Dictionary of Eighteenth-Century German Philosophers. London: Bloomsbury.
Latour, B. (1999) Pandora’s Hope: Essays on the Reality of Science Studies. Cambridge: Harvard University Press.
Leonardi, P.M. (2011) “When Flexible Routines Meet Flexible Technologies: Affordance, Constraints, and the Imbrication of Human and Material Agencies,” MIS Quarterly. Vol. 35, No. 1, Pp. 147 – 167.
Letich, H. & Lissack, Ml. (2009) “Making Room for Affordance,” E:CO. Vol. 11(3) :61 – 72.
Lippard, L. (1972) Six Years: The Dematerialization of the Art Object. Berkeley: University of California Press.
Lister, M. et al. (2009) New Media: A Critical Introduction. New York: Taylor & Francis.
Lovink, G. (2014) “Hermes on the Hudson: Notes on Media Theory After Snowden,” eflux. Vol. 54.
Manovich, L. (2001) The Language of New Media. Cambridge: MIT Press.
Massumi, B. (2015) Ontopower: War, Powers and the State of Perception. Durham: Duke University Press.
McGrenere, J. and Ho, W. (2000) “Affordances: Clarifying and Evolving a Concept,” in Proceedings of Graphics Interface: 179-186.
McVeigh-Schultz, J. & Baym, N.K. (2015) “Thinking of You: Vernacular Affordance in the Context of the Microsocial Relationship App, Couple,” Social Media + Society, https://doi.org/10.1177/2056305115604649
Mitchell, W.J.T. & Hansen, M. (2010) Critical Terms for Media Studies. Chicago: University of Chicago Press.
Moloney, J. et al. (2018) “The Affordances of Virtual Reality to Enable the Sensory Representation of Multi-Dimensional Data for Immersive Analytics: From Experience to Insight,” Big Data. Vol. 5(53): 1-19.
Morineau, T. et al. (2009) “Turing Machine as Ecological Model for Task Analysis,” Theoretical Issues in Ergonomics Science. Vol. 10(6): 511 – 529.
Nagy, P. & Neff, G. (2015) “Imagined Affordance: Reconstructing a Keyword for Communication Theory,” Social Media + Society. July-December 2015: 1 – 9.
Neff G., J.T., McVeigh-Schultz J., Gillespie, T. (2012) “Affordances, Technical Agency, and the Politics of Technologies of Cultural Production,” Journal of Broadcasting & Electronic Media, Vol. 56: 299–313.
Noll, A. M. (2016) “Early Digital Computer Art at Bell Telephone Laboratories, Incorporate,” Leonardo, Vol. 49(1): 55 – 65.
Norman, D. (1988) The Psychology of Everyday Things. New York: Basic Books.
Norman, D. (1999) “Affordance, Conventions and Design,” Interactions 6(3): 38-43.
Norman, D. (2009) The Design of Future Things. Philadelphia: Perseus Books/Basic Books.
Norman, D. (2013) The Design of Everyday Things: Revised and Expanded Edition. New York: Basic Books.
Oliver, J., G. Savičić and D. Vasiliev (2011-2019) Critical Engineering Manifesto. https://criticalengineering.org/
Oliver, M. (2005) “The Problem with Affordance,” E-Learning and Digital Media, Vol. 2(4): 402-413
Patterson, Z. (2015) Peripheral Vision: Bell Labs, the S-C 4020, and the Origins of Computer Art. Cambridge: MIT Press.
Parisi, L. (2013) Contagious Architecture: Computation, Aesthetics and Space. Cambridge: MIT Press.
Park, S. & Rhee, O.J. (2013) “Affordance in Interactive Media Art Exhibition,” International Journal of Asia Digital Art Design: 93-99.
Parikka, J. (2015) “The City and the City: London 2012 Visual (Un)Commons,” Postdigital Aesthetics: Art, Computation and Design. Eds. David M. Berry & Michael Dieter. New York: Palgrave Macmillan: 203 – 218
Pressman, R. & Maxim, B. (2014) Software Engineering: A Practitioner’s Approach (8th ed.). McGraw-Hill Education.
Pucillo F. & Cascini, G. (2013) “A Framework for User Experience, Needs and Affordances,” Design Studies 35(2): 160-179.
Pozzi, G. et al (2014) “Affordance Theory in the IS Discipline: A Review and Synthesis of the Literature.” Proceedings for Twentieth Americas Conference on Information Systems.
Ruecker et al. (2011) Visual Interface Design for Digital Cultural Heritage: A Guide to Rich Prospect Browsing. Surrey: Ashgate Publishing.
Saponaro, G. et al. (2018) “Learning at the Ends: From Hand to Tool Affordances in Humanoid Robots.” Proceedings: IEEE International Conference on Development and Learning and on Epigenetic Robotics. Access: https://arxiv.org/abs/1804.03022
Scarlett, A. (2018). “Realizing Affordance in the Techno-Industrial Artist Residency,” Schloss-Post.com. Access: https://schloss-post.com/realizing-affordance-techno-industrial-artist-residency/
Schrock, A.R. (2015) “Communicative Affordances of Mobile Media: Portability, Availability, Locatability, and Multimediality,” International Journal of Communication. Vol. 9: 1229 – 1246.
Shanken, E. (2015) Systems. Cambridge: MIT Press.
Shaw, A. (2017) “Encoding and Decoding Affordances: Stuart Hall and Interactive Media Technologies,” Media Culture & Society. Vol 39(4): 592 – 602.
Sloterdijk, P. (2010) “Das Zeug zur Macht,” in Der Welt Über die Straße Helfen. Designstudien im Anschluss an Eine Philosophische Überlegung. Munich: Fink: 7-26.
Smith, B.C. (1996) On the Origin of Objects. Cambridge: MIT Press.
Soegaard, M. (n.d.) “Affordances,” in The Glossary of Human Computer Interaction. Access: https://www.interaction-design.org/literature/book/the-glossary-of-human-computer-interaction/affordances
Stone et al. (2005) User Interface Design and Evaluation. New York: Elsevier.
Striphas, T. (2015) “Algorithmic Culture,” European Journal of Cultural Studies. Vol. 18(4-5).
Sutcliffe, A.G. et al. (2011) “Social Mediating Technologies: Social Affordances and Functionalities,” International Journal of Human-Computer Interaction. Vol. 27(11): 1037 – 1065.
Von Borries, F. (2016) Weltentwerfen. Eine Politische Designtheorie. Berlin: Suhrkamp.
Wang, Xiaolong et al. (2017) “Binge Watching: Scaling Affordance Learning from Sitcoms.” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR): Access:
http://www.cs.cmu.edu/~xiaolonw/papers/CVPR_2017_VAE_affordance.pdf
Wells, A. J. (2002) “Gibson’s Affordances and Turing’s Theory of Computation,” Ecological Psychology, Vol. 14(3): 140 – 180.
Zeilinger, M. (2018) “Plotting Critical Research Practice in Digital Art” in Golding, S. (Ed.) Parsing Digital, London: Austrian Cultural Forum: 22-37.
Zielinski, S. (2008) Deep Time of the Media: Toward an Archaeology of Hearing and Seeing by Technical Means. Cambridge: MIT Press.
Notes
[1] Video excerpts from the piece, showing details from the apparatuses and their interaction, can be found as part of a short text the artist contributed to an online collection developed as part of the larger Rethinking Affordance project. See https://schloss-post.com/rotating-divinatory-hexagrams.
[2] Beyond the conjoining of Hahn’s mechanical calculator with the Bosch robotic arm, Deep Time Machine Learning also thematizes other aspects of how to make the computational (i.e., machine vision or algorithmic operations) human-legible, and how, in turn, to make human expression computable. In doing so, the work extends its backwards-and-forwards reach into two additional directions not addressed here, which are represented, respectively, by a Faustkeil (a paleolithic hand-axe) that features in the form of an ultra-high resolution 3D print, and by the European Union’s tentative steps towards the issuing of policy and ethics directives at the intersection of humanity and AI.
[3] In Chapter 8 of this latter book, Gibson outlines a kind of ‘pre-history’ of his affordance concept, which references Gestalt theory and foundational theories of the psychology of perception.
[4] In addition to this theoretical oversight, Norman’s call to streamline and ensure ‘correct’ use through the communicative clarity of design also encourages a narrowing of both the real and perceived affordances that are (or might be) realizable through the use of digital devices and operations. Olia Lialina (this issue) responds critically to Norman’s approach here, reminding us that, unlike physical objects and environments, the digital is theoretically capable of modelling anything, adopting processually malleable and aesthetically unprecedented forms. Norman, she argues, fails to appropriately recognize these possibilities, and encourages designers to actively hide them for the sake of controlled usability.
[5] Echoing Schrock’s (2015) conceptualization of ‘communicative affordances’, boyd defines networked publics as “publics that are restructured by networked technologies” (39). They are, as a result, “simultaneously (1) the space constructed through networked technologies and (2) the imagined collective that emerges as a result of the intersection of people, technology, and practice” (ibid.).
[6] As Shaw (2017) has noted in her consideration of similarities between the recognition of the affordances offered by interactive media technologies and Stuart Hall’s theorization of encoding and decoding, “by introducing imagination to affordances … [Nagy & Neff] also acknowledge that there are aspects of mediated experiences that are invisible to users. Algorithms, for instance, affect what users can and cannot do in online spaces, but operate out of view” (600).
[7] While it is critical to parse the ideological grounds of algorithmic affordances, it is important to acknowledge that much of what unfolds algorithmically eludes understanding – even to those who author them (Finn, 2018: 35).
[8] The event program, including a full list of participating artists and researchers, can be found at http://www.akademie-solitude.de/en/events/symposium-rethinking-affordance~no3926/.
[9] Additional documentation of the work is available at https://vimeo.com/266917634 and https://schloss-post.com/ways-sitting-wip/.
[10] This is the case, for example, for Scriptocoin, a “Crypto-Pharma Ecosystem Built on Blockchain” promising to “Permanently Revolutionize the Pharmaceutical Industry Paradigm.” At the time of writing, a ‘Token Sale Pre-ICO,’ i.e., a sale of project shares taking place before the cryptocurrency powering the project is actually deployed, is underway, but the platform’s live link to its white paper literally just leads to white paper – a PDF that simply states, “white paper Coming Soon.”
[11] Additional contributions to the exhibition, not discussed in detail here, also included: an installation by Situated Systems (Sherri Wasserman/US, Georgina Voss/UK, Debbie Chachra/CAN/US, and Ingrid Burrington/US) documenting the outcome of the artists’ stay at the Pier 9 Artist-in-Residence (AiR) program, which they spent researching and analysing how the military-industrial complex has shaped technological culture and innovation emerging from the Bay Area; German artist Sebastian Schmieg’s I Will Say Whatever You Want In Front Of A Pizza (2017), a critical exploration of new types of labour exploitation afforded by digital ‘gig economy’ platforms, which takes the form of a video essay produced entirely within Prezi (a web-based presentation tool); Bryan Cera’s Prosumption and Alienation (2018), a series of ceramic tea cups created on a custom-built 3D printer; and Martin Zeilinger’s Iterative Schotter (2017), a series of potter prints which explores how early computer art engaged with the affordances of new technologies, following an approach of iterative recording reproductions of Georg Nees’ influential generative art work, Schotter (ca. 1965).
Ashley Scarlett is an Assistant Professor in Critical and Creative Studies at the Alberta University of the Arts, Canada.
Email: ashley.scarlett@auarts.ca
Martin Zeilinger is Senior Lecturer in Computational Arts and Technology at Abertay University, UK.
Email: m.zeilinger@abertay.ac.uk
This article is from the special issue, Rethinking Affordance (Media Theory 3.1), edited by Ashley Scarlett and Martin Zeilinger.
The official version of record is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/article/view/78
The full issue is available here: http://journalcontent.mediatheoryjournal.org/index.php/mt/issue/view/4
Leave a Reply