Theremin for the Deaf
Imagine a world without acoustics: The beauty of Bach’s sonatas would never have graced the ears of captive audiences through the past centuries, and the spectacular Lady Gaga would likely still be known simply as Stefani Germanotta — if her name was known at all. Without sound waves there would be no music — at least that has been the prevailing opinion thus far. But for around 500,000 people worldwide (EURO-CIU 2017), hearing is not bound to acoustics. These people wear a cochlear implant (CI), a hearing aid implanted in the inner ear of the auditory system which makes it possible to hear more than just Lady Gaga and Bach. Although the function of the CI can and has been explained, attempting to show that the perception of a defined sound can be conveyed with the absence sound waves remains unexplored. Yet, this type of hearing aesthetics is an integral part of the CI user’s daily life; they are not only able to receive acoustic signals, but furthermore have access to the electrical and electromagnetic environment. In 2018, a project titled “Theremin for the Deaf” was conducted in Berlin, Germany which explored both theoretical and practical consequences of CI user’s experience of sound without acoustics. By pairing those implications with a Theremin — a musical instrument that does not necessarily requires acoustics — the project aimed to provoke a reassessment of the relationship between acoustics and ear. Bypassing the auditory link in this way creates the problem of understanding of hearing in a new way which is no longer dependent on the irritation of the ear with air pressure. Finally, the “Theremin for the Deaf” turns a hearing prosthesis into a hearing extension by showing the media technology’s actual potential. This reconceptualization of hearing is the subject which is explored in this article.
Structurally, this article is separated into two parts. The first section will open by providing a more thorough background on the development of the cochlear implant and its ability to bypass inner-ear mechanics. Finally, the article culminates with an application of these ideas on the questions posed by the project “Theremin for the Deaf,” followed by concluding remarks.
In common linguistic usage, hearing is firmly linked to acoustics: the rhythmic pattern of air that irritates our eardrum and ultimately stimulates the auditory nerve through mechanical processes in the middle and inner ear. Although information and communication theorist Warren Weaver defined the brain as the target of messages as early as 1949, he also linked the ‘mechanical parts’ of the ear with the nervous system: “When I talk to you, my brain is the information source, yours the destination; my vocal system is the transmitter, and your ear and the associated eighth nerve is the receiver” (1964, p. 7). But centuries earlier in 1780, Luigi Galvani had already epistemically broken the supposed connection between ear and acoustics with his frog galvanoscope, a method to prove the presence of electricity (Miyazaki 2016, p. 128). Galvani’s discovery of bioelectrical sensitivity, that is, the stimulation of nerve cells by electricity, also inspired Alessandro Volta to conduct a self-experiment in which he stimulated his ears using a voltaic pile (ibid.). As a result, Volta heard an audible event that was mediated without acoustics — thus initiating the history of the cochlear implant (Ochsner & Stock 2014, p. 409).
According to the media scientist Shintaro Miyazaki, the cochlear implant (CI) “is a technical ensemble that receives acoustic signals by a microphone, stores them in electronics, transmits them, processes them, and finally sends them to the implanted electrode in the cochlea as a sequence of bioelectric signals” (2016, p. 128). However, the sound processor located behind the ear of the CI user does not only process acoustic signals picked up via microphone to enable auditory participation with the social and cultural environment. Rather, this miniaturized computer also processes electrical and electromagnetic data, which uses the fast Fourier transformation and other algorithms to organize the electrodes implanted in the cochlea, in order that the acoustic liberated signals are audible to the CI user. For instance, modern CI’s can be connected to a smartphone or other media devices which are Bluetooth®-capable. Consequently, the CI bypasses the mechanical part of the ear — the eardrum, ossicles (malleus, incus, and stapes) and cochlea — and enables hearing without acoustics.
Miyazaki describes the CI as a technologically advanced hearing system which bridges the dysfunctional hair cells using implanted electrodes and ensures hearing by stimulating the intact auditory nerve fibers (2016, p. 126). Through a sound processor located on the outside of the head, incoming signals are processed and transmitted via an implanted magnetic coil to up to 22 independently addressable microelectrodes which are distributed along the basilar membrane to guarantee the highest possible frequency range (ibid.). As a result, the sound processor organizes an audible sound that uses algorithms to “physically address the correct electrodes in an appropriate manner” allowing the user to “understand the electrical signals … that are otherwise meaningless to the auditory nerve” (Miyazaki 2013, p. 8). Therefore, the CI user would indeed hear a sound without algorithms — as Volta had described it in his self-experiment — but only the correct order of operations in the CI-processor, i.e. the correct sequence of bioelectric signals, meticulously controlled by algorithms, forms the sound.
In this way, the sound provoked by the CI differs from that of the sonic which is, as described by the musicologist Peter Wicke, the combination of acoustics and the cultural-historical dimension (2016). Even if the CI user is able to perceive this ‘culturalized sound’ via the microphone, the CI’s algorithms first intervene in the shaping of the sound. Therefore, it is not primarily the cultural practice that shapes the sound. It is rather the bit strings that are converted into bioelectrical signals in the CI’s processor. Consequently, the sound is not bound to the acoustic event but to the computer-controlled operations that organize the bioelectric signals which audibly happen to the CI user. The CI, thus, expands audible perception by a media-technologically operationalized form of sound, according to media scientist Wolfgang Ernst so-called the ‘sonik’ [1], which previously could only be heard by techno-mathematical apparatuses, since the signals move in electrical space and not in the acoustic one (2008, p. 2).
The project “Theremin for the Deaf” shows how an extension of the human ear forms out of a hearing prosthesis; a musical instrument that shapes sound without acoustics; a relation between human being and technology — in this case the CI — that goes hand in hand with a new hearing aesthetic; a critique of our understanding of hearing. On 15 February 2018, the “Theremin for the Deaf” was presented at Humboldt University (Berlin, Germany) to the public for the first time. In addition, this unique musical instrument — probably the first one which works without acoustics — was exhibited at the symposium of the Deutsche Cochlea Implant Gesellschaft e.V. [German Cochlear Implant Society] in Hamburg (25–27 May 2018) and since then has become an integral part of the collection at the Media Archaeological Fundus (Humboldt University).
The project’s aim is to display the existence of sound beyond acoustics, provoked and organized by an implanted hearing system that challenges the typical understanding of physiological and acoustic-bound hearing. It consists of a technological aspect — which deals with soundless hearing based on the object — and a performative aspect — which, in a bizarre way, guides the spectators to this topic. To achieve the greatest possible effect, the “Theremin for the Deaf” was built as a musical instrument, since the metaphorical body of music is only conveyed via acoustics.[2]
The theremin is an electric musical instrument developed by Leon Theremin [3], who immigrated to the USA in the early 1920s (Glinsky 2000, p. 1f, 32f). Its exotic interface makes it to an extraordinary musical device in terms of how to provoke the sound, as the composer Albert Glinsky pointed out:
It was a brilliant scheme: an electromagnetic field generated by high-frequency oscillators could detect extremely small capacitances in the human hand […] and made possible a very subtle control interface. The problem […] was to orient his right hand in free space. With no tactile point of reference, the basic gesture toward and away from the antenna made only a continuous rising and falling siren sound. To slice that sound into separate pitches required the stopping of the sound between notes (p. 25f).
While other musical instruments have a visual reference point for the tones, such as the keyboard of a piano, the theremin is exempt from this. Thus, due to the lack of visual information, the theremin’s sound cannot be imagined, unlike the sounds played on a piano. Even if the sound that is played on a piano does not correspond to the actual reference point — i.e. pressed C = sounding C — or does not even produce sound — for example, because it is a MIDI controller whose assignment is missing — the possibility of the sounding through visual information remains. The theremin evades this cultural technique — the symbolic assignment of the sound. Consequently, the sound material can only be accessed by sonified electrical signals, since the theremin lacks both the sound body and the visual reference.
The sonification of the theremin, i.e. the conversion of electrical signals into sound waves via loudspeakers, serves the recipient as an audible demonstration of the technological processes that result from playing this extraordinary instrument. With reference to Ernst, without transformation of the electrical signals into acoustics, its sound is, thus, an “exclusive product of the electro-acoustic space” — a product of the sonik, “the emancipation from the cultural or anthropological binding of sound, … sound that has become mathematical” (2008, p. 6). Due to this operative and acousticless sound, the theremin is excellently suited to exclude acoustics and to question the cultural concept of hearing.
However, to hear the ‘science fiction’ and ‘laser-like’ sound of the theremin without sonification, a CI is required. The idea is that a direct connection between the theremin and the sound processor of the CI ignores the acoustics and, thus, contradicts the current understanding of acoustic-based hearing. This is realized by a Bluetooth® capable audio transmitter. In this way, CI users can receive and listen to the theremin signals — without any acoustics (!). In Addition, the project included a presentation of the “Theremin for the Deaf” in the form of a concert. However, the problem of the project was the following: How can one illustrate the inaudible sounds of the theremin for non-Cl users? Furthermore, is it possible to prove that the theremin produces a sound?
To show the operation of the theremin, a digital oscilloscope is used to make the electrical signals of the theremin perceptible to non-Cl users. Nevertheless, this only provides an insufficient proof of the sounds formed by the theremin; although the screen of the measuring device for continuous signals can show changes in the sine wave while making music with the theremin, this is not yet a proof for acoustic liberated listening. What can be seen here is just a sine wave controlled by a bizarre-looking operator who is trying to control the theremin. Only the implanted hearing system enables the sound perception of the “Theremin for the Deaf,” which is why the presence of CI wearers as ‘evaluators’ is a prerequisite for this project.
The performances in 2018 made it clear that silence is not synonymous with the absence of sound. Through the presentation of “Theremin for the Deaf,” the audience was shown visually how sound in electrical space eluded the physiological ear but at the same moment was omnipresent. Based on the feedback from the CI and non-CI users, it became clear that the criticism of acoustic-bound hearing was well received. The CI — originally developed to enable the participation of deaf people in the acoustic environment — was intended to catapult its users into active and conscious ascendency at the moment of the project’s presentation; they were able to hear both the acoustic and the electro-acoustic environment.
In this way, the “Theremin for the Deaf” disempowers the hegemony of acoustic hearing and makes it perceptible to those who are present at the moment of the performance. Nonetheless, only the CI users had access to the sound of the “Theremin for the Deaf,” which is coupled with the curious image of the theremin player — the only aspect of the performance available to the non-CI users. Additionally, the results clearly show that sound can exist without acoustics. With this understanding of sound and music, perhaps we need not worry about the fate of Bach and Lady Gaga in a world without acoustics after all.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
[1] In German, there is a difference between the two terms sonic (“das Sonische”) and sonik (“die Sonik”). According to Wolfgang Ernst, the term sonik includes electronics, computer science, and mediamatics (Ernst 2015, p. 13).
[2] Music as such is also conveyed through sheet music, which is only accessible to those who can decode the symbols of the notation. Sound, however, cannot be transcribed symbolically and is, therefore, dependent on acoustics.
[3] According to Robert Moog, the birth name of Leon Theremin is Lev Sergeyewich Termen. (Glinsky 2000, p. ix)
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Ernst, W. (2008). “Zum Begriff des Sonischen (Mit medienarchäologischem Ohr erhört/vernommen) [The concept of the sonic (heard/interrogated with a media-archaeological ear)]”. In: PopScriptum 10 — Das Sonische-Sounds zwischen Akustik und Ästhetik, [Online] Available from: https://edoc.hu-berlin.de/bitstream/handle/18452/21055/pst10_ernst.pdf?sequence=1&isAllowed=y [Accessed 10th July 2020].
Ernst, W. (2015). Im Medium erklingt die Zeit. Technologische Tempor(e)alitäten und das Sonische als ihre privilegierte Erkenntnisform [Time resounds in the medium. Technological tempor(e)alities and the sonic as their privileged form of knowledge]. Berlin: Kadmos.
EURO-CIU (2017). “What is the Cochlear Implant?”. European Association of Cochlear Implant Users. [Online] Available from: http://eurociu.eu/what-is-the-ci- [Accessed 10th July 2020].
Glinsky, A. (2000). Theremin. Ether Music and Espionage. Urbana, Chicago, Springfield: Uni. of Illinois.
Miyazaki, S. (2013). Algorhythmisiert. Eine Medienarchäologie digitaler Signale und (un)erhörter Zeiteffekte [Algorhythmic. A media archaeology of digital signals and (un)heard time effects], Berlin: Kadmos.
Miyazaki, S. (2016). “Elektrode im Ohr. Gewebe-Metall-Schaltkreise und Cochlea-Implantate — bis 1984 [Electrode in the ear. Tissue-metal circuits and cochlear implants — until 1984]”. In: SenseAbility. Mediale Praktiken des Sehens und Hörens, ed B. Ochsner and R. Stock, 125–145. Bielefeld: transcript.
Ochsner, B. and Stock, R. (2014). “Das Hören des Cochlea-Implantats [The hearing of the cochlear implant]”. In: Historische Anthropologie, 22(3), 408–425.
Weaver, W. (1964). “Recent Contributions to the Mathematical Theory of Communication”. In: The Mathematical Theory of Communication, ed. W. Weaver and C. E. Shannon, 1–28. Urbana: University of Illinois Press.
Wicke, P. (2016). “The Sonic: Sound Concepts of Popular Culture”. In: Sound as Popular Culture. A Research Companion, ed. J.G. Papenburg and H. Schulze, 23–30. Cambridge: MIT Press.
