◁ The Fortnightly Tambourine

Musical Instruments and Digital Computers

During my PhD, I went a bit out of my way to comment on the history of digital musical instruments. I felt that it could aid in motivating the reader for the myriad of more complex topics to come. The original section, with minor corrections, follows here.

The theory was applied to the development of the open-source Piano Forte digital instrument. More info about the project can be found at Omnes Sonos. The full thesis is available at https://app.uff.br/riuff/handle/1/29781.

Music is a staple of humanity’s creativity. Montagu defines music as sounds generated deliberately with the aim of provoking emotion.¹ Musical instruments, consequently, can broadly be defined as any tool used for that purpose. In this context, music can be traced back to time immemorial, with its origins probably motivated by ritual practices performed by early humans. Whereas the very first musical instrument was most probably the voice, it might have been shortly followed by other rudimentary musical instruments, likely of a percussive nature²: Evidence of the use of simple instruments, such as drums and bone flutes, can be traced back to the Paleolithic era, more than 30,000 years ago³.

During the course of human history, musical instruments became more elaborate, incorporating technological advancements, responding to and also helping to evolve the predominant aesthetic of music in each society. Despite reaching, along this long period of evolution, a high degree of sophistication in their design, before the 20th century, musical instruments necessarily relied on mechanical vibrations to generate their sounds.

In a comprehensive timeline of the technology associated with electronic music, Stubbs marks the year 1876, with the invention of the Musical Telegraph by Elisha Gray — a device that broadcast single notes along telegraph lines — as the beginning of the electronic instruments’ era.⁴

Regarding stringed instruments, in 1890, George Breed, a United States Naval Officer, filed a patent for an electrified guitar. Electricity, however, was not used for amplification purposes, serving instead to continually vibrate the guitar strings, leaving the instrument’s sound acoustic in nature. The patent also suggests the use of this method for an electric piano, albeit in a less detailed way. The guitar’s design, in addition to differing considerably from what we now consider an electric guitar, presented some idiosyncrasies that probably prevented its commercial success⁵.

Other authors place the origin of this new electronic era for musical instruments at the invention of the Telharmonium, patented in 1897 by Thaddeus Cahill, due to the influence the instrument eventually exerted. The Telharmonium, also known as Dynamophone, was a machine weighing around two hundred tons, intended to synthesize music with the use of sinusoidal tones of different frequencies generated with modified dynamos, to be broadcast in real-time over telephone lines⁶.

Still others, such as Montagu, attribute the beginning of electronic instruments to the invention of the Theremin by Lev Termen, around 1920, on the grounds of the popularity the instrument accrued at the time, both in its original USSR and in the USA.⁷

About a decade later, the first commercially successful guitars started to emerge. Notorious among them was the Gibson ES-150 in 1936, with a single pickup responsible for capturing the vibration of the metallic strings⁸. Electric guitars, due to the mechanical origins of the sound generated, are not regarded as purely electronic instruments.

Be that as it may, the Moog synthesizer was the instrument responsible for popularizing electronic instruments. Implementing the ideas published by Robert Moog⁹, it is largely considered the first analog synthesizer, and the first commercial synthesizer.

Wendy Carlos’s 1968 album Switched-On Bach, featuring Bach compositions arranged for the Moog synthesizer, achieved critical and popular acclaim¹⁰, helping to popularize the instrument, which would also be used by bands such as The Beatles, Rolling Stones, and Yes.

The theoretical foundation underlying the developments introduced by the process of electrification, especially concerning the area that came to be known as digital signal processing, can be traced back to a few centuries before those events, however.

The work of d’Alembert and the introduction of differential equations¹¹, Euler, with the invention of integral transforms¹², Bernoulli, and most notably Fourier, formed the theoretical basis for the birth of DSP¹³: how this process happened is described in more detail in Chapter Theory.

For the theory to be applied in the form it is known nowadays, however, computers were needed, and it was only after the Second World War that such machines would first emerge, with examples such as the ENIAC and the IAS, dating from the mid to late 1940s. With growing access to computers, the area of DSP started to consolidate itself in subsequent decades.

Fortunately, parallel to that revolution, another was taking place, one that would eventually lead to the development of modern digital computers. The need to process large amounts of data, both in Europe and the US, led to an increasing interest in calculating machines. A punched card tabulating system was developed for the US government to be used in the 1890 census and became so popular in the first half of the twentieth century that it originated IBM. A few decades later, the company consolidated itself as a business machine manufacturer, with an income of 20 million dollars by 1928.

This quest for computing machines, and the rudiments of modern computer science, can be traced back at least a century earlier, however, as described in Kelly¹⁴, to the efforts of Charles Babbage.

Motivated by the desire to improve the process of manufacturing navigational tables for the English government, Babbage designed his first famous computation machine, the Difference Engine, in the early 1820s. Despite securing funding and completing a working prototype, Babbage never completed a full-scale Difference Engine. Technical problems encountered along the way led him to investigate the state of the art of English mechanical systems and eventually conceptualize, about a decade later, the Analytical Engine, first mentioned in a statement for the English government in 1834.

By suggesting that funding should be shifted to this new machine without having completed the former and the tables it was supposed to produce, Babbage undermined the government’s confidence in him. Despite this, Babbage continued to work on the Analytical Engine for the rest of his life and secured the interest of people such as Ada Lovelace, responsible for the most extensive account of the machine at the time.

Ultimately, however, his efforts were insufficient to ensure the construction of the machine, with Babbage’s failure motivating scientists to momentarily pursue another path to computing machines: analog computing. This was unfortunate, since Babbage’s design for the Analytical Engine exhibited all the characteristics encountered in a modern digital computer, such as separation of arithmetic and storage, and would likely be Turing-complete¹⁵.

Nevertheless, the beginning of the 20th century saw the ascension of analog computing, with the term analog coming from the word analogy. Analog computers were single-purpose machines built as scale models of the problems they were intended to solve, such as dam building and electrical grid design.

In 1937, inspired in part by Babbage’s concepts, Aiken proposed to IBM the first designs of the Mark I, which was finalized, after a series of delays, in January 1943, becoming the first fully automatic computation machine. It was electromechanical in principle, with an abundance of moving parts, and was eventually superseded by strictly electronic designs.

Parallel to those developments, in the field of mathematics, Turing and Church were making advancements in the theory of logic, independently arriving at similar results in the mid-1930s. Turing’s approach, however, involved a conceptual computer, later known as the Turing Machine, and a proof, described in his article ``On Computable Numbers with an Application to the Entscheidungsproblem’’¹⁶, that this idealized machine could compute any function.

While staying at Princeton University, Turing met John von Neumann, who would go on to play a prominent role in the invention of the modern electronic computer, inspired in part by Turing’s work.

John von Neumann became involved in the computer development scene when he became aware of the ongoing efforts for the construction of the ENIAC, around 1944, a computer that used thousands of valves and was built to use the decimal system. Neumann started to work as a consultant for the team and soon identified many potential problems with the machine, leading him to start designing a successor capable of addressing those problems.

His report ``A First Draft of a Report on the EDVAC’’¹⁷, originally from June 30, 1945, laid the basis for modern digital computers. When finally operational, at the beginning of 1950, the EDVAC ushered in the new era of digital computers. From this point on, the process of popularization of the computer, from a mathematical machine to a general-purpose appliance, would continue, eventually culminating in the creation and widespread adoption of personal computers.

Since the early days of digital computers, even when mainframes were the only available machines, restricted to organizations with considerable resources, people found ways and motivation to overcome technical limitations and develop artistic applications, like video games, in their — and the machines’ — spare time.

At the 1940 New York World’s Fair, for example, the Nimatron was first exhibited¹⁸. It was a single-purpose, electromechanical machine designed to play the game of Nim, described in Redheffer¹⁹.

Another example is a computer chess game for programmable machines, written by Turing in 1947, that could not be implemented due to the limitations of available computers at the time²⁰. In the following year, the patent for the cathode-ray tube amusement device, considered the first known example of an interactive electronic game, was granted²¹.

Regarding musical applications, in 1957, while working at the Bell Telephone Laboratories, Max Mathews created MUSIC I, a programming environment intended to simulate the circuitry for analog musical synthesizers in digital computers²². His efforts are described in Mathews²³, and inaugurated the quest for digital sound synthesis.

In the seminal paper ``The Digital Computer as a Musical Instrument’’²⁴, Max Vernon Mathews, after years of using computers to analyze sounds at Bell Labs, argued that digital computers could be used not only to aid in composition and other high-level music-related tasks, but also to generate sounds that could be reproduced by loudspeakers.

The paper highlights that computers are theoretically capable of synthesizing any sound, proceeding to describe the process in which discrete numbers can be transformed into sounds and, in so doing, introducing the concept of discrete numbers as samples from the instantaneous pressure of a sound wave.

To convert those numbers to sound, a digital-to-analog device, capable of generating pulses proportional to the magnitude of each number, is to be used, and those pulses smoothed a posteriori. The paper also highlights the practical necessity of compact sound representations.

Mathews went on to implement, in 1968, the successor to the MUSIC I-IV family, Music V, in Fortran, the first music programming language to be implemented in a portable programming language²⁵.

Around that time, in 1965, Cooley and Tukey introduced what is now known as the FFT algorithm, in the paper ``An algorithm for the machine calculation of complex Fourier series’’²⁶, one of the 10 most influential algorithms of the 20th century²⁷.

The FFT algorithm, by exploiting symmetries, reduced the complexity of the calculation of the DFT from \(O(N^2)\) to \(O(N \log_2(N))\), where \(N\) is the length of the original discrete signal, for cases when \(N\) is a power of \(2\). Although this algorithm was originally discovered by Gauss²⁸, and rediscovered at least a handful of times in the approximately 100 years between Gauss and Cooley²⁹,³⁰, it became famous in the form proposed by the latter.