Introduction
Interactive computer music has become popular among composers as a means for bridging signal processing and computer composition with contemporary performance practices, allowing the musical competence of musicians to shape both the acoustic and electronic material. Furthermore, interactivity is often an integral part of sound installations, where various actions by the visitors influence the presentation and processing of the material. These expression types are well known, and the topic of numerous books, journal articles and conference presentations. Little work, however, has been done at the intersection of computer music and computer games. The computer game industry, however, generates more revenue annually than the Hollywood film industry as a whole, and is hardly a negligible sector of contemporary popular culture. Thus, computer gaming has become a tempting arena for creative work from an artistic perspective, an activity that in turn might contribute to enriching the gaming domain itself.  This article will describe perspectives that have been active in the creation of the computer music game Construction Drive, and discuss approaches and concerns that have emerged in the work with this new genre.

Construction Drive is an interactive art project that bridges computer animation and music, in the shape of a computer music game for the Windows operating system 1 that aims to challenge a common notion that games are only entertaining, and that art is only serious. The sounds and images are concerned with the nature of art and the riddle of perception and construction of meaning. The graphical environment supports the treatment of the sound material as a type of musical game: The player navigates in a virtual world that is crafted from a well-known painting, and performs the musical piece by ”driving” in the game environment as if in a motorcycle simulation. This is a compositional technique that has also been employed in games such as the one based on the film Jurassic Park, for the Microsoft Xbox (Schütze 2003).

Sound design and narrative
Construction Drive contains both singular sounds and pre-rendered sections with particular musical significance. Its sounding material is gathered from recordings of natural sound, as well as from recordings of readings from the poet and art critic Guillaume Apollinaire (1880–1918) on themes related to the artistic style of this particular painting. Apollinaire wrote extensively about the avant-garde of his time, and the texts used in Construction Drive are excerpts from the Everyman edition of Apollinaire’s writings, translated and edited by Robert Chandler. The text snippets are read by Howard Gamble, and in addition to these sound sources, the game utilizes recordings of motorcycle engines, bicycles and rain. The sound material has been developed using computer music techniques, and is constructed with the intention of having an inner coherence that makes it interesting and rewarding to explore. A performance of the game takes shape as an exploration of what the game has to offer: a landscape strewn with sounds that are activated as the player enters their active zone. The mixing of sounds is based on the actions of the player, and there are few restrictions of movement placed on the player. The game can be quit and re-entered at any time, however it is over when the player has solved a riddle by combining the found clues of recorded text, which are randomly placed in the environment with every new game. When the player needs to start a new game, the sounds and cues are redistributed in a different manner. Musical training is not required - the construction of the work depends solely on the ear and eye - but an experienced and critical user will have an easier time “getting something good”.

Electronically processed sound is the primary sound material Construction Drive, and has allowed great freedom to create a narrative with different levels of abstraction by playing on the representational and semantic values in the sounds themselves. Such computer music techniques allow sounds to be manipulated towards more or less clarity, and were used to blur the simple semantics that often follows from the use of text. Thus, a context of ambiguity was established in Construction Drive; an ambiguity designed to open up for the perception of more than simple, referential listening. Digital signal processing opens up the sounds for interpretation, and was also used for artificial ”ageing” of the recorded voice, which was done in order to give the player an impression of authenticity where voices from the past gave clues for how to solve a riddle in a landscape covered with remains of life that had come to an end in the same past. What happened at that time? Which clues can be found in the voices? The reference to J.G. Ballard’s perspectives when describing contemporary times was intended to be clear, almost equally explicit as in the work Concrete Net, which was created in the late 1990s.2

The narrative in Construction Drive is not a combination of what in film is described as diegetic and non-diegetic sounds – sounds that only the viewer hears, or that the characters in the film may also hear. Rather, the sounds are what the player hears, since there are no characters in the game, and the sounds are a combination of sounds emerging from the exploration, sounds that make up the riddle, and status sounds that lets the player know that his/her actions have been effectuated. However, since the game lies within the simulation paradigm, the signal sounds blend with the exploration- and riddle-sounds into one experience, although they have three distinct functions. Blurring and ambiguity are natural elements in most sonic experiences, as meaning is abstracted from complex auditory scenes based on the combination, presence or absence of sounds that we hear or expect to hear (Bregman 1990), so this approach is not problematic for the player to grasp. Some sounds represent specific actions with enough clarity to be considered sound icons, whereas others are ambiguous and may refer to a diversity of actions, leaving the listener free to “connect the dots”. These types of aesthetic and analytic issues are also frequently discussed within the framework of spectromorphology (Smalley 1986, 1997), as a further development of Pierre Schaeffer’s ideas as he set them forth in his principal work Traité des Objects Musicaux (1966).

Interaction
Composing interactive music is most often quite different from writing music to be reproduced from a score. In interactive works such as games, the composer relinquishes different degrees of control of the distribution of sound events throughout the piece to users of the interface, be it a musical instrument, a sensor, or a meta-level controller. This approach to composition has its parallels in more conventional music with chance-based operations and routines that are triggered from external events that are not controlled by the composer. One can think of Xenakis’ sound clouds, Cage’s chance operations with I Ching and Nordheim’s weather controlled installation works. In many computer games, composers employ sequencing software like Microsoft’s Direct Music, where soundtracks are built from General MIDI-libraries and recorded sound, and where orchestration changes on cues from user actions, game events, and the player’s location in the game environment. This is an effective means of maintaining a high degree of real-time control of the soundtrack, but is better suited as a tool for creating a game music track with mood changes and effects than computer music in the conventional sense. A recent example on creative use along these lines is the much-appraised game Limbo, which was released in 2010. When Construction Drive was created, there were limited numbers of signal-processing routines that the console were powerful enough to run, and great care had to be taken in managing the available memory. As an example Playstation, had 1MB RAM available for loading all sound material that should be looped or otherwise processed. Computer music has generally not been limited in the same manner, but has relied on heavy signal processing and detailed control of timbral development.

In Construction Drive, the sequence of sounds resulting from player movement can be different every time the game is played, and likewise the spacing of events. This type of situation is challenging for the composer. It is nonetheless possible to cluster sounds into different areas of timbral and thematic character, enabling a good degree of control over the coherence in the piece, although not the details. Furthermore, the visual elements reinforce the possibility of creating a thematic space, despite the non-linearity of the video narrative, where the user has few restrictions in choosing where in the landscape to drive. An important consideration has thus been to secure musical results that are “good enough” for the players to want to return for another round, and certain concessions have also been made to the game paradigm, which demands that there is a point, reason or goal for playing.

Composing interactive music involves balancing the need for compositional control and development of material with the degree of freedom that performers need in order to have their real-time composition significantly determine the resulting musical form. This balancing act is equally important in all interactive music for concerts, installations and games. The implementation of control features should be developed with the performers/public in mind; there is for example a need for the public to understand the user interface in an installation in order to interact meaningfully with it, from their perspective. In Construction Drive, the development of the music happens through the player’s movement in the landscape of the game, where location determines the sound material, and direction of movement, leaning angle and speed control the processing of the material. The sounding material is linked to the landscape model, and with few restrictions it allows user freedom in selecting musical direction: navigating by ear.

In a sense, this kind of composition can be said to be a meta-composition – in the sense that a constrained environment defines the actual composition only through the raw material and parameter scope and range for the signal processing. As mentioned earlier, this idea is not new, historically. Many kinds of process-oriented sound installations employ similar principles for turning the audience into performers, by capturing movements or actions through sensors, and letting those data control aspects of the sound. However, there were not, and still aren’t, many examples of interactive works situated within a game paradigm, where the game itself is a sonic game, with the exception of a few specialized games for the blind. Construction Drive consists of an interactive environment, and the finished work for the player is a journey in this interactive environment – but only one of an infinite number of possible journeys.

Storytelling
A narrative is a story, while storytelling is an action, in some ways the differences are similar to composition and composing. Composing can be abstract storytelling, or the telling of abstract stories - perhaps both. Extending Varèse’s definition of music as organized sound, music may also be thought of as contextualizing sounds, where single components become meaningful or change meaning when considered as part of a whole. In Construction Drive, extra-musical ideas are interwoven with the goal of mirroring and addressing the complexities in nature and human interaction, context-dependent and difficult to predict as they are. The object of this composition has been taken from ideas of action and consequence, specifically how perceptions are structured and developed into patterns, and how resonance in the sense of emotional recognition emerges - in short, how our myths are formed. While myth today is often understood as “a false statement”, the Greek mythos refers to powerful truths about the world, often encapsulated in stories.

The composer can tell several stories at the same time by modelling complex situations from dissimilar elements, and the totality might be difficult to grasp without an understanding of the parts. The parts might also lose their significance if removed from the whole. In this complex weave, elements vie for attention, and what may be considered particularly important parts of the compositions are also merely a small part of this larger sonic amalgam. The intention is that when the elements in the music that are most difficult to hear - those, which the surrounding sounds mask and offer the most resistance - are noticed, the listener will be rewarded with a sense of direction or continuity. The resistance requires effort, and listeners may need to overcome their own resistance as well - not becoming bored - because it may take a while before the idea is uncovered. The possibilities for these kinds of reactions are composed into the works, and as such, they enter into a tradition of works that employ a systematic stretching of the audience’s patience. (The composer has deliberately included the same need for endurance in the audience in the first three minutes of his work When Timbre Comes Apart.) By stretching the listener’s patience and expectation of musical development in time, the composer demands that listeners be creative in finding their own patterns rather than waiting for the composer’s “message” or statement. As a complementary point - a very strong sense of clarity might be deceptive, as Czeslaw Milosz suggests in his 1951 novel The Captive Mind: “When someone is honestly 55% right, that’s very good, and if someone is 60% right, it’s wonderful and great luck, but if someone is 75% right, it is suspicious, and whoever says he is 100% right is a fanatic!” Uncertainty and perception of ambiguity is a human and compositional element that can be reinforced or quelled by either visual or auditory means - or both.

Imagery and mapping
The visual part of the game environment of Construction Drive is modelled on Joan Miro’s surrealist painting L’estel matinal (The Morning Star) from 1940. Apart from the associations engendered by the painting itself, the image has been used directly in two ways in Construction Drive. The painting has been mapped onto a digitally modelled, three-dimensional landscape, with geological formations that largely follow lines suggested in the image. Geometric and amorphous shapes in the painting have been made three-dimensional and situated as objects in the landscape according to their original placement. The background in the painting has been reinterpreted into a 3D-landscape, and Miro’s rich collection of disparate objects - loosely connected visually through a web of lines - has been modelled into 3-D objects that are placed in the landscape, like abandoned structures. Colours and textures in all the modelled objects have been kept close to Miro’s palette in the painting, while the “geological” formations are free reinterpretations. Some of the objects have been lifted away off the surface, and either hang above the landscape or stand as bridges in it.

The underlay and concrete use of L’estel Matinal is not made clear to the player until the closing sequence, and then only briefly, and since no particular emphasis is placed on the use of this particular image, only players that are familiar with Miro or this painting will leave the game conscious of this reference. Apollinaire’s textual contribution dates the ideas, as the language he uses is different from the one used today. The texts and the painting have served as inspiration for the compositional idea of a type surrealist discussion with recognizable ideas and words, but with little recognizable logic; in some ways akin to the claim that art has an internal logic not easily explained in other ways than through the works themselves.

Musical notation, listening/analysis scores, and plots in time- and spectral domains represent different aspects of music. Each representation type focuses on different elements or parameters of the musical idea or object, and these representations serve as tools for the composer/artist, facilitating different manipulations of the musical material. However, not all such forms of notation are intended to be interesting visual objects in their own right, although several musical scores can be considered as visual art, such as for example Agglomaration (1960) by Anestis Logothesis, December 1952 by Earle Brown, and Five Piano Pieces for David Tudor (1959) by Sylvano Busotti.

Mapping can be considered a subset of representation, and in binary format, data can be ported with relative ease from one domain to another. However, the data may not make much sense when ported into the domain of another medium, as was shown for example in the phase vocoder experiments with re-synthesis of graphics files in the software program Ceres, originally authored by Øyvind Hammer at NOTAM3 in 1994.4 A scanned image was not a good source for re-synthesis when read as a spectrum. To represent music or sound visually in a cross-media approach, decisions must be made about which aspects are to be represented; in other words, the mapping process involves interpretation. Cross-disciplinary representations may also include auditory display systems such as sonification, although the goals for these are most often not artistic. Mapping data types and values onto sounds also involve an interpretation, which can distort perceptions of how data are connected.

Cross-disciplinary representations are not trivial from a timbral point of view, and it has been argued that they make more sense when informed by psychoacoustics and when aware of the expectations they set up for more complete auditory scenes. An example of this is the use of sound in computer games, in which the feedback and user-interface sounds provide auditory information on system status and user actions, and the music changes dynamically according to position, action, and environment (Schütze 2003). New possibilities for narratives arise when other forms of representation and expression are added to sound, and the composer must consider which links, if any, are desirable to establish between the forms of representation in the specific work. There are many interesting examples from interactive dance and dramatic performances in which different methods for extracting data from the dancers’ movements have been employed (Siegel 1998; Bevilacqua, Ridenour, and Cuccia 2002; see also Todor Todoroff’s 1998 work En Jeu and his collaborative work In Between from 2000). In these examples, the data have been linked to sound generation and processing, and in some instances to lighting and digital scenography, and they all have clear limitations. In the reverse direction, video imagery can be based on the mapping of parameters taken directly from the music, as in Roger Dannenberg’s work Uncertainty Principle (2000), and an example of current software for that type of processing is the Max-extension Jitter. However, the challenge remains the same; mere mirroring is problematic, because a one-to-one migration in between the senses is impossible – and video mirroring of sound most often look like screen savers under development.

Closing remarks
Composing for computer animation entails a relatively new art process that requires audio-visual coherence in order to be interesting. We can say that this was not a real possibility until around 1995, with the introduction of the computer platform Silicon Graphics and its unique graphics processing methods. As elsewhere in the digital domain, the opportunities easily overshadow the artistic intentions, and surface becomes the content. The process raises questions. How should the visuals relate to the sounds? Should they be referential or non-referential, concrete or symbolic; should they be recognizable as representations we are familiar with, or should they be abstract? Should the visuals somehow be derived from the sound, or are they being independent from it? In addition, there are aspects of movement: colour, light, and camera angle and camera speed. Weighting and balancing of visual and sonic elements both complicate and enrich interactive compositions such as Construction Drive.

The game genre also poses challenges, and slow dwelling on timbral development might become pitted against the idea of action necessary for driving the game forward.

For many composers, making music is a type of research, a personal exploration of ideas and how they may be given audible form. Combining music with visual media changes and recharges this notion of research, and presents opportunities to engage with recent commercial technologies, such as MUDs. When considering the content in most published games, it is clear that this largely is an unexplored niche for art, holding the promise of new audiences, perhaps particularly in combination with social perspectives where more than one interact with the game algorithms.

References
Apollinaire, Guillaume (1902-1918). Everyman poetry, 1902-1918. Republished Everyman paperbacks 2001. New York, New York.

Bevilacqua, F., J. Ridenour, and D. J. Cuccia. 2002. “3D Motion Capture Data: Motion Analysis and Mapping to Music.” In Proceedings of the 2002 Workshop/Symposium on Sensing and Input for Media-centric Systems. Santa Barbara, California.

Bregman, Albert. 1990. Auditory Scene Analysis. Cambridge, Massachusetts: MIT Press.

Milosz, C. 1951. The Captive Mind. New York: Vintage.

Schaeffer, P. 1966. Traité des Objects Musicaux. Paris: Editions du Seuil.

Schütze, S, 2003. “The Creation of an Audio Environment As Part of a Computer Game World: The Design for Jurassic Park—Operation Genesis on the Xbox As a Broad Concept for Surround Installation Creation.” Organised Sound 8(2):170–180.

Siegel, W. 1998. “The Challenges of Interactive Dance: An Overview and Case Study.” Computer Music Journal 22(4):29–43.

Smalley, D. 1986. “Spectro-Morphology and Structuring Processes.” In S. Emmerson, ed. The Language of Electroacoustic Music. London: Macmillan, pp. 61–93.

Smalley, D. 1997. “Spectromorphology: Explaining Sound-Shapes.” Organised Sound 2(2):107–126.

• 1. The development history of this game was very long, due to unforeseen delays caused by other tasks that needed concentration. Initially, the development was aimed at the Sony Playstation 1 gaming console, however, briefly after the development work was started, Playstation 2 came out, and Microsoft Xbox was also introduced on the market. The development was then moved to Xbox. Following a period of negotiations with the Xbox team at Microsoft, it was decided to move it to a normal Windows environment, and the idea of using a dedicated gaming console was scrapped.
• 2. Rudi, Jøran (1997) Concrete Net. The work is available on Aurora Records (ADVDV5501). The DVD contains the works When Timbre Comes Apart, Concrete Net and Planet. Oslo: Aurora. 2001. An essay can be located at this address: http://users.notam02.no/~joranru/cn/index.html.
• 3. Norwegian Center for Technology in Music and the Arts, www.notam02.no
• 4. Find information of the software and link to the sourcecode here: http://www.notam02.no/web/2006/01/ceres-2/?lang=en