Marsden, Alan (2004) Novagen: a combination of Eyesweb and an elaboration-network representation for the generation of melodies under gestural control. In: COST287-ConGAS Symposium on Gesture Interfaces for Multimedia Systems, 2004-03-29.
Abstract
A system which generates melodies influenced by the movements of a dancer is described. Underlying the melody-generation is a representation based on the theory of the early 20th-century German musicologist, Heinrich Schenker: a melody is derived from a simple background by layers of elaboration. Overall, the theory has similarities to a generative grammar. Generation of melodies is achieved by repeatedly applying elaborations to the background to achieve the desired number of notes. Elaborations are selected by a weighted random process which can take into account the pattern of elaborations used earlier in the melody. A number of parameters control this process, both by setting the relative weights for different elaborations and by controlling the number of notes generated, their distribution throughout the bar, and the degree of similarity of the generated pattern to previous sections of the melody. These parameters are adjusted via MIDI messages from an Eyesweb application which tracks a dancer via video to categorise the pose or movement observed into one of four categories, and to determine the degree of 'activity' in the movement. The result is real-time generation of a novel melodic stream which appears meaningfully related to the dancer's movements.