Leveraging Game Mechanics for Dynamic Music Co-Creation

Escarce Junior, Mario and Soriano Marcolino, Leandro (2025) Leveraging Game Mechanics for Dynamic Music Co-Creation. PhD thesis, Lancaster University.

Text (2025escarcephd)
2025escarcephd.pdf - Published Version
Download (28MB)

Abstract

This thesis explores game mechanics for creating coherent artworks, with a specific focus on music composition. The investigation spans three interconnected areas of partially-autonomous and autonomous systems: implicit cooperation, meta-interactivity, and autonomous generation. The main research question is: How do different degrees of human-machine collaboration, ranging from implicit cooperation to meta-interactivity and machine autonomy, impact the quality, user experience, and aesthetic attributes of music created within virtual environments? The thesis begins by exploring co-creativity within games through an algorithm fostering unwitting cooperation between humans and gameplay mechanics, termed implicit cooperation. This approach enables cooperative music emergence, fostering engaging artistic collaborations and pleasant musical experiences. Next, it introduces meta-interactivity for music creation, empowering novices to achieve unexpected outcomes in composition and practice. Using imagetic elements in a virtual environment, this approach converts ludic interactions into music. A user study involving experts and novices highlights its potential to unlock creativity in individuals with limited musical training, while also prompting questions about the role of human sentiment and expressivity in dynamic artistic creation. Lastly, this thesis presents an autonomous system for dynamically generating immersive soundscapes for games and artistic installations. This system simultaneously produces music and images, preserving human intent and coherence. An algorithm for audiovisual instance generation demonstrates its effectiveness compared to alternatives. Through these explorations, this thesis sheds light on the evolving landscape of music co-creation, proposing novel interactive experiences based on game mechanics. It aims to contribute to the ongoing debate on the collaborative potential between humans and autonomous systems, with a specific emphasis on their transformative influence within the domains of music and games. By examining the impact of partially and fully autonomous systems on human sentiment, this research offers insights into the evolving relationship between humans and technology, as well as the intricate interplay between music and imagery in audiovisual works, presenting promising avenues for future research and innovation in this domain.