The Aesthetics of Disharmony: Harnessing Sounds and Images for Dynamic Soundscapes Generation

Escarce Junior, Mario and Rossmann Martins, Georgia and Soriano Marcolino, Leandro and Rubegni, Elisa (2023) The Aesthetics of Disharmony: Harnessing Sounds and Images for Dynamic Soundscapes Generation. ACM - PACMHCI CHI PLAY, 7 (CHI PL): 399. pp. 665-698.

Text (Mario Escarce - Solato)
Solato_CHI_Play_2023_Camera_Ready_Pure_Version.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (23MB)

Abstract

This work presents an autonomous approach that explores the dynamic generation of relaxing soundscapes for games and artistic installations. Differently from past works, this system can generate music and images simultaneously, preserving human intent and coherency. We present our algorithm for the generation of audiovisual instances and also a system based on this approach, verifying the quality of the outcomes it can produce in light of current approaches for the generation of images and music. We also instigate the discussion around the new paradigm in arts, where the creative process is delegated to autonomous systems, with limited human participation. Our user study (N=74) shows that our approach overcomes current deep learning models in terms of quality, being recognized as human production, as if the outcome were being generated out of an endless musical improvisation performance.