SEED Research & Announcements Blogs Publications Open Source Careers Contact Us Research & Announcements Blogs Publications Open Source Careers Contact Us

AIIDE22: Neural Synthesis of Sound Effects Using Generative Models

This research paper was presented at the 18th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE).

Authors: Sergi Andreu (KTH), Monica Villanueva Aylagas (SEED).

Neural Synthesis of Sound Effects Using Flow-Based Deep Generative Models

Download the full research paper (1 MB PDF).

Creating variations of sound effects for video games is a time-consuming task that grows with the size and complexity of the games themselves. The process usually comprises recording source material and mixing different layers of sound to create sound effects that are perceived as diverse during gameplay.

In this work, we present a method to generate controllable variations of sound effects that can be used in the creative process of sound designers. We adopt WaveFlow, a generative flow model that works directly on raw audio and has proven to perform well for speech synthesis. Using a lower-dimensional mel spectrogram as the conditioner allows both user controllability and a way for the network to generate more diversity. Additionally, it gives the model style transfer capabilities.

We evaluate several models in terms of the quality and variability of the generated sounds using both quantitative and subjective evaluations. The results suggest that there is a trade-off between quality and diversity. Nevertheless, our method achieves a quality level similar to that of the training set while generating perceivable variations according to a perceptual study that includes game audio experts.

Evaluating Data-Driven Co-Speech Gestures of Embodied Conversational Agents through Real-Time Interaction

Related News

Gigi Lightning Talks

SEED
Sep 26, 2024
SEED brought together developers to show off their prowess using the Gigi rapid prototyping platform for real-time rendering.

SEED's Adventure in Gameplay Innovation

SEED
Sep 13, 2024
SEED is branching out into the world of game mechanics, storytelling magic, and interactive wonders.

Objective Metrics for Evalutating Gesture Generation are Almost Useless

SEED
Sep 10, 2024
How do you evaluate something as subjective and ephemeral as human body language for natural and lifelike qualities?