Nao Tokui and Qosmo

Neural Beatbox is a musical experiment that lies at the intersection of the artists' many interests in everyday sound, rhythms, and fascination with pleasant surprises that the introduction of AI brings.

Nao has always been fascinated by the sound around us and its ability to stimulate our imagination. He has recorded various soundscapes by himself, and with Qosmo, has created an AI-based website that presents images of Google StreetView with matching "imaginary" soundscapes.

As a DJ, rhythm is always at the core of Nao’s music practice. The artist trained a Variational Autoencoder(VAE)-based rhythm generator model with hundreds of publicly available MIDI data of dance music rhythms. Here is a visualization of potential rhythm patterns in the model’s latent space and audio samples of selected patterns. 

Neural Beatbox consists of two components: (1) a CNN-based sound classification model that segments and classifies the recorded sound to create an instant drum kit and (2) the rhythm-generation model described in the last section.

By integrating those two components and two independent passions of the artist, Neural Beatbox keeps generating exciting beats based on everyday sounds around us. The artists found the slight imperfection of AI models attractive. And they see a future where AI doesn’t only replicate music we humans have created, but also creates something novel and, in turn, inspires human creativity. 

To create a unique piece of music, Nao also trained a SampleRNN model on the dataset of field recordings he has collected over the years. After a few days of training, the model started generating infinite fictional soundscapes. 

Then, he extended his rhythm-generation model to incorporate the GAN (Generative Adversarial Networks) framework. He added an additional discriminator that classifies dance music genres of generated rhythm patterns.This Creative-GAN scheme incentivizes the generator to "confuse" the classifier as well; in turn, the artist anticipates unique rhythm patterns that defy the genre classification coming out of the model. 

Finally, Nao fused everything (SampleRNN/Creative-GAN rhythm generation and MelodyVAE model) into his composition, “Synthetic Soundscape.” He didn’t add or delete a single MIDI note manually by himself. His models generated all sequences, and he relished the idiosyncrasy of those sequences and stitched together appropriate ones. The resulting composition reflects the uncertainty of this time and the reminiscence of “normal” life. In this work, Nao illustrates that embracing AI's organic nature leads to unique outcomes and extends creativity.