Cultivating chaos: Nao Tokui and Qosmo make the best use of AI’s somewhat unpredictable behaviors to think and create out of the box.
Neural Beatbox/Synthetic Soundscape 2021 AI-Guided Music Experiments
In this solo variation of Neural Beatbox, visitors improvise a continuous, generative drum session made of their own sounds and a remix of their face. With the help of a neural network model, the recordings are classified into drums and new beats are generated. As those audiovisual fragments play back in rhythm, the visitors establish a dialog with their past selves.
The experience is optimized for Chrome and Firefox on desktop computers. If user’s camera and microphone are enabled, audiovisual content will be used in the recording. No audiovisual data is permanently stored.
Neural Beatbox is a musical experiment that lies at the intersection of the artists' many interests in everyday sound, rhythms, and fascination with pleasant surprises that the introduction of AI brings.
Nao has always been fascinated by the sound around us and its ability to stimulate our imagination. He has recorded various soundscapes by himself, and with Qosmo, has created an AI-based website that presents images of Google StreetView with matching "imaginary" soundscapes.
As a DJ, rhythm is always at the core of Nao’s music practice. The artist trained a Variational Autoencoder(VAE)-based rhythm generator model with hundreds of publicly available MIDI data of dance music rhythms. Here is a visualization of potential rhythm patterns in the model’s latent space and audio samples of selected patterns.
Neural Beatbox consists of two components: (1) a CNN-based sound classification model that segments and classifies the recorded sound to create an instant drum kit and (2) the rhythm-generation model described in the last section.
By integrating those two components and two independent passions of the artist, Neural Beatbox keeps generating exciting beats based on everyday sounds around us. The artists found the slight imperfection of AI models attractive. And they see a future where AI doesn’t only replicate music we humans have created, but also creates something novel and, in turn, inspires human creativity.
To create a unique piece of music, Nao also trained a SampleRNN model on the dataset of field recordings he has collected over the years. After a few days of training, the model started generating infinite fictional soundscapes.
Then, he extended his rhythm-generation model to incorporate the GAN (Generative Adversarial Networks) framework. He added an additional discriminator that classifies dance music genres of generated rhythm patterns.This Creative-GAN scheme incentivizes the generator to "confuse" the classifier as well; in turn, the artist anticipates unique rhythm patterns that defy the genre classification coming out of the model.
Finally, Nao fused everything (SampleRNN/Creative-GAN rhythm generation and MelodyVAE model) into his composition, “Synthetic Soundscape.” He didn’t add or delete a single MIDI note manually by himself. His models generated all sequences, and he relished the idiosyncrasy of those sequences and stitched together appropriate ones. The resulting composition reflects the uncertainty of this time and the reminiscence of “normal” life. In this work, Nao illustrates that embracing AI's organic nature leads to unique outcomes and extends creativity.
Nao Tokui is an artist and researcher based in Japan. While pursuing his Ph.D. at The University of Tokyo, he released his first music album and other singles, including a 12-inch with Nujabes, a legendary Japanese hip-hop producer. Since then, he has been exploring the potential expansion of human creativity through the use of AI. His recent works include AI DJ Project where a machine learning-based DJ plays alongside a human DJ (Nao, himself), taking turns selecting and playing songs one at a time, as well as a collaboration with Brian Eno. His works have been exhibited at the New York MoMA, Barbican Centre (London), InterCommunication Center (Tokyo), and more. He published his first book on AI and creativity in January 2021.
Twitter | Medium | SoundCloud | Website
Computational Creativity and Beyond. Qosmo is a group of artists, researchers, and designers founded and led by Nao Tokui based in Tokyo. It has the core idea that capturing the organic nature present in AI technologies and generative methods can aid the creative process and enrich artistic expressions. The Experience: Robin Jungers Visualization: Ryosuke Nakajima Diagram, Typography: Naoki Ise
Twitter | Medium | Instagram | Website
Nao Tokui, an artist and researcher based in Japan, will lead a beatbox-making workshop using his web-based app, Neural Beatbox. This app will enable attendees to collectively create beats and rhythms using their own sounds and, in turn, the AI will continuously generate new rhythms. Come join Nao and other attendees for this fun and collaborative experience!
Dive into sounds of the landscape compiled together in an immersive AI generated electronic track.