Evan King

TL;DR: I’m launching the Mirage, a hardware sampler that generates new sounds in real-time using embedded generative audio models. It combines traditional sampling with voice control and audio “model bending”, giving musicians new ways to explore and create unique sounds.

I’m seeking collaborators and early testers. Contact me or sign up as a tester to help shape Mirage’s future.

Introduction

Lately I’ve been asking myself: Why is AI art boring, and how do we fix it?

AI art needs human context

Art’s idiosyncrasies signal something about the artist’s skills and limits, and about the work’s historical context. This gives it depth and meaning, which makes it interesting: works like Guernica are more than oil paint on a canvas.

Black box AI art is not shaped by a person’s artistic process, but by learned probability distributions. These distributions separate the art from the situations that created it, unless the artist shapes the training data. Projects like TimeCapsuleLLM are interesting precisely because their datasets are intentionally curated: AI practice is the creative tool rather than AI itself. Working with AI in this way requires technical knowledge and GPU compute that is inaccessible to most people.

Current AI tools limit artistic agency

Most AI tools provide few to no knobs to play with. The process instead centers around “prompting strategies”. This design philosophy seems to stem from the assumption that a creator’s goal is to minimize their effort: input a few prompts and get the finished result back. Efficiency may be the goal when generating assets for a corporate presentation, but people creating for other reasons have different needs. They create because they want to explore the outcomes of different processes for self-expression, which is often orthogonal to efficiency.

Tools that support this exploration give people the freedom to invent, break, reassemble, and ultimately own their own processes. If guitar amplifiers were so boring as to protect people from distortion or feedback, modern music would not exist. ChatGPT briefly experienced its own kind of distortion in Feburary 2024, when a bug in token prediction caused the model to generate uncanny and improbable text. Though OpenAI unceremoniously resolved the incident and ChatGPT returned to its predictable self, I have to imagine some were wondering: how can I make it do more of that?

A bug in ChatGPT makes it more interesting. Image credit: /u/Own_Secretary4549 on Reddit

Why do generative AI tools smooth over the out-of-distribution quirks that could otherwise make them more interesting? They are built on the assumption that artists want instant gratification, rather than opportunities to develop new modes for expressing their ideas.

How do we make AI art less boring?

As a researcher and a musician, I’ve been exploring this topic for several years. I’ve trained models to generate uncanny plants and insects for album art; coaxed haunting sounds of out audio models by mangling their inputs; developed poetic songwriting tools that induce hallucinations from speech recognition models. I’ve developed the take that “AI art” is definitionally boring – that is, if a black box AI model controls the entire process, the output is “AI art” and is therefore boring. It doesn’t say much about the person who made it and what their talents and limitations are as an artist. Art that involves AI as a tool that affords a human new creative agency is not really AI art, but just art. And art is not boring.

The goal is therefore not to make AI art less boring, but to make AI more compatible with human creativity. We can do this by designing tools that allow people to develop their own creative processes. These tools should be explorable, tweakable, and most importantly breakable – they can’t be overly prescriptive about their use.

The piano ain’t got no wrong notes.

–– Thelonious Monk

With a bit of hardware development, embedded speech/audio AI, and music knowledge under my belt I set out to build the creative tool I was looking for.

Introducing the Mirage

The best creative tools give you room to develop your own way of doing things within a core set of constraints. That’s the idea behind the Mirage. It’s a new breed of hardware sampler that lets musicians create new samples on-the-fly using generative audio models. It’s the first piece of musical hardware to embed generative audio AI directly in the box, without any requirement for an internet connection or subscription. Users prompt the Mirage’s audio models via a voice interface, powered by Moonshine.

Most importantly, the Mirage lets you deliberately break its audio model to create weird, glitchy sounds — I call this model bending (inspired by circuit bending). My hope is that model bending will inspire weird, uncanny, and interesting sounds that meet the moment, shaped by the touch of musicians who use it as a new tool in their creative processes.

Features

In addition to a familiar set of hardware sampler features – a step sequencer, multiple audio channels, sample slicing, FX, CV and MIDI I/O – the Mirage incorporates three new audio generation concepts. I’ll introduce each with a short demo video.

Blank-slate generation

In an empty audio channel, simply give a short prompt for a sound you’d like to hear and the Mirage generates it:

Guided generation

The Mirage can also build off of existing sounds using your text prompt as a guide. Let’s say I have a nice pad sound, and I want a sample in a certain genre with a matching timbre:

Model bending

I developed some interesting ways to glitch the model’s audio decoder while building the Mirage’s embedded audio generation. Each of these model bending parameters has a knob attached, allowing musicians to tweak the model’s behavior and generate musically inspiring outputs.

What’s next?

What I’ve shown here is the second revision of the Mirage hardware. It’s a platform I’ll continue to iterate on, with some avenues left to explore:

• User testing. The Mirage’s UX makes sense to me given workflows I’m personally used to. Now I want to understand how others use it. Does it have the right basic features? Are you able to get the sounds you want? Does it fit in your overall workflow?
• Product design. The Mirage’s electronics hardware and software are manufacturable, but the enclosure design is a 3D-printed prototype. I’d love to chat with people who have industrial design and manufacturing experience to see how – given demand – the fit and finish could be polished and moved into small-batch production.
• Partnerships. Do you work in the music hardware space? If so, please reach out. The Mirage is the first salvo in what I believe will be a rich market for hardware products with embedded audio AI.

If the Mirage has caught your interest, please contact me or sign up as a tester.

Thanks for reading!