What Does Generative Music Look Like?

The Evolution of ever.fm’s “Mandala” Visualizer, from stick-figures to beautiful art pieces.

Seeing is Hearing

On ever.fm, what you see is what you hear; every rendition is accompanied by a beautiful visualization. These “sound mandalas” serve two important functions: 1) they beautify the ever.fm experience, and 2) they help you make sense of what you’re hearing.

At first, the colourful, circular visuals might seem random, but they’re far from it. They’re generated from the audio data in a song and show a circular “timeline” of the piece, each concentric ring representing a different track in the master recording. Furthermore, the artist is able to set a number of parameters that affect how the visualizer will behave. In this way, the visuals are truly an extension of the artist’s vision.

Dolls, Blobs and Stick Figures

When I first approached Andreas, my co-founder, about creating a visualizer for ever.fm, my ideas were, to put it mildly, a hot mess. I knew we needed visuals so that listeners could see the change from one rendition to another (even if they might not always hear the change), but that was pretty much it.

My initial ideas look pretty hokey in retrospect: I imagined a kind of side-scrolling video-game animation in which a figure would walk through a landscape, the background of which would represent the waveforms on each track. Seriously. That was my initial idea. We still roll our eyes about it.

One of my initial sketches for the visualizer. 🙄

I also brainstormed a few other possibilities: a doll-like figure whose outfit changes with each rendition of a song, a zodiac-looking arch with “constellations” generated from the audio data, and a “blob” that would float on-screen and change shape based on a rendition’s audio properties. We had a long way to go but we knew two important things: 1) we needed a visualizer, and 2) the visuals should be generated from the waveform data in a song. It was a start.

At some stage, our approach became (mercifully) more abstract. This was largely Andreas’ influence, as I recall. After a few of weeks of meetings, the dolls and stick-figures disappeared from our sketches and were replaced by concentric rings and line segments. We started a mood board and filled it with colourful, geometric, mid-century abstract art.

Early sketches for the mandala, exploring how each ring could represent part of the master recording.

On our weekly calls, we debated: how abstract is too abstract? How literal is too literal? How do we make each rendition unique and yet make them like they belong to a “family” of renditions? How do we ensure that every rendition is beautiful? Or do we? We were inspired by Tyler Hobbs’ work and devoured his essay on “long-form” generative art.

More sketches. We’ve come a long way from dolls and stick figures…

We took stock. Here’s what we had so far:

  • Circular design
  • Each ring represents an audio track
  • Allow artist to customize colours

At this point Andreas took over. He made quantum leap after quantum leap, putting together the visual editor in just a few weeks. Clarity is magic.

The final touch: animating the visual so that it spins like a post-modern record as the song plays. VoilĂ ! The story of the ever.fm visualizer. I’ll admit, I really love the results. I love that the visuals are flexible enough to evoke an artist’s unique vision and personality. I love that the visuals are engaging enough to pull the listener into a piece of music but abstract enough that they don’t distract from the listening experience. I love that ever.fm is an art gallery of sorts, an online audio-visual Guggenheim where music and art are beautifully entwined.

There are still many avenues to explore when it comes to visualizing generative music. Beyond spinning like a record, what other animations could help to bring the visuals to life? Could we add animated transitions between renditions to make the UX more seamless? How do ever.fm visuals interface with AR and VR? There’s always room to grow, dream and be creative. But for now, the ever.fm visualizer gives artists a ton of room to play, explore and express new facets of their songs.

Um… Why Does ever.fm Exist?

What’s the point of ever.fm? Why create recordings that change?

For thousands of years, songs were ephemeral and ever-changing. Songs were carried on the wind, growing like wildflowers wherever they landed. For thousands of years, songs were alive. Then, suddenly, they were turned to stone.

Edison may not have set out to create the record industry (after all, his machines were designed to replace stenographers, not musicians) but, nevertheless, the phonograph cast a powerful spell and there was no going back.

For thousands of years, songs were alive. Then, suddenly, they were turned to stone.

For the first century of its existence, the value of recorded sound followed a general upward trend, peaking in the mid-1990s with the convergence of cassettes and CDs. Then, in 2002, came Napster, the dawn of digital music sharing and two decades of economic tailspin for the music industry. Music-industry revenue declined sharply from a peak of $21.5 billion in 2000 to $6.9 billion in 2015. Since then, the industry has seen growth due to the mass adoption of streaming music services like Spotify, Apple Music and Tidal. However, the market value of a recording has changed forever: Spotify currently pays rights holders a fraction of a cent per stream. An artist must garner over 1,000 streams of a song just to buy a coffee.

An artist must garner over 1,000 streams [on Spotify] just to buy a coffee.

Success in streaming hinges on one thing: volume. Because of the way Spotify’s payouts are structured, artists with millions (or billions) of streams get a disproportionate cut of streaming revenues. That obscure band you binge-listened to all month long? They probably didn’t get penny from your subscription fee; it all went to Drake and Taylor Swift. While there is a movement among streaming services like Deezer and Tidal to adopt a user-centric payout model that might compensate artists more fairly, this approach has not been taken up by industry-leader Spotify.

“[Streaming] works if you’ve got thousands or millions of songs,” says Mark Mulligan of tracking firm MIDiA Research. “But if you’ve only got 20 or 30 or 100 songs then it doesn’t. You need scale of catalog to benefit.” Big artists get bigger, small artists get smaller. Without a drastic re-imagining of the recorded-music medium, the dream of the independent musician, fostered by 1990s-era optimism, is dead.

So let’s re-imagine it.

In the beginning was the wax cylinder. Then came a succession of physical formats: vinyl, 8-track, cassette, and CD. Finally, music was set free from its physical form with the advent of the mp3. Throughout this metamorphosis, one thing remains constant: recordings never change. Play, stop, repeat is all we ever get. It’s as true today as it was in 1877. One listen is like every other; it makes no difference if I listen now or later. There is little urgency or scarcity in this world of static music. More than ever, music has become the proverbial “wallpaper” in our lives: ubiquitous and forgettable.

But what if recordings could change? What if every listen was non-fungible? What if we realize the promise of disembodied music? Before the existence of streaming, blockchain and NFTs, this was unthinkable. Now, it may be a way to rebuild the value of music in the web3 era.

What if recordings could change? What if every listen was non-fungible?

ever.fm has created a new format for recorded music, one in which a song is reborn every time it plays. It’s streaming music with superpowers. As a song plays, the listener can shuffle the inner workings of the music with the touch of a button, cycling through myriad possible renditions until they find the one they love. These renditions are generated on-the-fly by ever.fm using two ingredients: audio samples and playback rules, both of which are created by the artist. With every rendition comes new discoveries: never-before-heard combinations of instruments, new lyrics, harmonies, alternate takes and more.

On ever.fm, a song is the sum of its renditions the way a flip-book is the sum of its pages. There is no definitive version of a recording. Instead, every song is a collection of possibilities, and each possibility is an NFT that can be collected, shared and re-sold.

On ever.fm, songs are alive.