What Does Generative Music Look Like?

The Evolution of ever.fm’s “Mandala” Visualizer, from stick-figures to beautiful art pieces.

Seeing is Hearing

On ever.fm, what you see is what you hear; every rendition is accompanied by a beautiful visualization. These “sound mandalas” serve two important functions: 1) they beautify the ever.fm experience, and 2) they help you make sense of what you’re hearing.

At first, the colourful, circular visuals might seem random, but they’re far from it. They’re generated from the audio data in a song and show a circular “timeline” of the piece, each concentric ring representing a different track in the master recording. Furthermore, the artist is able to set a number of parameters that affect how the visualizer will behave. In this way, the visuals are truly an extension of the artist’s vision.

Dolls, Blobs and Stick Figures

When I first approached Andreas, my co-founder, about creating a visualizer for ever.fm, my ideas were, to put it mildly, a hot mess. I knew we needed visuals so that listeners could see the change from one rendition to another (even if they might not always hear the change), but that was pretty much it.

My initial ideas look pretty hokey in retrospect: I imagined a kind of side-scrolling video-game animation in which a figure would walk through a landscape, the background of which would represent the waveforms on each track. Seriously. That was my initial idea. We still roll our eyes about it.

One of my initial sketches for the visualizer. 🙄

I also brainstormed a few other possibilities: a doll-like figure whose outfit changes with each rendition of a song, a zodiac-looking arch with “constellations” generated from the audio data, and a “blob” that would float on-screen and change shape based on a rendition’s audio properties. We had a long way to go but we knew two important things: 1) we needed a visualizer, and 2) the visuals should be generated from the waveform data in a song. It was a start.

At some stage, our approach became (mercifully) more abstract. This was largely Andreas’ influence, as I recall. After a few of weeks of meetings, the dolls and stick-figures disappeared from our sketches and were replaced by concentric rings and line segments. We started a mood board and filled it with colourful, geometric, mid-century abstract art.

Early sketches for the mandala, exploring how each ring could represent part of the master recording.

On our weekly calls, we debated: how abstract is too abstract? How literal is too literal? How do we make each rendition unique and yet make them like they belong to a “family” of renditions? How do we ensure that every rendition is beautiful? Or do we? We were inspired by Tyler Hobbs’ work and devoured his essay on “long-form” generative art.

More sketches. We’ve come a long way from dolls and stick figures…

We took stock. Here’s what we had so far:

  • Circular design
  • Each ring represents an audio track
  • Allow artist to customize colours

At this point Andreas took over. He made quantum leap after quantum leap, putting together the visual editor in just a few weeks. Clarity is magic.

The final touch: animating the visual so that it spins like a post-modern record as the song plays. Voilà! The story of the ever.fm visualizer. I’ll admit, I really love the results. I love that the visuals are flexible enough to evoke an artist’s unique vision and personality. I love that the visuals are engaging enough to pull the listener into a piece of music but abstract enough that they don’t distract from the listening experience. I love that ever.fm is an art gallery of sorts, an online audio-visual Guggenheim where music and art are beautifully entwined.

There are still many avenues to explore when it comes to visualizing generative music. Beyond spinning like a record, what other animations could help to bring the visuals to life? Could we add animated transitions between renditions to make the UX more seamless? How do ever.fm visuals interface with AR and VR? There’s always room to grow, dream and be creative. But for now, the ever.fm visualizer gives artists a ton of room to play, explore and express new facets of their songs.

Published by

James Hill

Co-founder of ever.fm, lifelong 'ukulele lover, father and husband. @jameshillmusiq