Google’s new AI-powered ‘Piano Genie’ lets you improvise melodies by pressing random buttons

AI-based tech initiatives are making great strides in fields such as art and music. The most recent example is the Piano Genie, an AI program from Google’s creative research team Magenta. The program allows you to extemporize on the piano by continuously pressing away at eight buttons.

Previously, Google created another  interactive web-based app that accompanies your piano riffing. The Piano Genie is an extension of Google’s AI-based product line.

The product team was inspired by Guitar Hero, a popular video game that provides a simpler method to playing a musical instrument. Unlike the Guitar Hero however, Piano Genie’s team wanted the user to make up melodies on the spot, rather than simply play along to prewritten hit songs. To enable such a function, they collected a massive dataset of classical piano and instructed the AI program to predict what notes follow each other, similar to how your phone can anticipate which text you’ll write next.

The machine learning aspect of the program is derived from a few basic AI components. The main component is a recurrent neural network, which is exceptionally apt at learning mimic sequential data, like texting or music. The neural network was filled with a dataset of piano music from a renowned international competition. The competition recorded all performances into a file format, preserving all the notes and their velocity (timbre and volume). This made the data especially useful.

The dataset from the competition was the main material used to construct a predictable model of which piano note patterns. This may cause some setbacks since any note Genie produces will stick to specific keys or scales. However, this can easily be rectified. The musicians at the competition were performing complex and often flashy pieces, adding sophistication to the program’s dataset.

One of the creators of Piano Genie, Chris Donahue, told tech site The Verge:

“I really wanted to design a tool that we could give to someone who doesn’t know how to play, and they’d be able to create music with some kind of intention”

Donahue said that many AI programs garner entire melodies from a single starting note or chord. What sets the Piano genie apart is that It doesn’t just improvise by note, providing the user with a sense of autonomy over the subsequent melody. One of the challenges with the tech is minimizing the latency to make sure each note is instantly ready to be played. However, this also creates a special feeling for the user, says Donahue, who has been a piano player himself for the past twenty odd years.

“When you’re playing it, it’s this really awesome experience where, occasionally, it will feel like it’s sort of reading your mind and play the exact note you’re intending to,” he says. “And then other times, it will completely disobey you but still do something reasonable.

In essence, this is the origin of the program’s name. Similar to a Genie, you can wish for what you want, but what you don’t always get what you asked for.

Donahue claims that programs such as Piano genie prove that AI can elevate the boundaries of human creativity. It combines human and machine-like elements, pairing our intuitional knowledge of which notes should be played with a programming system’s ability to decipher which notes should follow.

Donahue says of the program’s new users:

“They have a tendency to timidly press a couple keys here and there at first, but then if I say, ‘Imagine you’re a concert pianist onstage at Carnegie Hall,’ they get it more and really go for it.”