Home > AI

AI piano duet – Music AI Google Experiment explained

AI duet piano Google Experiment by Yotam Mann.

Reviewed By: Kevin Pocock

Last Updated on September 5, 2023
Google AI piano duet
PC Guide is reader-supported. When you buy through links on our site, we may earn an affiliate commission. Read More
You can trust PC Guide: Our team of experts use a combination of independent consumer research, in-depth testing where appropriate - which will be flagged as such, and market analysis when recommending products, software and services. Find out how we test here.

AI is cropping up in all sorts of tools this year. Magenta is an open-source AI music research project that aims to be the foundation of many more. Despite existing since 2017, and not being an official Google project, the AI duet piano experiment published by Google employee Yotam Mann is one such example.

What is the AI that plays the piano with you?

The AI that plays piano with you, also know as duetting, is called “AI Duet“. The aptly named instrument was published as a Google experiment in 2017 by developer and computer scientist Yotam Mann.


AMD Ryzen 7 9800X3D launches today!

AMD has finally launched the highly anticipated 9800X3D, vowing for gaming CPU supremacy for another year. Below are the latest listings we can find

Prices correct as of November 7th, 2024.


Essential AI Tools

More Deals Coming Soon!

You can play along on desktop by clicking the keys or by typing with the center row of the keyboard. Users can also play the piano keys on mobile by tapping the screen.

What is the AI duet Google experiment?

The AI duet Google experiment is an artificial intelligence that uses the Magenta AI model to duet the user on a digital piano.

Specifically, it TensorFlow, Tone.js and open-source tools from the Magenta project in the back-end. The front-end client then “creates short MIDI files using the players’s input which is sent to a Flask server. The server takes that MIDI input and “continues” it using Magenta and TensorFlow”. This continuation follows the western rules of contemporary music, assuring that the duet will be in key with the user. As a result, it feels as though you are playing a piano duet with a computer.

Google Creative Lab hosts many such experiments. By creating its own organic melodies, Mann hopes the serendipity of new experiences will benefit a real pianist. More than just a fun toy, the art project can generate musical concepts and rhythmic patterns that will inspire written music or a human performer.

This incredible phone app even supports MIDI keyboard input using Web Midi API and the WebMIDI library.

How does the AI piano duet work?

It all starts with the neural network. As with any artificial intelligence, Magenta uses a large dataset of examples – in this case auditory, musical, and visual art. The Magenta neural net can then invent a unique output similar to those examples, by learning the commonalities through machine learning.

This AI research is part of an ongoing push for independent developers to continue the fascinating exploration of the creative side of AI. Thanks to Google’s open-source code, computational music is seeing a boon comparative to other verticals in the world of AI. None as much as NLP (Natural Language Processing) thanks to ChatGPT, but that almost goes without saying.

What AI does Google use?

Google’s current most powerful artifical neural network is called PaLM 2. This is the LLM (Large Language Model) that powers Google Bard. However, the Californian technology giant is developing multiple models for multimodal capabilities. Neural network technology can apply to any form of digital media, whether that be text, audio, image, or video. In other words, AI can make books, music, art, and films.

Google’s other musical experiments are powered by MusicLM, the big tech firm’s AI text-to-music generator.

Steve is an AI Content Writer for PC Guide, writing about all things artificial intelligence. He currently leads the AI reviews on the website.