AI piano duet – Music AI Google Experiment explained

AI duet piano Google Experiment by Yotam Mann.

Google AI piano duet

You can trust PC GuideOur team of experts use a combination of independent consumer research, in-depth testing where appropriate – which will be flagged as such, and market analysis when recommending products, software and services. Find out how we test here.

Last Updated on

AI is cropping up in all sorts of tools this year. Magenta is an open-source AI music research project that aims to be the foundation of many more. Despite existing since 2017, and not being an official Google project, the AI duet piano experiment published by Google employee Yotam Mann is one such example.

What is the AI that plays the piano with you?

The AI that plays piano with you, also know as duetting, is called “AI Duet“. The aptly named instrument was published as a Google experiment in 2017 by developer and computer scientist Yotam Mann.

Essential AI Tools

Editor’s pick
Only $0.00019 per word!

Content Guardian – AI Content Checker – One-click, Eight Checks

8 Market leading AI Content Checkers in ONE click. The only 8-in-1 AI content detector platform in the world. We integrate with leading AI content detectors to give unparalleled confidence that your content appear to be written by a human.
EXCLUSIVE DEAL 10,000 free bonus credits

Jasper AI

On-brand AI content wherever you create. 100,000+ customers creating real content with Jasper. One AI tool, all the best models.


10x Your Content Output With AI. Key features – No duplicate content, full control, in built AI content checker. Free trial available.

Experience the full power of an AI content generator that delivers premium results in seconds. 8 million users enjoy writing blogs 10x faster, effortlessly creating higher converting social media posts or writing more engaging emails. Sign up for a free trial.


Create SEO-optimized and plagiarism-free content for your blogs, ads, emails, and website 10X faster. Start for free. No credit card required.

You can play along on desktop by clicking the keys or by typing with the center row of the keyboard. Users can also play the piano keys on mobile by tapping the screen.

What is the AI duet Google experiment?

The AI duet Google experiment is an artificial intelligence that uses the Magenta AI model to duet the user on a digital piano.

Specifically, it TensorFlow, Tone.js and open-source tools from the Magenta project in the back-end. The front-end client then “creates short MIDI files using the players’s input which is sent to a Flask server. The server takes that MIDI input and “continues” it using Magenta and TensorFlow”. This continuation follows the western rules of contemporary music, assuring that the duet will be in key with the user. As a result, it feels as though you are playing a piano duet with a computer.

Google Creative Lab hosts many such experiments. By creating its own organic melodies, Mann hopes the serendipity of new experiences will benefit a real pianist. More than just a fun toy, the art project can generate musical concepts and rhythmic patterns that will inspire written music or a human performer.

This incredible phone app even supports MIDI keyboard input using Web Midi API and the WebMIDI library.

How does the AI piano duet work?

It all starts with the neural network. As with any artificial intelligence, Magenta uses a large dataset of examples – in this case auditory, musical, and visual art. The Magenta neural net can then invent a unique output similar to those examples, by learning the commonalities through machine learning.

This AI research is part of an ongoing push for independent developers to continue the fascinating exploration of the creative side of AI. Thanks to Google’s open-source code, computational music is seeing a boon comparative to other verticals in the world of AI. None as much as NLP (Natural Language Processing) thanks to ChatGPT, but that almost goes without saying.

What AI does Google use?

Google’s current most powerful artifical neural network is called PaLM 2. This is the LLM (Large Language Model) that powers Google Bard. However, the Californian technology giant is developing multiple models for multimodal capabilities. Neural network technology can apply to any form of digital media, whether that be text, audio, image, or video. In other words, AI can make books, music, art, and films.

Google’s other musical experiments are powered by MusicLM, the big tech firm’s AI text-to-music generator.

Steve is the AI Content Writer for PC Guide, writing about all things artificial intelligence. He currently leads the AI reviews on the website.