Transform music into something a little bit crazy with Google’s Tone Transfer tool

This year at Google I/O 2022, Google showcased a lot of breakthroughs in new AI tech and security. One of these cool advancements from a couple of years ago made its way to the stage in the pre-show. It’s called Tone Transfer from Google’s Magenta team, and it lets you turn normal instruments or sounds into something completely different.

What is Tone Transfer from Google?

Developed by the Magenta team from Google and announced back in October 2020, Tone Transfer is a pretty neat tool that allows artists to input sounds and get something completely different out of it. For instance, you can take the sound of people singing, birds chirping, or even the dull and sharp bangs of pots and pans, and turn that sound into a flute or violin.

This project is based on a machine learning process called Differentiable Digital Signal Processing (DDSP.) In one example, the team set the model up with a 10-minute recording of a violin for it to process and run through its system. After that, the team started monitoring the program’s progress and its ability to recreate that sound. After one hour of training in the example, the recreated sound is similar to a synth being played emotionlessly. After 10 hours of training, the model is able to almost indistinguishably replicate the sound of a violin. The reason for this is that the model, by this point, is able to control several different audio signatures provided by the instrument.

In all, the Tone Transfer project from Google’s Magenta is incredibly interesting and goes to show how far machine learning has come as a whole. AI being able to develop and reproduce musical instruments by itself is one incredible feat.

How to create your own sounds in Tone Transfer

Magenta at Google has made Tone Transfer available for the public to try out for themselves. You can take a sample provided in the program, or you can upload your own. After that, you get to choose what that sound gets morphed into. Here are the available end result sounds:

• Flute
• Saxophone
• Trumpet
• Violin

Simply choose a source sound and click the play icon under the sound signature to get an idea of what the source sounds like. After that, choose an output sound. Click play again to hear the sound recreated and turned into a new instrument.

If you want to upload your own sound sample, you can hit Add your own under the source sample section. If your device is compatible, you can either record a sound or upload a file from your computer. Once you’re done recording or uploading, hit Done and you’ll be able to listen to your recording. If you’re happy with it, hit Transform, and Tone Transfer will start to process the sample.

Tip: If you want to get a good result, make sure you’re choosing a sound that only has one tone playing at a time. This way, the ML model can differentiate and better create an end result.

While playing, you can hit the slider icon to tune the end result. You can adjust the mix, octave, and loudness to your liking.

In all, Tone Transfer from Google and the agent team is pretty cool. The model demonstrates how far a lot of this tech has come and provides a good example of what it can be used for. We definitely look forward to what Magenta is working on in the future.