Nvidia unveiled Fugatto for music, audio generation 

Nvidia on Monday introduced a new artificial intelligence model, “Fugatto,” designed to create music and audio, including modifying voices and generating unique sounds.

According to reports, “fugatto” technology is aimed at the producers of music, films, and video games. However, Nvidia stated that it has no immediate plans to publicly release the technology.

The Fugatto model, a short form for Foundational Generative Audio Transformer Opus 1, is a leap forward in audio AI technology. Unlike many AI models, it can create both unique sounds from text descriptions and modify existing audio recordings, including novel sounds such as making a trumpet bark like a dog.

What makes it unique from other AI technologies is its ability to take in and modify existing audio, for example by taking a line played on a piano and transforming it into a line sung by a human voice, or by taking a spoken word recording and changing the accent used and the mood expressed.

Bryan Catanzaro, vice president of applied deep learning research at Nvidia stated, “If we think about synthetic audio over the past 50 years, music sounds different now because of computers, because of synthesisers.”

“I think that generative AI is going to bring new capabilities to music, to video games, and to ordinary folks that want to create things.”

While companies such as OpenAI are negotiating with Hollywood studios over whether and how AI could be used in the entertainment industry, the relationship between tech and Hollywood has become tense, particularly after Hollywood star Scarlett Johansson accused OpenAI of imitating her voice.

Nvidia’s new model was trained on open-source data, and the company said it is still debating whether and how to release it publicly.

“Any generative technology always carries some risks, because people might use that to generate things that we would prefer they don’t,” Catanzaro said. “We need to be careful about that, which is why we don’t have immediate plans to release this.”

Creators of generative AI models have yet to determine how to prevent abuse of the technology, such as a user generating misinformation or infringing on copyrights by generating copyrighted characters.

REUTERS/Chidimma Gold