Harmonizing Multilingual AI & Human Voices: The Art of Technology

By Talia Zur Baruch, GlobalSaké & LocLearn Founder

"The human voice is the most beautiful instrument of all, but it is the most difficult to play." (Richard Strauss)

In March, I had a business trip from San Francisco to Frankfurt, Toulouse, and Tel Aviv, traveling through the time machine--fast forward into an eternal sunset, then rewinding back to an endless sunrise, spinning round the globe. 

It puts things in perspective when we teleport ourselves from one geo environment to another. Beyond the language and culture differences, our biology responds to the new ecosystem. The air, the water, the soundtrack, what the local people care about—all that instantly switches, in the blink of a clicker, when we land in a new place. 

Brutalism vs Classicism

One of the striking differences that hits you when you move from one geo to another is architecture. You can understand people on a deeper layer from the context of the buildings they breathe in. Toulouse, an ancient fort city of 2,000 years (8th century BC), is draped in pink brick and white stone with 18th century Neoclassical architecture--Greek Doric and Roman order columns, decorative rounded details inspired by Nature’s organic shapes.

Click to Tel Aviv, the sea-salt city that never sleeps. Its architecture responded to the devastation brought by the two world wars with Bauhaus in the aftermath of WW-I and Brutalism post WW-II. (Tel Aviv is officially a UNESCO World Heritage Bauhaus City). It sought to rebuild politically, socially, and physically, with revolutionary idealism and an optimistic trust in mechanical technology.

In the face of horror and destruction, these buildings responded with the beauty & durability of brut materials, like exposed concrete. Shifting away from decorative elements to bare bone, minimalist aesthetics. Function over Form. My mother, Barbara Zur, was a renowned Bauhaus Architect & Town Planner, designing resilient neighborhoods for people to breathe in. 

Which reminds me of the best film I saw this year: “The Brutalist”—Monumental, must see! And here’s also how I tie this prelude to the theme of this newsletter—"Harmonizing Multilingual AI & Human Voices: The Art of Technology” (thank you kindly for your patience!)

In “The Brutalist,” filmmakers leveraged Respeecher AI tool to refine the Hungarian dialogues of lead actors Adrien Brody and Felicity Jones. The tool blended their voices with the native pronunciation of editor Dávid Jancsó, enhancing specific phonetic nuances to ensure authenticity for Hungarian-speaking audiences. Although Brody, whose mother is Hungarian, speaks the language, rendering an authentic Hungarian accent in the English dialogues was challenging.

The deployment of AI in "The Brutalist" sparked controversy over the impact of technology on artistic expression. It fueled a wider discussion about the ethical implications of AI in creative fields, questioning the balance between technological advancement and the preservation of artistic integrity.

That said, modifying actors' voices has been a long-standing practice in the industry. And when used thoughtfully, AI technological interventions can enhance, rather than detract from the actor’s performance. By employing technology as a tool to amplify artistic expression, we can achieve a harmonious blend of human creativity and innovative tech, enriching the artistic experience without overshadowing the integral human contribution.