Rosetta Stone for iPhone adds AI to identify objects for live translations
Language teaching developer Rosetta Stone adding augmented reality and machine learning to its to enabling users to identify and translate the names of real-world objects.
Historically, language translation apps were confined to whatever developers could store in their on-board databases, limitations that were surpassed with internet connections and AI-assisted text recognition tools. Today, language teaching developer Rosetta Stone is taking the next step forward by adding augmented reality and machine learning to its iPhone app, enabling users to identify and translate the names of real-world objects — the first time object recognition technology has been used in a language app in this manner.

The machine learning technology is a new component of Rosetta Stone's Dynamic Immersion feature, which lets the user simulate being in a foreign language environment, surrounded by new words, phrases, and native speakers' voices. A scavenger hunt-like game called Seek & Speak has the user point the iPhone camera at an object and see the vocabulary word in a chosen language, then practice a conversation using the word. Seek & Speak is launching in beta for English, French, German, Italian, and Spanish, with plans to add additional languages and challenges during 2019.

These five languages are also getting two more improvements: Embedded Translations, enabling Dynamic Immersion users to long-press any word during the experience to reveal its translation, and an updated Phrasebook — an offline collection of over 164 spoken phrases that can be used in common situations. Rosetta Stone also says it's using machine learning to continuously improve the performance of its TruAccent speech recognition feature, leveraging thousands of hours of recorded speech data to help users match their pronunciation of words to that of native speakers.

"I'm incredibly proud of our team of software developers behind this product. Their use of augmented reality and machine learning creates an incredibly rewarding learning experience for our customers. Our iOS mobile app brings language immersion to the next level, with technologies that are only now widely available to consumers through mobile phones."

Matt Hulett
Rosetta Stone president
Rosetta Stone for iPhone now supports 30 languages, up from a prior 24, though features vary based on the language. Some of the features depend on an active internet connection, but others — including Phrasebook — are stored within the app so they can be used even when you're traveling abroad without data services.

Made on