Communicating with customers and patients who speak a language other than English presents a daily challenge for a lot of people in the U.S. This is especially true in real-time conversations, when accuracy, speed, and clarity are essential, like in healthcare.
Recent advancements in AI have led to a flood of AI-generated translation devices on the market claiming to do marvelous, near error-free work.
While their features and price points — from $40 to $500 — seem enticing, the real question remains: Do these devices actually work as promised?
We were curious, so we tried a few out. Here are our findings.
The Devices We Tested
- Languages: 138 (offers offline translations in 16+ popular languages)
- Accuracy claim: up to 98%
- 14 app functions: online translation, offline translation, photo translation, phone translation, group translation, enter translation, smart recording, simultaneous interpretation, language learning, ChatGPT, unit conversion, exchange rate conversion, AI assistant, and multimedia
- Languages: 144 (and accents)
- Accuracy claim: up to 97% in eight languages
- 5 translation modes: touch mode, speaker mode, free talk mode, offline mode, and photo translation
Timekettle M3 Language Translator Earbuds
- Languages: 40 and 93 accents
- Accuracy claim: 95% maximum. The box did have a disclaimer stating accents, noise, and different expressions can affect the accuracy. The 95% accuracy claim is only applied to some languages, not all.
- 4 translation modes: touch mode, listen mode, speaker mode, and offline mode
How We Chose and Tested Each Translation Device
We selected the Timekettle M3 Translator Earbuds based on their popularity via Amazon reviews. It was the highest rated of the earbud-based devices. We chose the Anfier M3 Translator Earbuds based on their likeness to the Timekettle device and the fact that they had the second-highest rating on Amazon. The third, S80 AL Translator, functions similarly to the other two, but it’s a handheld device as opposed to an earbud-based model.
We tested all the features on each device and selected the mode that performed the most consistently across all three for comparison. The mode we chose closely resembled simultaneous interpretation.
Length of statements, what we were looking for, and languages tested
For the AI comparison testing, we chose three different languages from three different language groups: Russian (Cyrillic), Chinese (double-byte), and French (romance).
We tested both longer statements (20 seconds in length) that were more conversational and statements that were more straightforward with shorter sentences.
Accuracy and meaning were of course important, but how easy the device was to use almost mattered more. (If you can’t get it to work properly, who cares how accurate it is!?) So we made note of how easy it was to change modes and languages, speaker volume, delay, and the like.
How These Translation Devices Work
It’s important to remember the difference between interpretation and translation. Interpretation focuses on spoken or signed words, while translation handles written content.
These devices record speech, convert it to text, translate the text, and then voice the translation in the desired language.
Examples and Analysis of AI Translation by Device
S80 AL Translator
Mode used: Simultaneous interpretation
French statement to English
Original: My name is Juliet Dupont. I am 28 years old. After my diploma, I worked for 3 years as an assistant in a small hotel in Bordeaux. My responsibilities were to welcome customers at the reception.
AI Translation: Diploma, I worked for 3 years in a little boxing to welcome customers to disappointment.
What stuck out to us immediately was the delay in catching the first few seconds of our statements, as evidenced by the French translation above. This delay is likely caused by the device’s need to initialize its speech recognition system, which takes a moment to process before capturing the audio input. Essentially, the device “listens” for a second to understand the speech patterns before it begins interpreting, similar to how buffering works in video streaming.
It struggled with sentence complexity, especially with longer or idiomatic phrases, and often mistranslated key elements (e.g., translating “hotel” as “boxing” in the French example). The delay in recognizing speech created awkward pauses in conversation, which made real-time use difficult.
Once you get past that initial lag, however, it performs well on shorter statements.
Anfier M3 Translator Earbuds
Mode used: speaker mode
Chinese statement to English
Original: Hello, friends! Welcome to my channel! I’m Zhi. The classes at my school have already ended this week. Now I’m starting to look for a job. Last week I submitted some resumes. This week, I have two interviews. I’m really looking forward to these two interviews. Because these are after submitting my resumes, received the fastest two interviews.
AI Translation: Hello friends, welcome to my channel, I’m Shi, this week my school classes have ended, now I’m looking for a job, last week I submitted some resumes, there are 2 interviews in this state, I am very much looking forward to these two interviews, because these are the 2 fastest interviews I received after I sent out my resume.
We tried speaker mode first, but it took a while before our phone’s speaker could pick up the statement. Once it picked up the voice, it did not translate anything.
We switched the device to a different mode, hoping to trigger a reset, and then retried speaker mode. This time, the phone speaker did pick up the statement, but the translations were poor, slow, and jumbled. They did not make a lot of sense. We also repeated the same statement twice, and the translation was different both times.
At times, the speaker input would stop while the person was speaking, which caused the translation to fall behind. We weren’t sure why this happened, exactly, but we guessed it was from the device picking up an irregular speech pattern, like a pause, and mistakenly assuming the person had finished speaking, causing it to stop the translation.
When the Anfier did catch what was said, it was about 90% accurate; however, it did not capture emotional nuance, and what was missed changed the meaning of the sentence.
Timekettle M3 Language Translator Earbuds
Russian statement to English
Original: A few months, but I have noticed that over the past month it has become more difficult for me to breathe. Sometimes I have the feeling that I can’t catch my breath.
AI Translation: A few months, but I have noticed that over the past month it has become much more difficult for me to breathe. Sometimes I have the feeling that I don’t know my breasts. How to say closed.
Overall, the Timekettle was easier to use and better designed than the other models. The translations were better, too, but the device still struggled with longer statements and context; for Russian, shown above, the Timekettle used the word “breasts” incorrectly in the sentence about breathing. This error seems like it’s from the AI’s struggle with nuanced phrasing and context, showing a gap in understanding more complex or idiomatic language.
Like the others, the translation was much clearer when the statements were much shorter. During testing, single sentences and short statements had near perfect accuracy. These statements were all written statements our testers were reading, and the delivery was clear with no pauses or adjustments during speech.
Our Overall Impression of AI Translation Devices
On accuracy
For all tested languages, the accuracy did not vary much. The AI was always able to achieve higher accuracy with simple, short sentences that use “proper” grammar.
Across all devices, longer and more conversational statements led to a significant drop in accuracy, while short, clear sentences resulted in better performance.
When the accuracy was decent, however, we found that the AI spoke so fast that we had to play the translation three times before we could comprehend what it was saying.
Additionally, with medical terminology, the AI tried to translate certain terms that, as a human medical interpreter can attest, need to be kept in their original language, especially when they are the Latin names (Candida, for example).
On overall usability
The devices had significant usability issues. All the devices cut off speech, missed information, or refused to register the speaker. The AI device was impatient regardless of modality, leading to a frustrating experience.
The instructions were unclear and the devices themselves finicky. The S80 had lags, screen issues, and required absolute silence to pick up the voice. The Timekettle and Anfier, though slightly better, still experienced lags, crashes, and interruptions when the speech recognition failed.
Even if everything worked technically perfectly, the AI spoke very quickly with a harsh, robotic, impersonal voice.
When to Use AI Translation Devices
Do these devices have imperfections? Yes.
Does that mean there are no uses for them? Of course not.
While the uses are not robust, the AI devices are cheap enough and widely available that they could be an excellent addition to certain businesses or industries, or for personal use.
The tested devices failed when the speaker had a conversational tone but were excellent with small, concise sentences regardless of subject matter.
They worked better and were made for two people in the same room talking to each other. AI translation devices could be used for instructions that are already written out or practiced updates that are given on a regular basis, for example.
They also could be a great stopgap to give to individuals in your community with a non-English language preference so they have some ownership in their communication. The devices reduce wait times and provide continuity that human interpreters cannot always do.
It’s always best to follow the NCIHC guidelines when exploring if an encounter can be serviced by an AI translator over a human interpreter.
If you’d like more content like this without surfing the web, subscribe to our newsletter!
