Fluent, confident, and wrong: the silent failure in machine translation
- Date —
- July 24, 2026
- Tags —
- low-resource languages, safety
A woman from Kabul sits across a desk from a caseworker in Zurich. She speaks Dari. The translation app on the desk is set to “Persian” — the only option it offers. It listens, and it produces smooth, grammatical, confident output.
Some of it is wrong. Not garbled — wrong. Dari and Iranian Farsi share a script and a family tree, but they have diverged in vocabulary, pronunciation, and register. A system trained overwhelmingly on Iranian Farsi will hear Dari through a Farsi filter: it substitutes the Iranian word, normalizes the Afghan idiom, smooths the sentence into something a Tehrani would say. The output reads beautifully. It just isn’t quite what she said.
And here is the part that makes this a design problem rather than a quality problem: nobody in the room can tell. The caseworker doesn’t speak Dari. The woman doesn’t speak German. The one property that would expose the error — a listener who knows both sides — is exactly what’s missing whenever machine translation is used at all. Researchers studying translation tools in asylum contexts have made this point precisely: because each party speaks only one language, mistakes have an unusually high chance of going uncaught.
This isn’t hypothetical. In one documented US case, a machine-translation tool swapped the pronouns in an Afghan refugee’s written statement from “I” to “we.” Her asylum application no longer matched her interview, and the claim was denied on that discrepancy. A crisis-translation collective working on Afghan cases estimated that a large share of the applications one of its translators handled contained machine-translation problems. The failure mode is never a crash. It’s fluency, in the wrong direction.
Sibling collapse
Dari–Farsi is one instance of a pattern we call sibling collapse: a low-resource language gets silently absorbed into a better-resourced relative. Kurdish is the starkest case. Kurmanji and Sorani are effectively different languages written in different scripts — Latin and Arabic — with limited mutual intelligibility. A system offering “Kurdish” as a single setting has already failed, before a word is spoken. Maghrebi Arabic (Darija) diverges sharply from the Modern Standard Arabic most models are trained on, and code-switches heavily with French; an MSA-trained system doesn’t refuse Darija, it mistranslates it politely.
The research community, notably, does not make this mistake. The FLORES-200 benchmark and the NLLB translation project assign these languages distinct identifiers: Dari is prs, Iranian Persian is pes; Kurmanji is kmr, Sorani is ckb; Moroccan Arabic is ary, Standard Arabic is arb. On paper, these are settled distinctions. In deployed consumer systems, they mostly aren’t: speech recognition in mainstream platforms typically offers a single “Persian,” no Dari, and thin-to-no Kurdish — and automatic language detection, where it exists, resolves ambiguity toward the dominant sibling, which is precisely the wrong default for the speaker who needs it most.
Why does the collapse happen? Data scarcity, in two layers. There is far less clean training text for Dari than for Farsi, so models lean on the sibling. But the deeper scarcity is evaluators: for many of these languages there are not enough native-speaker assessors to even measure whether output is good. You cannot benchmark your way to quality in a language where you can’t build a trustworthy benchmark. That second gap is where most of the real work is.
What we do about it
poly is built around the assumption that this failure mode exists and must be engineered against, not hoped away.
Language is forced, never detected. Our routing treats Dari and Farsi — and Kurmanji and Sorani — as distinct routes by construction. There is no code path in which one collapses into the other, because the sibling is never a fallback.
The transcript is the safety surface. Every speaker sees their own words as the system heard them, on screen, before anything else happens. Mishearing — the first place sibling collapse strikes — becomes visible to the one person who can catch it: the speaker.
Quality claims come from native speakers. For low-resource languages we publish per-language quality ratings only when we have evidence, and evidence means evaluation with people who actually speak the language. Where the system struggles, the rating says so.
None of this makes a low-resource language high-resource. What it does is convert a silent failure into a visible one — and a visible error in a conversation is recoverable in a way an invisible one never is.
If you speak Dari, Sorani, Kurmanji, Tigrinya, or Darija and want to help us evaluate translation in your language, we’d like to work with you — see Collaborate.
Sources: Rest of World, “AI translation jeopardizes Afghan asylum claims” (2023); Thomson Reuters Foundation / Context, “AI’s ‘insane’ translation mistakes endanger US asylum cases” (2023); FLORES-200 / NLLB language codes (Meta AI, openlanguagedata.org).