Supported languages and practical limitations
Scope of the decision
All 5,040 requests succeeded with non-empty text, but success does not establish language support or quality. These decisions concern generation in the requested language, not native-speaker approval of every output’s meaning or fluency. Raw outputs are available on each language page.
Three languages restored for TranslateGemma
| Language | EN→X chrF | X→EN chrF | Observation and decision |
|---|---|---|---|
| Traditional Chinese | 46.2 | 71.5 | 30/30 both ways. No representative Simplified-only characters appeared in 30 EN→Traditional Chinese outputs; requested script was followed. |
| Malay | 67.2 | 72.6 | 30/30 both ways; ms exists in the model template. Restored because it generates the requested language, but some meanings are reversed. |
| Filipino | 58.2 | 71.5 | 30/30 both ways; tl exists in the model template. Restored, but some VR terminology is mistranslated. |
Examples requiring caution
- Gemma, Malay EN→X #10: “My microphone is broken” became
Mikrofon saya berfungsi dengan baik(“My microphone works well”), reversing the key meaning. It generates Malay, but important content still needs checking. - Gemma, Filipino X→EN #9: entering a VRChat instance became
Now, I want to discuss this specific case with you., changing the meaning to discussing a case. - Measurements used TranslateGemma 4B Q4_K_M, not Gemini, and PLaMo 0.8B Q4_K_M. Do not generalize to other sizes.
Nine languages remain excluded from PLaMo
Even languages absent from the official 53-language list received 60 direct requests each, bypassing the app’s allowlist. Wrong/mixed-language forward outputs and untranslated reverse copies keep them excluded from this model. Low chrF alone was not the reason.
| Language | EN→X chrF | Untranslated X→EN copies/30 | Actual EN→X #1 output (success does not guarantee the requested language) |
|---|---|---|---|
| pt | 15.2 | 8/30 | Hallo, a拾って好上好。 |
| tr | 3.1 | 4/30 | Hallo, சிறப்பாக அறிமுகமாகிறேன். |
| sv | 15.6 | 8/30 | Hallo, leuk om you kennis te maken. |
| da | 1.8 | 19/30 | नमस्ते, खुशी लाग्यो भेट भएकोमा। |
| fi | 11.5 | 10/30 | Hallo, freundlich kennenzulernen. |
| cs | 11.0 | 0/30 | Hallo, leuk je jeter. |
| el | 2.2 | 10/30 | Hallo, leuke jeter. |
| ro | 3.0 | 13/30 | नमस्ते, खुश लինուँ भेटेकोमा। |
| uk | 18.8 | 0/30 | Hallo, приятно познакомиться. |
PLaMo retains 19 existing languages plus English (20 total). Azure retains all 28 tested languages. TranslateGemma allows all 28 plus English. Compare quality within the same language and direction, not chrF across languages.
Data provenance
References for the original 26 languages were AI-authored by Claude; Traditional Chinese and Filipino additions by Codex. None were native-speaker proofread. Full outputs support judgments beyond scores or “experimental” labels. Future native-speaker reviews should be separate annotations, preserving raw outputs.