What is the code-mixing tax?

// definition

The code-mixing tax is the accuracy a language model loses when a question is written in mixed or romanized script rather than in a single native script.

It is measured by asking the same question in five forms and comparing the results:

// the central finding

Writing in Latin letters costs more than mixing languages does.

This is the counterintuitive result and the reason the metric exists. Intuition says that mixing two languages in one sentence should be the hard case. In our 2026 measurement across 16 models, the switch from Devanagari to Latin script consistently costs more accuracy than the switch from pure Hindi to mixed Hinglish.

For hundreds of millions of people who type Indian languages in Latin script on a phone keyboard, that is the form their questions actually arrive in.

// floor-adjusted retention

Raw accuracy loss overstates how well weak models hold up, because a model scoring near the guessing floor in English has little left to lose.

Floor-adjusted retention corrects for this by measuring how much of a model's headroom above the guessing floor survives the script change. It is the figure to compare across models.

The difference is substantial. A model dropping from 32.13 to 25.00 looks like it retained 77.8% by the naive measure, when in fact it had almost nothing above the floor to begin with.

// what the 2026 run found

16 models, 1,024 general-knowledge questions and 955 maths word problems, each asked in all five forms.

Retention on romanized Hindi ranged from 87.2% at the top to 18.2% at the bottom of the computable range. Several models had English baselines too low for retention to be computed at all.

The strongest results came from models built in India. The weakest came from models with high English scores and no mixed-script robustness, where a 96.13 English maths score fell to 48.80 on romanized Hindi.

Full results are on the code-mixing tax report.

// why it is a deployment concern rather than a curiosity

Any consumer product taking free-text input from Indian users will receive romanized input, unprompted and in volume. A model evaluated only in English or only in native script will look substantially better in testing than it performs in production.

// where the term comes from

Code-mixing is a standard linguistic term. The framing of the measurable accuracy loss as a tax, and the floor-adjusted retention measure, are Sthānika AI's.

// citation

Sthānika AI (2026). The code-mixing tax. https://sthanika.ai/research/codemix-tax-2026