Hindi NLP Is Hard, and That's the Point

Building a voice assistant for Hindi and Hinglish speakers exposed every assumption baked into English-first AI tooling. A technical deep dive into the Bharat Biz Agent language pipeline.

The Hinglish problem

Hinglish isn't Hindi with English words swapped in. It's a fluid code-switching language where speakers move between registers mid-sentence based on context, relationship, and topic. "Mujhe ek meeting schedule karni hai for tomorrow" is natural speech, not broken language. Most NLP pipelines treat it as broken.

What breaks first

Language detection breaks first. Standard classifiers trained on clean monolingual text assign low confidence to Hinglish and often misclassify it as noise. We ended up building a custom classifier trained specifically on mixed-register Indian business speech.

Tokenization breaks second. Hindi is written in Devanagari; Hinglish in Devanagari, Roman, or both in the same sentence. Tokenizers that work well for one script need adaptation for the other.

The transliteration layer

One breakthrough was adding an explicit transliteration step before LLM inference. Romanized Hindi ("aap kaise hain") gets converted to Devanagari before being passed to the model. This dramatically improved response quality.

What the pipeline looks like

Audio → speech-to-text (custom fine-tuned model for Indian accents) → language detection → transliteration if needed → intent classification → LLM inference in detected language → TTS in matching register.

Each step is a place where English-first assumptions cause failures. Each step required Indian-specific training data or adaptation.

Why it's worth it

The model of "AI tools are for English speakers" is wrong and it's going to be proven wrong over the next decade at scale. Building for Hinglish now means building for 600 million people who currently get second-class AI experiences.