Urdu and the Nastaliq script: why OCR and search struggle

Urdu is written in an extended Arabic script, and the Unicode Standard notes that it is often written in the ornate Nastaliq style (Unicode 17.0, Chapter 9). Nastaliq slopes down from right to left, overlaps letters, and changes each letter's shape with its neighbors. Arabic is usually printed in the more upright Naskh style, so optical character recognition (OCR) built for Arabic does not carry over directly. A 2016 peer-reviewed study names the problems: diagonal writing, high cursiveness, context sensitivity, and overlapping characters (Naz and colleagues). Spoken Urdu is very close to Hindi, which is written in a different script.

Nastaliq against Naskh

Naz and colleagues describe Naskh and Nastaliq as the two commonly followed styles: Arabic uses Naskh, and Urdu and Persian follow Nastaliq. Both run right to left. In Naskh, a letter has up to four shapes: initial, medial, final, and isolated. In Nastaliq, a letter takes several different forms rather than four, and the shape depends on the letters on both sides. The authors put the number of Urdu ligatures, the joined letter groups, at more than 25,000 in one passage and about 26,000 in another, both citing Lehal (2012).

The slope causes a second problem. In Nastaliq the pen moves from right to left and also from top to bottom, so letters sit at different heights and a line has no single baseline. The authors say ascenders and descenders, with their oblique angles and long tails, lead to errors in baseline detection.

The authors of UTRNet, a 2023 Urdu OCR paper, compare the inventories. Arabic print is mostly upright and has 28 letters. They count 45 main Urdu letters, plus punctuation, honorific marks, digits, and borrowed characters, for 181 distinct glyphs to recognize. Several Urdu letters share one shape and differ only in their dots.

What published OCR results show

The 2016 study, published in SpringerPlus, tested a multidimensional long short-term memory network, a neural model that scans an image in four directions. The test set came from the Urdu Printed Text Images (UPTI) dataset, synthetic text lines drawn from the Jang newspaper. The version the authors used held 10,000 line images; the UTRNet authors count 10,063 samples. Accuracy was measured by edit distance, the number of insertions, deletions, and substitutions needed to turn the output into the correct text. The model reached 98 percent, against 94.85 percent for an earlier 2013 result.

Most of the remaining errors were spaces. Nastaliq leaves a space-like gap after every letter that does not join the next one, and word spaces are no wider than that gap. The model made 304 insertion and 279 deletion errors on spaces. Ignoring them, accuracy rose to 99.82 percent.

Real pages are harder. The UTRNet authors built UTRSet-Real, over 11,000 annotated real-world lines. Their model, trained on UPTI, scored 99.17 percent on UPTI's own validation lines, 64.73 percent on UTRSet-Real, and 76.15 percent on a third set, IIITH. Trained on UTRSet-Real, it scored 92.97 percent on real lines. The authors conclude that UPTI is not representative of real-world scenarios. Transformer-based OCR models scored between 26.13 and 41.17 percent on the real lines, which the authors attribute to overfitting the small training data.

Urdu and Hindi: one spoken language, two scripts

Two computational linguistics papers describe the pair. Durrani and colleagues write in 2010 that Hindi, an official language of India, is written in Devanagari, and Urdu, the national language of Pakistan and a state language in India, in Perso-Arabic script. They state that the spoken forms are very similar and that most verbs and closed-class words, such as pronouns and case markers, are the same. In a 5,000-word sample of BBC news, about 62 percent of Hindi word types were also Urdu vocabulary. Literary text, such as novels or history, draws more on Sanskrit for Hindi and on Persian and Arabic for Urdu.

Malik, Boitet, and Bhattacharyya treat converting between the two as inter-dialectal transliteration. They write that the two languages share grammar, morphology, and a huge vocabulary, and that the major difference is script and the source of borrowed words.

The scripts do not map one to one. Durrani and colleagues note that the "z" sound has one spelling in Hindi but can be written with four letters in Urdu: ذ, ز, ض, and ظ. Their Hindi-to-Urdu transliterator was 81.6 percent accurate. A system can treat the two as one language when spoken and must still handle two writing systems on paper.

How Unicode encodes Urdu letters

The Unicode locale chart for Urdu lists the main letters a standard Urdu text uses. They include tteh, ddal, and rreh (ٹ, ڈ, ڑ), keheh (U+06A9), heh goal (U+06C1), heh doachashmee (U+06BE), Farsi yeh (U+06CC), and yeh barree (U+06D2). The Arabic heh (U+0647) and Arabic yeh (U+064A) appear only in the chart's auxiliary list.

Chapter 9 explains several of these letters:

• Heh. The isolated forms of the Arabic heh and of heh goal look alike, so two words that look the same can differ in code points.
• Yeh barree. Urdu adopted it as a distinct letter for the /e/ vowel. The letter joins only on its right side, so /e/ at the start or middle of a word is written with Farsi yeh.
• Noon ghunna (U+06BA). The dotless noon marks nasalized vowels at word ends. Mid-word, the regular noon is used, and Unicode notes that entry software may emit the regular noon when the noon ghunna key is typed mid-word.
• Hamza on heh. For Urdu, Unicode directs use of heh goal with hamza above (U+06C2), not U+06C0.

A search index that compares raw code points will treat each variant spelling as a different word.

What to check

• Was the OCR accuracy measured on real scanned pages or on synthetic lines such as UPTI?
• Does the OCR engine model Nastaliq, or was it built for Naskh-style Arabic print?
• How does it handle the space-like gap after a non-joining letter, the largest error source in the 2016 study?
• Does search treat the Arabic heh and heh goal, and the Arabic yeh and Farsi yeh, as the same letter?
• Can the system match Urdu text to Hindi text in Devanagari, where one Hindi spelling can map to four Urdu letters?

Azcuna's BABEL reads, translates, and searches foreign-language documents entirely on-device, with no cloud and no connection, and keeps every claim tied to the passage it came from.

Sources

1. 1The Unicode Standard, Version 17.0, Core Specification, Chapter 9: Middle East-I · The Unicode Consortium
2. 2Locale Data Summary for Urdu [ur] · The Unicode Consortium (Common Locale Data Repository charts)
3. 3Urdu Nasta'liq text recognition using implicit segmentation based on multi-dimensional long short term memory neural networks · SpringerPlus 5:2010 (November 25, 2016), Naz and colleagues; open-access copy on PubMed Central
4. 4UTRNet: High-Resolution Urdu Text Recognition In Printed Documents · arXiv (version 3, August 23, 2023), preprint of the ICDAR 2023 paper by Rahman, Ghosh, and Arora
5. 5Hindi-to-Urdu Machine Translation through Transliteration · Durrani, Sajjad, Fraser, and Schmid, ACL 2010, Uppsala (ACL Anthology)
6. 6Hindi Urdu Machine Transliteration using Finite-State Transducers · Malik, Boitet, and Bhattacharyya, COLING 2008, Manchester (ACL Anthology)

Where this shows up in our work

• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.