Levantine Arabic (apc, ajp) for machines

Levantine Arabic is the spoken Arabic of Syria, Lebanon, Jordan, and the Palestinian territories. Meta's NLLB-200 model and the FLORES-200 benchmark split it in two: apc_Arab, North Levantine, and ajp_Arab, South Levantine, both listed as low resource in the No Language Left Behind paper. The ISO 639-3 registration authority took the opposite course in 2023 and merged the two codes into one, apc, renamed Levantine Arabic. This post covers what separates Levantine from Modern Standard Arabic (MSA) in text, what the north-south split means, what public data exists, what NLLB reports under each code, where systems fail, and how to evaluate one. The post on why Arabic is hard for machines covers the script and morphology all varieties share.

apc and ajp: two NLLB codes, one ISO code

The codes tell a short history. The FLORES-200 README lists apc_Arab as North Levantine Arabic and ajp_Arab as South Levantine Arabic, following the ISO 639-3 codes as they stood in 2022. The NLLB paper's language table names Ammani, the dialect of Amman, as the specific variety for ajp_Arab and names no specific variety for apc_Arab. On January 20, 2023, under change request 2022-006, the registration authority adopted two updates: apc was renamed from North Levantine Arabic to Levantine Arabic, and its scope was expanded "by inclusion of [ajp] South Levantine Arabic". The ajp page now shows the code as deprecated, with the remedy "Merge into [apc]". Both remain members of the macrolanguage ara, Arabic, which has 30 member codes, 28 of them active.

The practical consequence is that a system built on current ISO codes has one Levantine label, while NLLB-200, FLORES-200, OPUS, and every paper that evaluated on them have two. A pipeline that maps apc to apc_Arab and silently drops ajp loses the South Levantine model direction, which, as the scores below show, is the one that performs better. The split itself is real in the linguistics: the Curras and Baladi paper by Al-Haff and colleagues says the Levantine family divides into Northern Levantine, including Lebanese and Syrian, and Southern Levantine, including Palestinian and Jordanian. The MADAR corpus and the CODA* orthography guidelines use the same two sub-regions, with Beirut, Damascus, and Aleppo in the north and Jerusalem, Amman, and Salt in the south.

Levantine Arabic vs MSA: what shows up in text

The MADAR paper gives the three-way qaf example: the MSA letter qaf is pronounced /q/ in Tunisian, as a glottal stop in Egyptian and Levantine, and as /g/ in Gulf Arabic. MADAR adds that dialect writers often spell words by their sound or by their etymology, so the letter is sometimes kept and sometimes replaced by a hamza. The future marker is the clearest grammatical signal: MADAR gives raH or Ha- in Levantine against sa- or sawfa in MSA, so I will write is sa-aktub in MSA and Ha-aktub in Palestinian. Zaidan and Callison-Burch, in their 2014 Computational Linguistics article, show a Google Translate output of a Levantine sentence in which the system transliterated rather than translated the first two words, aymta, when, and raH, the future marker, both of them, the paper notes, relatively frequent function words.

Negation separates the Levant from MSA and also splits the Levant internally. MSA negates with the separate particles laa, lam, and lan and changes the verb form. The dialects put ma before the verb, and the CODA* guidelines' Table 5 shows the southern pattern adding a suffix: Amman ma biddnash, we do not want, with -sh, against Damascus ma qaal, he did not say, with nothing after the verb. The same guidelines' Table 7 shows how the number eight is written and said across the Levant: thamanye in Salt, keeping the interdental th of MSA; tamanye in Amman; tamaane in Damascus. CODA keeps the MSA letter thaa in all three spellings, but a writer who spells by sound produces three strings for one word.

The Shami corpus paper by Abu Kwaik and colleagues lists further traits: the Palestinian word aysh, what, where MSA would say ay shay', what thing; a phrase like kifak, how are you, which is written identically across dialects but pronounced differently in each country; and a sociolinguistic layer in which Jordanian splits into urban, rural, and Bedouin varieties and many Palestinians level their dialect toward Syrian and Lebanese. Zaidan and Callison-Burch summarize the Levantine group as dialects that differ in pronunciation and intonation but are "largely equivalent in written form", which is good news for a shared model and bad news for anyone trying to tell them apart from text.

Lebanese adds two more things. The Shami paper notes that Lebanese texts contain a lot of French and Arabizi, Arabic written in Latin letters. The Bies and colleagues paper on Arabizi explains that the letter choices follow local pronunciation and the local second language: j represents the letter jeem in Levantine Arabizi where Egyptian uses g, and ch represents sheen where French dominates, as in North Africa and Lebanon, against sh where English does. A Levantine Arabizi converter trained on Egyptian conventions will read Levantine j and ch wrong.

What public data exists for Levantine Arabic

Parallel data under the two codes is almost nonexistent. The OPUS API for English-North Levantine lists two corpora and 992 aligned pairs: translatewiki with 974 and Tatoeba with 18. The API for English-South Levantine lists one corpus, Tatoeba, with 59 pairs. Neither code has a Wikipedia: the Wikimedia site matrix shows a Wikipedia for Arabic, Egyptian Arabic, and Moroccan Arabic only.

The larger Levantine parallel sets live under other labels. The AraBench paper by Sajjad and colleagues describes the Arabic-Dialect/English Parallel Text from Zbib and colleagues, with about 138,000 Levantine-English sentences from weblogs, the largest dialect-English training set they found, plus a Multi-Dialectal Corpus in which Egyptian-English sentences were re-translated by Palestinian, Syrian, and Jordanian speakers into their own dialects, and 8,000 more sentences into Syrian only. MADAR has 2,000 travel-phrase sentences translated into each of its 25 city dialects, including Jerusalem, Amman, Salt, Beirut, Damascus, and Aleppo, and 12,000 for Beirut, all parallel with English, French, and MSA; the sentences average 6.5 words.

Monolingual and annotated resources are better. Shami holds 117,805 sentences and about 13.15 million tokens from web sources: Syrian 48,159 sentences, Jordanian 32,078, Palestinian 21,264, and Lebanese 16,304, on topics that include politics, education, society, health care, and housekeeping. Curras, the Palestinian corpus revised in the 2022 paper, has 56,000 tokens with morphological annotation, and Baladi adds 9,600 Lebanese tokens; the paper says Lebanese is spoken by about 6 million people in Lebanon and almost twice that number abroad. Nabra, by Nayouf and colleagues, is about 60,000 Syrian words in more than 6,000 sentences from social media, scripts, song lyrics, and proverbs, covering ten local dialects: Aleppo, Damascus, Deir-ezzur, Hama, Homs, Huran, Latakia, Mardin, Raqqah, and Suwayda. The NADI shared tasks, described below, supply tweets with country labels; NADI 2020 had 1,595 Syrian, 905 Lebanese, 634 Jordanian, and 624 Palestinian tweets.

What NLLB-200 reports for apc_Arab and ajp_Arab

The NLLB paper defines low resource as fewer than 1 million publicly available de-duplicated sentence pairs with any other language, and both Levantine codes qualify. For the FLORES-200 test sets, the paper says the vendor teams either translated each Arabic variety from English or adapted the MSA set; footnote 7 names the four adapted sets, and Section 8.7.2 says South Levantine and Egyptian were translated from English. North Levantine is not placed in either group in the text. The posts on NLLB-200 and FLORES-200 explain the models and the benchmark.

The metrics files linked from the fairseq NLLB README give these chrF++ scores against English:

The 54.5B file adds spBLEU with the SPM-200 tokenizer: 44.8 into English for North Levantine and 48.2 for South Levantine, against 48.3 for MSA. On the 3.3B model MSA scores 65.8 into English and 55.0 from English, so South Levantine trails MSA by only 1.6 chrF++ into English, and North Levantine by 3.0. The 54.5B file also scores the pair with each other: North into South Levantine 51.1 chrF++, South into North 47.7, and both into MSA at 53.7 and 54.1. The model treats them as three languages.

Section 8.7.2 explains why the South Levantine score should be read with care. Table 43 measures each dialect reference against the MSA reference in FLORES-200 devtest: South Levantine is the furthest of the six dialects measured, with 42.13 percent of tokens per sentence absent from the MSA sentence and a reference-to-reference spBLEU of 32.6. The model nevertheless scores better on it than on dialects whose references sit closer to MSA. The paper offers two hypotheses: the dialect may be closer to MSA than the measurement suggests, with the apparent difference an artifact of translating from English rather than adapting, or text translated directly from English may be semantically and syntactically closer to the English source and so easier to translate. Table 44 shows the model's English-to-South Levantine output is 24.50 percent dialectal at the sentence level and scores 58.5 spBLEU against its own MSA output, so what the model generates is nearer to MSA than the human reference is.

The paper also evaluated on MADAR, mapping 16 of the 25 city dialects to its 8 codes and pairing each with MSA. Table 58 gives, in BLEU and chrF++, 20.2 and 46.6 from MSA into North Levantine and 26.5 and 48.3 back; 17.6 and 43.5 from MSA into South Levantine and 27.5 and 49.1 back. Those are travel phrases, not the Wikinews, Wikijunior, and Wikivoyage text of FLORES-200.

What goes wrong

The first failure is MSA output for Levantine input, or Levantine input read as MSA. AraBench's Levantine experiment is the cleanest measurement: a system trained on about 40 million MSA-English pairs scored 11.0 and 10.8 BLEU on Levantine dev and test sets, worse than a system trained on the 136,000 Levantine pairs alone, which scored 16.5 and 16.4. Fine-tuning the MSA system on the Levantine data reached 20.4 and 19.9. For Levantine, unlike Egyptian in the same paper, MSA data alone was not enough to beat a small in-dialect system. The NLLB dialectness figures above are the other side of the same problem: a model asked for Levantine produces something closer to MSA than a Levantine speaker would write.

The second failure is dialect identification, both of Levantine against other dialects and within the Levant. The NLLB paper's own language identifier was not scored on it; the caption of Table 5 says the Arabic languoids were merged after linguistic analysis when the identifier was evaluated. The Shami paper's four-way classifier reached 52 percent accuracy on the whole corpus, which the authors attribute to the great overlap between the dialects and the dispersion of lexical items between categories; only Jordanian against Lebanese reached 90 percent. NADI 2024 reported annotator agreement of Fleiss kappa 0.59 for Palestine and 0.54 for Syria, lower than Egypt's 0.69; Jordan labels were excluded from the test set, so the paper's regional analysis represents the Levant with Palestine and Syria only. The task moved to multi-label identification because a sentence can be valid in several dialects, and cites a finding that about 66 percent of a single-label classifier's errors were not true errors. The best system reached 50.57 macro F1.

The third failure is translating into MSA, the task programs often want so that MSA tools can run downstream. NADI 2023 and NADI 2024 both ran a dialect-to-MSA translation subtask with Jordanian and Palestinian among four dialects. In 2023 the best open-track system scored 22.03 BLEU on Jordanian and 17.29 on Palestinian; in 2024 the best system scored 21.37 and 20.62. The organizers concluded that the task remains hard.

The fourth failure is routing. The TARJAMAT paper by Kadaoui and colleagues tested NLLB-200's distilled 1.3B model on 200 Jordanian and 200 Palestinian sentences of at least ten words transcribed from television series. With MSA declared as the source, NLLB scored 13.07 BLEU on Jordanian and 14.98 on Palestinian. The authors then mapped both to South Levantine, ajp_Arab: Jordanian rose to 18.5, but Palestinian fell to 12.56. ChatGPT scored 25.74 and 29.47 zero-shot and Google Translate 29.35 and 25.78. Declaring the dialect helped one South Levantine variety and hurt the other, which is a warning against assuming a code label fixes the problem.

Handwriting and scans from the Levant

One public dataset of historic Arabic handwriting comes from Lebanese archives. The Muharaf dataset by Saeed and colleagues holds 1,644 page images of historic handwritten Arabic from Lebanese archives, dated from the early 19th century to the early 21st, with 36,311 transcribed text lines and 4,867 text regions: personal letters, poems, and records, on pages ranging from clean white backgrounds to creased sheets with ink bleed. The paper notes that earlier datasets such as KHATT and MADCAT were written by scribes copying given text under controlled conditions, while Muharaf is scanned originals, which is the kind of document an analyst receives. Transcriptions were checked by a Lebanese Arabic expert who was also a historian. A recognition engine measured only on printed Arabic has not been measured on this. The post on OCR of scanned documents covers the recognition stage.

U.S. government materials

Two public U.S. government documents bear on Levantine Arabic as a language of work. The Department of Education's ERIC index holds Levantine Arabic: Introduction to Pronunciation, a 1971 Foreign Service Institute course sponsored by the Peace Corps, 107 pages with 19 recorded drill sections on the points of phonological interference between Levantine Arabic and American English. The National Geospatial-Intelligence Agency hosts the BGN/PCGN 1956 romanization system for Arabic, applied to geographic names in Jordan, Syria, the West Bank and Gaza Strip, and other Arabic-language areas. The system is based "insofar as possible on fully pointed Modern Standard Arabic", and its notes say that recognizing dialectal and idiosyncratic deviations is essential and that colloquial forms found on authoritative sources, such as Bu for Abu, should be kept. The post on transliterating names covers why one name has many spellings.

How to evaluate a Levantine Arabic system

• Test both codes. FLORES-200 apc_Arab and ajp_Arab devtest give sentences from Wikinews, Wikijunior, and Wikivoyage; MADAR gives travel phrases for six Levantine cities; Shami and NADI give social text; Curras, Baladi, and Nabra give annotated tokens for Palestinian, Lebanese, and Syrian.
• Report direction and model size. On NLLB's 3.3B model, Levantine into English is about 15 chrF++ points above English into Levantine.
• Measure how dialectal the output is against an MSA translation of the same input, as the NLLB paper does; a Levantine system whose output matches MSA token for token has not produced Levantine.
• Spot-check the future marker raH, the question words aymta and aysh, and the negation suffix -sh, which separate Levantine from MSA and south from north.
• Test Lebanese text with French and Arabizi mixed in, and check that the converter reads j and ch the Levantine way.
• If the system routes by dialect, compare routed and unrouted scores on each variety separately, since TARJAMAT found the South Levantine label helped Jordanian and hurt Palestinian.

What to check

• Which code does the system claim: the current ISO code apc for all of Levantine, NLLB's apc_Arab for the north, ajp_Arab for the south, or only ar and arb?
• Was the FLORES-200 South Levantine score quoted with the NLLB paper's caution that the set was translated from English and may be skewed?
• Which countries' text will the system read, and does the test set include Syrian, Lebanese, Jordanian, and Palestinian material in the same proportions?
• Does the pipeline normalize qaf and hamza, attached and detached ma, and the th-t-s spellings of the same word before translating or indexing?
• Are the documents typed, scanned print, or handwriting from archives, and what error rate does the recognizer report on handwriting like Muharaf's?
• Does the dialect identifier output one label or several, and what does it do when a sentence is valid in two dialects?

Azcuna's BABEL reads, translates, and searches foreign-language documents entirely on-device, with no cloud and no connection, and keeps every claim tied to the passage it came from.

Questions people ask

What is apc_Arab?

apc_Arab is the NLLB-200 and FLORES-200 code for North Levantine Arabic in Arabic script. The first part is the ISO 639-3 code apc, which the registration authority now names Levantine Arabic after a January 20, 2023 change that merged the South Levantine code ajp into it. The second part is the ISO 15924 script code Arab. The No Language Left Behind paper lists apc_Arab as a low-resource Semitic language with no specific variety named.

What is the difference between North and South Levantine Arabic?

The MADAR and CODA* papers divide the Levant into a North sub-region, with Beirut, Damascus, and Aleppo, and a South sub-region, with Jerusalem, Amman, and Salt. The Curras and Baladi paper says Palestinian represents Southern Levantine and Lebanese represents Northern. NLLB-200 names Ammani as the specific variety of its South Levantine code. CODA* examples show Amman and Cairo adding the suffix -sh to negated verbs where Damascus uses the particle ma alone.

Does NLLB-200 translate Levantine Arabic?

Yes, under two codes. The FLORES-200 README lists apc_Arab, North Levantine Arabic, and ajp_Arab, South Levantine Arabic. The NLLB metrics file for the 3.3B dense model gives chrF++ of 62.8 from North Levantine into English and 47.9 from English into it, and 64.2 and 49.7 for South Levantine. The 54.5B mixture-of-experts model reaches 64.9 and 67.4 into English, with spBLEU of 44.8 and 48.2.

Why is the South Levantine FLORES score almost as high as MSA?

The No Language Left Behind paper reports that the South Levantine and Egyptian test sets were translated directly from English, while four other dialect sets were adapted from the MSA translation. Table 43 shows the South Levantine reference is the furthest of six dialects from the MSA reference, 32.6 spBLEU, yet the model scores well on it. The authors offer two explanations: the dialect may be closer to MSA than the measurement suggests, or text translated straight from English may be easier to translate back.

Is Levantine Arabic one dialect or several?

ISO 639-3 treats it as one individual language, apc, since the 2023 merger. The Shami corpus paper says Levantine dialects are frequently treated as one despite their differences, and notes that Jordanian alone splits into urban, rural, and Bedouin varieties. The Nabra corpus covers ten local Syrian dialects, including Aleppo, Damascus, Deir-ezzur, Homs, and Latakia. In MADAR's lexical comparison of 25 city dialects, Amman and Jerusalem were the most similar pair, with 54.4 percent vocabulary overlap.

How hard is it to tell Syrian, Lebanese, Jordanian, and Palestinian apart?

Hard, from text alone. The Shami corpus paper reports 52 percent accuracy when classifying all four, because of the great overlap between the dialects, and 90 percent only for the easiest pair, Jordanian versus Lebanese. Zaidan and Callison-Burch describe the Levantine dialects as differing in pronunciation and intonation but largely equivalent in written form. In NADI 2024, annotators from Palestine and Syria reached Fleiss kappa of only 0.59 and 0.54 on country labels.

Where can I get Levantine Arabic data?

The OPUS API lists 992 English-North Levantine pairs, 974 of them from translatewiki, and 59 English-South Levantine pairs from Tatoeba. Shami has 117,805 monolingual sentences from Palestinian, Jordanian, Syrian, and Lebanese web sources. MADAR has 12,000 Beirut sentences and 2,000 each for Damascus, Aleppo, Jerusalem, Amman, and Salt, parallel with English and MSA. Curras has 56,000 annotated Palestinian tokens, Baladi 9,600 Lebanese, and Nabra about 60,000 Syrian words.

How does Levantine Arabic negation differ from MSA?

MSA negates with the separate particles laa, lam, and lan and changes the verb form. The Levantine dialects, like other dialects, use the particle ma before the verb, and the CODA* guidelines show the southern cities adding a suffix: Amman ma biddnash, we do not want, against Damascus ma qaal, he did not say, with no suffix. Zaidan and Callison-Burch show a Google Translate output that transliterated rather than translated the Levantine future marker raH and the word for when.

What is the U.S. government connection to Levantine Arabic?

The Department of Education's ERIC index holds a 1971 Foreign Service Institute course, Levantine Arabic: Introduction to Pronunciation, sponsored by the Peace Corps, with 19 drill sections on the sounds that interfere between Levantine Arabic and American English. The National Geospatial-Intelligence Agency publishes the BGN/PCGN 1956 romanization system for Arabic place names in Jordan, Syria, and the West Bank and Gaza Strip, which is based on fully pointed MSA but keeps colloquial forms found on authoritative sources.

Sources

1. 1No Language Left Behind: Scaling Human-Centered Machine Translation · arXiv (NLLB Team, Meta AI), v3 of August 25, 2022
2. 2FLORES-200 (README) · Meta AI Research (facebookresearch on GitHub)
3. 3fairseq, NLLB branch: No Language Left Behind (README) · Meta AI Research (facebookresearch on GitHub)
4. 4NLLB-200 3.3B dense model: FLORES-200 metrics (metrics.csv) · Meta AI Research
5. 5NLLB-200 54.5B MoE model: FLORES-200 metrics (metrics.csv) · Meta AI Research
6. 6NLLB-200-Distilled 600M model: FLORES-200 metrics (metrics.csv) · Meta AI Research
7. 7NLLB-200 1.3B dense model: FLORES-200 metrics (metrics.csv) · Meta AI Research
8. 8ISO 639-3 code apc: Levantine Arabic · SIL International, ISO 639-3 Registration Authority
9. 9ISO 639-3 code ajp: South Levantine Arabic (deprecated) · SIL International, ISO 639-3 Registration Authority
10. 10ISO 639-3 code ara: Arabic (macrolanguage) · SIL International, ISO 639-3 Registration Authority
11. 11The MADAR Arabic Dialect Corpus and Lexicon · LREC 2018 (Bouamor et al.), European Language Resources Association
12. 12Unified Guidelines and Resources for Arabic Dialect Orthography · LREC 2018 (Habash et al.), European Language Resources Association
13. 13Shami: A Corpus of Levantine Arabic Dialects · LREC 2018 (Abu Kwaik, Saad, Chatzikyriakidis, Dobnik), European Language Resources Association
14. 14Curras + Baladi: Towards a Levantine Corpus · LREC 2022 (Al-Haff, Jarrar, Hammouda, Zaraket), European Language Resources Association
15. 15Nabra: Syrian Arabic Dialects with Morphological Annotations · ArabicNLP 2023 (Nayouf et al.), Association for Computational Linguistics
16. 16Arabic Dialect Identification · Computational Linguistics 40(1), 2014 (Zaidan and Callison-Burch), MIT Press
17. 17AraBench: Benchmarking Dialectal Arabic-English Machine Translation · COLING 2020 (Sajjad et al.), International Committee on Computational Linguistics
18. 18Transliteration of Arabizi into Arabic Orthography: Developing a Parallel Annotated Arabizi-Arabic Script SMS/Chat Corpus · EMNLP 2014 Workshop on Arabic Natural Language Processing (Bies et al.), Association for Computational Linguistics
19. 19NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task · arXiv (Abdul-Mageed, Zhang, Bouamor, Habash), WANLP 2020
20. 20NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task · arXiv (Abdul-Mageed et al.), October 24, 2023; ArabicNLP 2023
21. 21NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task · ArabicNLP 2024 (Abdul-Mageed et al.), Association for Computational Linguistics
22. 22TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties · arXiv (Kadaoui et al.), v2 of October 23, 2023; ArabicNLP 2023
23. 23OPUS API: corpora for English-North Levantine Arabic (source=en, target=apc) · OPUS, the open parallel corpus (University of Helsinki / NLPL)
24. 24OPUS API: corpora for English-South Levantine Arabic (source=en, target=ajp) · OPUS, the open parallel corpus (University of Helsinki / NLPL)
25. 25Wikimedia site matrix (API: action=sitematrix) · Wikimedia Foundation
26. 26Muharaf: Manuscripts of Handwritten Arabic Dataset · arXiv (Saeed et al.), 2024
27. 27Levantine Arabic: Introduction to Pronunciation (ERIC ED054667) · Foreign Service Institute, U.S. Department of State, 1971; indexed by ERIC, U.S. Department of Education
28. 28Romanization of Arabic, BGN/PCGN 1956 System (Revised Presentation 2019) · U.S. Board on Geographic Names and Permanent Committee on Geographical Names, hosted by the National Geospatial-Intelligence Agency

Where this shows up in our work

• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.