mai

Maithili मैथिली

ISO 639-3: mai I L

49,509 Words in vocabulary

4.37x Best compression

0.8575 Best isotropy

Sample text

Excerpts from Maithili Wikipedia articles.

एक अवधी व्यंजन छी। एकर मुख्य घटक बासमती चावल छी। भारत क खाना के व्यंजन

एक पूर्वी भारतक उड़िया व्यंजन छी। सन्दर्भ सामग्रीसभ बाह्य जडीसभ एहो सभ देखी खानप...

एक दक्षिण भारतीय खाना छी। सन्दर्भ सामग्रीसभ बाह्य जडीसभ एहो सभ देखी भारतीय खाना

The 20 most frequently used words in Maithili Wikipedia.

Explore Maithili interactively with browser-based demos.

Key metrics for all model types at a glance.

from wikilangs import tokenizer
tok = tokenizer('latest', 'mai', 32000)
tokens = tok.tokenize("Your text here")

from wikilangs import ngram
ng = ngram('latest', 'mai', gram_size=3)
score = ng.score("Your text here")

from wikilangs import markov
mc = markov('latest', 'mai', depth=3)
text = mc.generate(length=50)

from wikilangs import vocabulary
vocab = vocabulary('latest', 'mai')
info = vocab.lookup("word")

from wikilangs import embeddings
emb = embeddings('latest', 'mai', dimension=64)
vec = emb.embed_word("word")

Model Type	Variants	Description
Tokenizers	8k, 16k, 32k, 64k	BPE tokenizers with different vocabulary sizes
N-gram (Word)	2, 3, 4, 5-gram	Word-level language models
N-gram (Subword)	2, 3, 4, 5-gram	Subword-level language models
Markov (Word)	Depth 1–5	Word-level text generation
Markov (Subword)	Depth 1–5	Subword-level text generation
Vocabulary	—	Word dictionary with frequency and IDF
Embeddings	32d, 64d, 128d	Position-aware word embeddings