Albanian shqip

ISO 639-1: sq ISO 639-3: sqi M L

445,748 Words in vocabulary

4.62x Best compression

0.7903 Best isotropy

Sample text

Excerpts from Albanian Wikipedia articles.

është vendbanim në Ish Republikën Jugosllave të Maqedonisë. në komunën e Novacës

Mbi vitin 390 p.e.s.. Ngjarje Lindje Vdekje 390 p.e.s. p.e.s.

Shqiponja Perandorake e Lindjes (Aquila heliaca) është një Shqiponjë e madhe mbr...

The 20 most frequently used words in Albanian Wikipedia.

Explore Albanian interactively with browser-based demos.

Key metrics for all model types at a glance.

from wikilangs import tokenizer
tok = tokenizer('latest', 'sq', 32000)
tokens = tok.tokenize("Your text here")

from wikilangs import ngram
ng = ngram('latest', 'sq', gram_size=3)
score = ng.score("Your text here")

from wikilangs import markov
mc = markov('latest', 'sq', depth=3)
text = mc.generate(length=50)

from wikilangs import vocabulary
vocab = vocabulary('latest', 'sq')
info = vocab.lookup("word")

from wikilangs import embeddings
emb = embeddings('latest', 'sq', dimension=64)
vec = emb.embed_word("word")

Model Type	Variants	Description
Tokenizers	8k, 16k, 32k, 64k	BPE tokenizers with different vocabulary sizes
N-gram (Word)	2, 3, 4, 5-gram	Word-level language models
N-gram (Subword)	2, 3, 4, 5-gram	Subword-level language models
Markov (Word)	Depth 1–5	Word-level text generation
Markov (Subword)	Depth 1–5	Subword-level text generation
Vocabulary	—	Word dictionary with frequency and IDF
Embeddings	32d, 64d, 128d	Position-aware word embeddings