Turkish language model from ufak AI
ufakzeka-1
A 151M parameter Turkish language model trained from scratch. Open weights, open data recipe, Apache-2.0.
Turkish language model from ufak AI
A 151M parameter Turkish language model trained from scratch. Open weights, open data recipe, Apache-2.0.
Our model and the baselines were measured with the same harness. The five models sit well above random guessing on four of the seven tasks and close to it on the other three. On ARC-e and XCOPA, two of those four, ufakzeka-1 is less than three points behind the Turkish models five times its size; on TurBLiMP, which measures grammar, it is 5 to 8 points behind.
Stones are proportional to parameter count.
| Model | Params | HellaSwag | ARC-c | ARC-e | XCOPA | Belebele | TurBLiMP | TurkishMMLU |
|---|---|---|---|---|---|---|---|---|
| ufakzeka-1 | 151M | 33.2 | 27.6 | 38.8 | 59.8 | 27.4 | 90.1 | 23.3 |
| ufakzeka-1-base | 151M | 35.3 | 27.6 | 39.1 | 60.0 | 27.6 | 92.4 | 19.2 |
| Kanarya-750M | 750M | 37.8 | 26.5 | 41.4 | 61.2 | 23.4 | 95.3 | 16.2 |
| turkish-gpt2-large | 774M | 35.3 | 25.4 | 40.4 | 60.6 | 22.4 | 98.2 | 19.0 |
| Qwen2.5-0.5B | 494M | 29.2 | 23.6 | 28.6 | 54.6 | 29.9 | 70.0 | 18.2 |
| Random guess | 25 | 25 | 25 | 50 | 25 | 50 | 20 |
Every model measured with the same harness and no worked examples; the score is the share of questions where the model finds the correct option most likely. HellaSwag and ARC are length-corrected accuracy, the rest raw. TurBLiMP averages 16 subsets, TurkishMMLU 9 subjects. Details in the model card.
The total and the API line are from the bills; pretraining is its run hours at list price, and fine-tuning and evaluation are the rest. Pretraining ran in August, fine-tuning and every measurement in September. The full ledger is in the repository.
The architecture is plain Qwen3; transformers needs no custom code. The GGUF files need a small tokenizer patch applied to llama.cpp; details in the docs.
llama-cli -m ufakzeka-1-q8_0.gguf --temp 0.3 --top-k 40 --top-p 0.9 --repeat-penalty 1.0 --dry-multiplier 0 -n 520