Turkish language model from ufak AI
ufakzeka-1
A 151M parameter Turkish language model trained from scratch. Open weights, open data recipe, Apache-2.0.
Turkish language model from ufak AI
A 151M parameter Turkish language model trained from scratch. Open weights, open data recipe, Apache-2.0.
Our model and the baselines were measured with the same harness. The three Turkish-trained models sit well above random guessing on four of the seven tasks and close to it on the other three; Qwen2.5-0.5B is well above it only on TurBLiMP. On ARC-e and XCOPA ufakzeka-1 is less than three points behind the two Turkish baselines, models five times its size; on TurBLiMP, which measures grammar, it is 5 to 8 points behind.
Stones are proportional to parameter count.
| Model | Params | HellaSwag | ARC-c | ARC-e | XCOPA | Belebele | TurBLiMP | TurkishMMLU |
|---|---|---|---|---|---|---|---|---|
| ufakzeka-1 | 151M | 33.2 | 27.6 | 38.8 | 59.8 | 27.4 | 90.1 | 23.3 |
| ufakzeka-1-base | 151M | 35.3 | 27.6 | 39.1 | 60.0 | 27.6 | 92.4 | 19.2 |
| Kanarya-750M | 750M | 37.8 | 26.5 | 41.4 | 61.2 | 23.4 | 95.3 | 16.2 |
| turkish-gpt2-large | 774M | 35.3 | 25.4 | 40.4 | 60.6 | 22.4 | 98.2 | 19.0 |
| Qwen2.5-0.5B | 494M | 29.2 | 23.6 | 28.6 | 54.6 | 29.9 | 70.0 | 18.2 |
| Random guess | 25 | 25 | 25 | 50 | 25 | 50 | 20 |
Every model measured with the same harness and no worked examples; the score is the share of questions where the model finds the correct option most likely. HellaSwag and ARC are length-corrected accuracy, the rest raw. TurBLiMP averages 16 subsets, TurkishMMLU 9 subjects. Details in the model card.
The total and the API line are from the bills; pretraining is its run hours at list price, and fine-tuning and evaluation are the rest. Pretraining ran in August, fine-tuning and every measurement in September. The full ledger is in the repository.
The architecture is plain Qwen3; transformers needs no custom code. The GGUF files run on a llama.cpp built after 18 September 2026; details in the docs.
llama-cli -m ufakzeka-1-q8_0.gguf --temp 0.3 --top-k 40 --top-p 0.9 --repeat-penalty 1.0 --dry-multiplier 0 -n 520