Abstract
Rather than a talk, this firstmost is an invitation to discussion. We’re working on pre-training LLMs for Arabic and evaluation at AI Init and will share the progress, as well as some afterthoughts. Right now our efforts comprise two directions of work: pre-training (we learn to pretrain various models from scratch) and evaluation (just have it in-house + to not only observe perplexity on the pretraining dashboard). Figuring out the potential collaboration directions is our key interest for this meeting.