every repo i found for training your own llm (nanochat, minimind) needed an nvidia gpu. i have a macbook air with no fan. so i made airgpt, which does the whole thing on a mac with one command. it downloads some text, trains a tokenizer, pretrains a small model, finetunes it for chat, and then you can talk to it. its all mlx.
it runs on time, not steps
you tell it how long to train, not how many steps. the hour preset takes an hour on any mac, a faster mac just gets through more text in that hour. the learning rate schedule runs off the clock too so it always finishes properly.
what an hour gets you
8M params, 55M tokens, val loss 2.27. it prints a sample every 2 minutes with the same prompt so you can watch it learn. 2 minutes in it was already writing "Once upon a time, there was a little boy named Tim. Timmy loved to play with his friends. One day, Timmy was a very excited." by the end its writing little stories with dialogue in them.
the chat part is pretty bad at this size. it says hi fine, then you ask it the capital of france and it says "the capital of france is the capital of the france of the tech of decimal".
what 8 hours gets you
the night preset is 40M params. it read 120M tokens overnight and got to val loss 2.11. the stories are longer and hold together better. the chat is still weak, now its "the capital of france is the capital of france", which is at least true.
it never throttled
the trainer reads the macs thermal state while it runs so you can see if its slowing down. it never did. the hour run stayed at nominal the whole time, and so did the 8 hour one, about 5.3k tokens a second from start to finish.
try it
git clone https://github.com/navthings/airgpt cd airgpt pip install -r requirements.txt python train.py --preset hour
or if you dont want to wait, both models are on hugging face. theres also a 2 minute benchmark to see how your mac compares, and a speedrun for the best loss in an hour.