Building a 200M parameter LLM from scratch in PyTorch

It's really easy to spin up Unsloth and fine-tune Llama 3 in an afternoon. I wanted to see what happens when you don't do that. I wanted to write the BPE tokenizer, implement RoPE, code the SwiGLU feed-forward blocks, and write the training loop in pure PyTorch from absolute scratch.

Source: [Dev.to](https://dev.to/gulding/building-a-200m-parameter-llm-from-scratch-in-pytorch-1c9k)