Nobody sat down in 2017 and designed the Transformer from first principles. What actually happened is messier and much more interesting: someone built a thing, it broke in a specific and annoying way, someone else patched the break, and that patch broke somewhere new. Twenty years of this and y...
Source: [Dev.to](https://dev.to/marjia_029/evolution-of-language-models-every-llm-breakthrough-was-just-a-bug-fix-1dj3)