Scaling Laws
The 2020 Kaplan paper said: bigger model, lower loss. The 2022 Hoffmann paper said: you were under-training. Compute goes into two buckets - parameters and tokens - and the split is not obvious.
Learn this with Professor KatalockOpens in KatalockAi chat: your AI engineering mentor teaches this lesson step by step, checks your understanding and reviews your code.