#12 PaLM: Scaling Language Modeling with Pathways
This important 2022 paper described how to train huge-at-the-time 540-billion parameter language model. It used the Pathways system, which for the first time allowed Google to create training runs across multiple TPUv4 pods, which was necessary to scale up far enough.