{"apiVersion":"v1","taxonomyVersion":1,"generatedAt":"2026-10-29T13:28:20.592Z","data":{"recordId":"wsa_4zvmfddnr4am","score":1,"confidence":"high","capped":false,"capReason":null,"sources":[{"n":1,"url":"https://developers.googleblog.com/integrating-torchtpu-into-torchtitan-and-optimizing-models-for-tpu/","publisher":"Google","authority":"owner-controlled","verification":"verified","lastRetrievedAt":"2026-10-28T00:02:38.647Z","detectedPublishedAt":"2026-10-20T00:00:00.000Z","rights":"original"}],"claims":[{"id":"16d2b789-527d-4569-b63b-efc7231af9eb","kind":"availability","statement":"The TorchTitan TPU fork is available at https://github.com/google-pytorch/torchtitan/.","citations":[{"source":1,"supportProbability":0.98,"supports":true,"quote":{"text":"The fork is available at https://github.com/google-pytorch/torchtitan/","rights":"original"}}]},{"id":"cdbf342d-3d63-4c68-acb8-73063538b66f","kind":"capability","statement":"TorchTitan TPU provides recipes for pre-training large dense models, MoE models, and diffusion transformers.","citations":[{"source":1,"supportProbability":0.94,"supports":true,"quote":{"text":"TorchTitan provides official, battle-tested recipes for pre-training large dense models (such as Llama 3 70B/405B and Qwen 3 8B/32B), Mixture-of-Experts (such as Qwen 3 235B MoE), and diffusion transformers (such as FLUX.1 12B).","rights":"original"}}]},{"id":"9ac0e54c-2d9f-4f1e-bf24-456f918c8856","kind":"capability","statement":"TorchTitan TPU can serve as the scalable trainer inside post-training pipelines such as VeRL for GRPO and RL alignment.","citations":[{"source":1,"supportProbability":0.98,"supports":true,"quote":{"text":"Beyond initial pre-training, TorchTitan serves as the scalable trainer inside post-training pipelines like VeRL for Group Relative Policy Optimization (GRPO) and Reinforcement Learning (RL) alignment.","rights":"original"}}]},{"id":"1dab5655-7071-45b6-b245-3d73e33fd5e6","kind":"capability","statement":"The TPU recipes can use SimpleFSDP and Graph Trainer to capture training computations for XLA.","citations":[{"source":1,"supportProbability":0.87,"supports":true,"quote":{"text":"Graph Trainer goes further (--tpu_config.use_graph_trainer) and lets the whole training step, forward, loss, backward and optimizer update, be captured as one program.","rights":"original"}}]},{"id":"38b5ba62-5f56-4d81-a076-403ab26a1084","kind":"capability","statement":"TorchTitan TPU recipes include optional Pallas kernels for attention, fused loss, and MoE expert matrix multiplication.","citations":[{"source":1,"supportProbability":0.9,"supports":true,"quote":{"text":"The TPU recipes replace standard attention with Splash Attention, a Pallas kernel that handles causal, sliding-window and packed-document masks without materializing the attention matrix; replace the final projection and cross-entropy loss with a fused Pallas kernel that never…","rights":"original"}}]}]},"nextCursor":null,"license":{"catalog":{"name":"CC BY 4.0","spdx":"CC-BY-4.0","url":"https://creativecommons.org/licenses/by/4.0/","attribution":"wowsuchai","attributionUrl":"https://wowsuchai.withglint.app","covers":["catalog metadata","classifications","generated summaries","feed event metadata"]},"excluded":{"covers":["source quotations","linked source content"],"terms":"Not licensed by wowsuchai. Source quotations and linked source content keep their original rights."}}}