Meta's AI Workloads Now Run on Google TPUs via New PyTorch Integration

More than 700 tokens per second is the speed benchmark that just made headlines for Google’s seventh-generation Ironwood TPU. This figure was achieved during a test run of the Moonshot AI Kimi K3 model, a massive open-weight system with 2.8 trillion parameters. The result suggests that Google’s custom silicon is now fast enough to compete directly with industry standards, a claim backed by fresh data released this week. It marks a significant step for the hardware, which has historically been reserved for Google’s internal cloud infrastructure.

The core of this development is a new PyTorch-native version of the SGLang inference engine. Until now, running AI models on TPUs required developers to use Google’s JAX framework. The new tooling allows the hardware to function as a native option for PyTorch, the dominant AI framework in the industry, without requiring extensive code rewrites. A corresponding open-source version of vLLM was also announced. This move is not theoretical; Meta, the creator of PyTorch, has confirmed that SGLang-TorchTPU is running in production environments today. Zongwei Zhou, Meta’s PyTorch-TPU lead, described the effort as a close three-way collaboration between Meta, Google, and the SGLang team. He noted that the new engine provides a remarkably mature serving stack for their operations.

Jana van Greunen, director of PyTorch engineering at Meta, emphasized that the company cannot be constrained by a single hardware architecture. She stated that the new integration enables the deployment of PyTorch workloads directly onto Google TPUs. This marks a return to TPUs for Meta, which previously used an older bridge called PyTorch/XLA between 2020 and 2023 before those contracts were canceled. Google has positioned the new TorchTPU initiative as a successor to that earlier bridge, specifically designed to meet the needs of teams like Meta that were reluctant to migrate to JAX.

While the technical integration is clear, the physical location of these chips remains a point of speculation. Reports from last year indicated that discussions involved Google Cloud TPU rentals for 2026, with potential purchases scheduled for 2027. Meta reportedly signed a multi-year rental agreement, though it is not yet public knowledge exactly where these production TPUs are housed. The announcement came just one day after Amazon revealed it was open-sourcing TorchNeuron, a similar native backend for its own Trainium chips. Both moves appear designed to loosen the software lock-in that has long favored Nvidia’s GPUs.

Performance comparisons show that the new setup is competitive. Inferact, a startup founded by the creators of vLLM, reported that their tuned setup on 16 Ironwood chips with speculative decoding outperformed a comparable setup on 16 Nvidia GB200s, which reached 452 tokens per second. At smaller batch sizes without speculative decoding, the new kernels deliver roughly 1.4 to 2 times the throughput of the GB200 baseline for specific models. The SGLang team plans to add support for more models, starting with DeepSeek V4.1 Flash and Reflection Beam. They aim to achieve Day 0 support for new open models by the first quarter of 2027. Further details are expected at the SGLang Summit in San Francisco in November.

This shift signals a growing trend among major tech firms to diversify their hardware dependencies. By making PyTorch work natively on TPUs, Google is lowering the barrier for external customers to adopt its chips. For users in New York and New Jersey who rely on these platforms, the practical impact will be seen in the efficiency and cost of the AI services they use daily. As the market for AI compute intensifies, the ability to run workloads on non-Nvidia hardware could influence pricing and availability in the coming years.