700 Tokens Per Second: Meta's Move to Google Chips Shakes Up AI Hardware Market

A single user can now generate 700 tokens per second on Google's seventh-generation Ironwood chips, a speed that significantly outpaces the 452 tokens per second achieved on Nvidia's GB200 hardware. This performance gap has become the focal point of a major strategic shift in the artificial intelligence landscape. Meta, the creator of the widely used PyTorch framework, has officially brought Google's Tensor Processing Units into its production environment. The move marks the first time Google has sold its custom AI chips to an external customer of this scale, challenging the long-standing dominance of Nvidia in the data center market.

The transition was made possible by a new, native version of the SGLang inference engine. Previously, running workloads on TPUs required the use of Google's JAX framework. The new SGLang-TorchTPU allows the hardware to function natively with PyTorch, meaning Meta engineers do not need to rewrite their existing code. Zongwei Zhou, Meta's PyTorch-TPU lead, confirmed that the system is already running in production. He described the result as a remarkably mature, PyTorch-native serving stack, highlighting a close three-way collaboration between Meta, Google, and the SGLang team. This team operates out of RadixArk, a startup founded by the creators of the SGLang engine.

Meta has long resisted being locked into a single hardware architecture. Jana van Greunen, director of PyTorch engineering at Meta, stated that the company cannot be constrained by one type of chip. She noted that the new TorchTPU integration enables the deployment of PyTorch workloads directly onto Google's hardware. Google confirmed that it co-developed this specific backend with Meta's PyTorch team to ensure a smooth transition. This effort follows an older bridge called PyTorch/XLA, which Google now labels as a predecessor. SemiAnalysis reported last year that Meta research teams used TPUs through that older bridge from 2020 to 2023 before canceling those contracts. The current push was designed specifically for Meta, which had previously declined to migrate its entire stack to JAX.

The timeline of this partnership reveals a gradual deepening of ties. Reports from last year indicated that Meta and Google were negotiating cloud TPU rentals for 2026, with hardware purchases planned for 2027. Meta subsequently signed a multi-year rental agreement, setting the stage for the production rollout announced this week. The location of these production TPUs remains unclear, but the shift signals a major change in how Meta plans to handle its massive AI workloads. The announcement landed just one day after Amazon revealed it was open-sourcing TorchNeuron, its own native PyTorch backend for Trainium chips. Both moves appear designed to loosen Nvidia's software lock-in and offer developers more flexible hardware options.

Performance metrics from the new setup are driving much of the industry attention. A tuned vLLM configuration, which serves Moonshot AI's 2.8-trillion-parameter Kimi K3 model, achieved the top speed on the Ironwood TPU. This test, conducted by Inferact, the startup behind vLLM, used 16 Ironwood chips with speculative decoding. Speculative decoding uses a small draft model to speed up inference. Without that method, the new kernels for Kimi K3 and Qwen 3.8 27B still delivered roughly 1.4 to 2 times the throughput of the GB200 baseline at small batch sizes. These figures suggest that Google's hardware is becoming a serious contender for high-throughput inference tasks.

Looking ahead, the ecosystem supporting these chips is expanding rapidly. SGLang-TorchTPU is scheduled to add support for more models, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk is also working toward Day 0 TPU support for new open models. This goal means the hardware will be ready to run new models on their release day. The company aims to have this capability in place by the first quarter of 2027. Google, Meta, and RadixArk plan to share further details about this roadmap at the SGLang Summit 2026. The event will take place in San Francisco on November 12 and 13. As these chips become more accessible, the pressure on Nvidia to maintain its market position will only increase.

This development represents a milestone in the diversification of AI infrastructure. For years, the industry relied heavily on a single supplier for high-performance computing. The successful integration of Google's TPUs into Meta's production pipeline demonstrates that alternative architectures can now match or exceed the performance of traditional GPU setups in specific workloads. The ability to run PyTorch natively without significant code changes lowers the barrier to entry for other large technology companies. As more vendors adopt this approach, the market may see a shift away from a single dominant player. The coming months will likely bring further announcements as other major tech firms evaluate similar partnerships.