TMT Breakout梳理Hot Chips上Meta MTIA、NVIDIA网络/LPU与Cerebras CS4进展，缺少可直接交易的新增订单或业绩量级

*You are reading TMTB Pro. For info or a trial, e-mail sales@tmtbreakout.com*

## Meta MTIA 300 and 400

**Speakers: Nagesh Loke, Cindy Chen and Jatinder Singh, Meta infrastructure silicon**

Summary: Meta walked through MTIA 400, the generation where its accelerator program stops being recommendation-only and takes on GenAI alongside DLRM, across both training and inference. The talk covered the five-chiplet package, the 72-accelerator scale-up domain, in-LLC collective reduction and the memory hierarchy built for 2 TB embedding tables — plus a GPU-parity claim on the prior generation that came without a named comparison point.

### 1. MTIA 400 is built for two workloads that want opposite things

**

The earlier parts existed because sparse recommendation models starved GPU compute and the TCO did not work. MTIA 400 has to serve both that memory-bound world — embedding tables now at 2 TB, parameter counts beyond 150 billion — and dense GenAI throughput scaling across thousands of accelerators. As Cindy Chen put it, “the key shift here is that we’re no longer building for recommendation only.” That means delivering a high bytes-per-flop ratio and peak flops in the same chip, which is the central design tension of the part.

### 2. Meta claims GPU parity on training, but named no GPU

**

On a production 150 billion parameter DLRM, Meta says MTIA 300 achieves GPU parity on end-to-end training at competitive TCO. No vendor, part or configuration was given for the GPU it was measured against. On micro-benchmarks they cited table-batched embedding speedups of 1.87x on the forward pass and 1.88x on backward propagation, and volunteered that these land below the 2.5x bandwidth advantage the chip was designed for because skewed inputs are more cache- and instruction-bound than memory-bound.

### 3. The question they would not answer is the one that matters

**

An audience member asked directly how much of the sparse-specific silicon goes underutilized during GenAI training, given the dual mandate. The answer did not address utilization at all — it restated that Meta owns the whole stack and curates it for its own workloads. Two other questions went the same way: whether the on-chip fabric supports multicast and reduction drew an explicit disclosure limit, and a question on what unnecessary GPU features they deliberately left out drew a restatement of the low-MFU argument rather than a list.

### 4. The performance step is architectural, not a process shrink

**

MTIA 400 delivers 3 PFLOPS of FP16/BF16 and 12 PFLOPS of MX4, with 5x the compute on FP16 GEMM versus MTIA 300 and 1.3x HBM capacity. Jatinder Singh was explicit that this “is not just by putting more transistors into the chip” but comes from denser MAC arrays, MX4 support, memory hierarchy choices and the collective workflow. Concretely: 4x MAC density, internal registers grown to buffer an entire 256x256 matrix multiply so the MACs stay fully utilized, and new hardware-accelerated horizontal reduction so layer norm and softmax no longer need a transpose first.

### 5. Five chiplets, 72 accelerators, and a roadmap pointed at inference

**

The package is two compute chiplets flanked by HBM, one SoC chiplet for PCIe Gen 6 host connectivity and two networking chiplets carrying custom RoCE. Four accelerators per compute board build into a scale-up domain of 72 tightly coupled over RoCE, with scale-out across racks over a PCIe switch — and the team confirmed scale-up remains Ethernet-based. Two follow-ons were previewed with no dates, nodes or volumes: MTIA 450 focused on GenAI inference with double the HBM bandwidth and new low-precision types, then MTIA 500 with a further 50% bandwidth increase. Worth noting Meta shares its MX scale across 16 elements rather than the 32 in the OCP standard.

## NVIDIA Spectrum-X and the five networks of an AI factory

**Speaker: NVIDIA’s SVP of networking**

Summary: NVIDIA used its Hot Chips slot to introduce a fifth network domain, “scale-in,” which replaces the traditional front-end network and is accelerated by BlueField-4. The technical substance sat in multi-plane scale-out done entirely in NIC hardware, and in co-packaged optics now described as in production.

### 1. A fifth network domain, announced from the stage

**

The framing is that an AI factory carries five distinct network infrastructures, not one: scale-up (NVLink), scale-out (Spectrum-X Ethernet), scale-across (Spectrum-XGS, linking remote factories so one job runs across them), context scale (cheap fast access to storage for inference context) and now scale-in. The argument is that the front-end/back-end split no longer holds for agentic AI — the job is no longer connecting clients to servers but pulling data, models, applications and storage into the factory, so that infrastructure cannot stay isolated from the other four. Scale-in is accelerated by BlueField-4, which adds security acceleration, storage acceleration and the ability to run full context frameworks. He told the room twice to “remember this day.”

### 2. Co-packaged optics is in production, and the driver is power

**

Last year’s announcement is now shipping, with NVIDIA running the switches internally. The motivation is that an AI factory is “measured by tokens per power,” so power spent outside compute is waste — and optics already accounts for almost 10% of compute power on the scale-out network, with the amount of optics doubling generation over generation. Copper stays wherever distance allows. The specific choices: 200G SerDes rather than 100G, a small optical engine on micro-ring modulator technology, TSMC packaging for mass production, a custom laser source cutting laser count by 4x, and detachable surface-mount fiber attach. Claimed result is up to 5x lower power on the scale-out infrastructure.

### 3. Multi-plane scale-out only works because it is in hardware

**

Where multi-rail drags a NIC’s full bandwidth to one switch, multi-plane splits the NIC’s lanes across switches: on Vera Rubin, 1.6 Tb/s per GPU becomes eight lanes of 200G going to eight different switches — eight parallel networks. Claimed payoff is 64x more scale from the same switches, half a million GPUs on a two-tier fabric, and a GPU that drops to 90% of bandwidth rather than zero when a port fails. Asked what is new about multi-port, his answer was that the load balancing and per-plane congestion control run in NIC hardware rather than software: recovery to 90% takes under three milliseconds versus about a second in software — “a second that there is no traffic.”

### 4. The problem being solved is jitter, not bandwidth

**

His recurring argument is that there is no single Ethernet, and none of the existing flavors suits distributed computing, because every GPU must stay synchronized with every other. That cannot be a switch alone: spreading traffic across all routes avoids congestion but creates out-of-order delivery, and reordering is a separate job. So the switch distributes traffic in hardware based on its own and neighboring port conditions, the NIC restores order directly into GPU or CPU memory with no buffer copies, and lossless is preferred over trimming — “if I can avoid dropping packets, I’m avoiding creation of jitter.” His numbers: 1.2x better than off-the-shelf Ethernet on a single job, 2x in multi-tenant AI cloud, and 2x to 14x higher collective performance with the latest NCCL release.

### 5. NVLink Fusion is the answer to everyone building their own XPU

**

He pushed back on calling scale-up a network at all — “scale up is not a network; it’s a computing infrastructure” — on the grounds that reduction operations run inside it, so it carries real compute alongside bandwidth and near-zero jitter. He closed on NVLink Fusion: any third party building its own XPU gets access to all five infrastructures rather than having to build them. Read alongside the custom-silicon programs presented elsewhere at the same conference, that is the commercial point of the whole talk. Notably, asked about disaggregated resources, connections to remote HBM, and optical interconnect using lasers versus micro-LEDs, he answered none of it.

## NVIDIA’s LPU and the LPX rack

**Speakers: Igor, VP leading next-generation LPU silicon and systems, and Santosh, LPU architecture team, NVIDIA**

Summary: This is the Groq team, now inside NVIDIA, presenting the LPX rack — a specialised low-latency inference tier that sits on top of Vera Rubin NVL72 and is in production now. The talk covered a fully deterministic, software-scheduled architecture, a third-party benchmark against the rest of the fast-inference field, and three ways of splitting a workload between GPUs and LPUs.

### 1. A deterministic inference rack that deploys anywhere a GPU rack does

**

LPX is positioned as specialised compute layered onto the Vera Rubin platform, adding user interactivity into the thousands of tokens per second per user. The commercial detail that matters is the deployment story: a single-chip module requiring no expensive packaging, landing on a cableless tray with no retimers, fitting a standard MGX rack. As presented, “we can deploy this rack anywhere we can deploy a GPU rack,” and it is in production now — no new datacenter infrastructure, no exotic packaging supply chain.

### 2. The speed claim, with the self-reported and independent numbers separated

**

The live demo ran 11,000 tokens per second on a 31 billion parameter model at large context, which was explicitly flagged as a single self-reported benchmark. The number they leaned on instead came from third-party benchmarking with Artificial Analysis: 4x better performance than the rest of the competitors combined, at 100k context. Their framing is that this level of interactivity paired with large context unlocks applications that do not exist yet.

### 3. Three ways to split a workload between GPU and LPU

**

Measured on a 2 trillion parameter frontier model at roughly 400k context. First, external draft-based speculative decoding: a small drafter runs entirely on LPU SRAM generating tokens at very low latency, GPUs verify them, and because the drafter is trained separately in post-training the acceptance rates beat internal methods — giving 3x higher throughput at 1,000 tokens per second per user. Second, attention disaggregation: attention and KV stay on GPU HBM while other layers sit on the LPUs, extending to 3,000 tokens per second per user. Third, full LPU decode for latency-critical work, reaching 7,000 to 8,000.

### 4. Determinism pays a power and thermal dividend nobody else can collect

**

Because the compiler knows charge demand down to the nanosecond, they pre-deliver charge ahead of the load to cancel the di/dt droop — cutting voltage compression by about 60%, which lets them drop the voltage, and power falls with the square of that drop. They put it at double-digit power savings at datacenter level. The same trick applies to heat: rather than letting some blocks hit the thermal limit and throttle, the compiler equalises heat across blocks and schedules around it, yielding about 10 to 11% more performance inside the same thermal envelope.

### 5. The architecture is the inverse of a GPU, and so is the network

**

Flat SRAM memory, no caches, no out-of-order execution, no branch prediction — fully deterministic, with instruction issue taking only 3 to 4% of die area and the rest given to compute, memory and die-to-die. Because execution is software-scheduled, the compiler reasons about every data movement. That extends to the network: each chip is both router and processor, eliminating switches and shrinking network diameter, with no adaptive routing, no congestion sensing, and reduced packet encoding since source and destination addresses are unnecessary. A rack holds 256 LPUs at 350 PFLOPS with 40 PB/s of aggregate SRAM bandwidth, gathering a terabyte across chips in under three microseconds.

## Cerebras CS4 and the Nexus rack platform

**Speaker: JP, co-founder and chief system architect, Cerebras Systems**

Summary: Cerebras introduced CS4, the first system to bring three wafer-scale engines together, built on a new modular rack platform called Nexus. The talk was heavily engineering-led — power delivery, wafer I/O, packaging and cooling — with a direct comparison against Rubin NVL72 and an unusually specific multi-year roadmap.

### 1. CS4 is in early access now, generally available later in Q3

**

The claim is 2x faster tokens and 10x more token capacity than CS3, and since they position CS3 as already up to 15 times faster than GPUs, CS4 takes that to up to 30x. The on-stage demo asked GPT-OSS to build a periodic table in HTML on CS4, CS3 and a GPU simultaneously: CS4 finished first at over 4,400 tokens per second, CS3 at over 2,300, with the GPU still running. Note the company is now public, which changes how these claims should be read.

### 2. Three wafers in one system, and a rack architecture that separates the axes

**

CS4 delivers 6x the system compute, memory bandwidth, fabric and I/O bandwidth of CS3, all at half the I/O latency. It is the first system on the Nexus platform, which splits power, compute and I/O into self-contained units so each can be upgraded independently — power at the front of the rack in modular supplies, compute at the rear in three pluggable “backpacks,” each holding one wafer-scale engine. The redesign carries 50% fewer components than CS3.

### 3. The power delivery work is the genuinely differentiated engineering

**

On a GPU, power converters sit around the package roughly 15 millimetres from the silicon. On CS4 the DC-to-DC converters sit 0.5 millimetres from the wafer — a claimed 100x reduction in power delivery distance versus GPUs and 10x versus CS3 — achieved by eliminating the printed circuit board from the power path entirely and placing AC-DC conversion a short uniform distance from the busbar. That is why the backpack is vertical rather than horizontal. The result is nearly 2x the power delivered to the wafer at almost the same voltage, with almost no resistive loss, which matters because resistive losses scale with the square of current.

### 4. The pitch is aimed squarely at NVL72’s cabling

**

He put a Rubin NVL72 rack on screen, pointed at the thousands of cables connecting the GPUs, and asked what they cost in performance, power, dollars and reliability. The argument is that on a wafer-scale system all of that cross-chip communication happens on the wafer instead: no cables, lower power, lower cost, higher reliability. The supporting numbers on wafer I/O: 2.4 Tb/s per wafer, double CS3; a new packet pipeline at 3 microseconds of network latency against 5 in CS3; and a direct wafer-link that bypasses the network entirely for wafer-to-wafer latency of 2 microseconds.

### 5. The roadmap is aggressive, and he flagged the aggressive part himself

**

Nexus was co-designed to carry multiple generations, and the stated intent is to double generation speed every year for several years through CS5 and CS6. CS5 arrives next year on a brand-new wafer, targeting 10,000 tokens per second per user on smaller models and up to 5,000 on multi-trillion parameter frontier models. He said outright that part of this “is still just a dream.” The architectural step he named beyond it is 3D DRAM stacking: wafer-scale SRAM keeps scaling, but DRAM would shrink the system footprint by an order of magnitude.