TMT Breakout 复盘 Hot Chips 上 AMD、NVIDIA、Intel 的 AI 芯片与机架架构演示

Some good presentations today:

-

**AMD MI455 / Helios System:** Steve Scott, David and Krishna, AMD

-

**NVIDIA Vera CPU:** Jonathan Evans and Polychronis Sakalakis, NVIDIA

-

**Intel Diamond Rapids Xeon:** Krishna, Intel Fellow and Chief Architect for Xeon CPUs

-

**Intel Crescent Island GPU:** Sumit and Hongjie, Intel

-

**Waymo Keynote / Physical AI Roadmap:** Daniel, Waymo

-

**Waymo Sensor Fusion Processor ASIC:** Sabrash and Peter, Waymo

-

**IBM Z Dual-ISA CPU + AI Accelerator:** Christian, IBM

-

**Fujitsu Monaka CPU:** Fujitsu Monaka architecture team

-

**Intel Wildcat Lake / Core Series 3:** Intel Core Series 3 lead SoC architect

-

**BOS Semiconductors Eagle-N Automotive AI Accelerator:** K. Ming, BOS Semiconductors

## **AMD MI455 and the Helios rack-scale AI system**

**Speakers: Steve Scott, David and Krishna, AMD**

Summary: AMD presented Helios as a 72-GPU rack-scale system built around MI455 GPUs, EPYC CPUs, Infinity Fabric and programmable Pensando networking. The focus was memory-semantic Ethernet scale-up, full-rack resilience, liquid cooling and the transition from electrical to optical scale-up.

**1. Helios turns 72 MI455 GPUs into one tightly coupled system**
**

Helios consists of 18 nodes, each containing one EPYC CPU and four MI455 GPUs. AMD said the rack delivers almost 1.7 PB/s of aggregate HBM bandwidth, more than 0.25 PB/s per direction across the scale-up network and over 40 TB/s of scale-out bandwidth. The architecture is designed so the full rack behaves like one system rather than a collection of independent servers. AMD uses one CPU per four GPUs, arguing that extensive modeling showed this provides the best price/performance and enough CPU throughput to keep the accelerators fed.

**2. AMD is carrying direct memory operations over Ethernet**
**

The MI455 GPUs expose direct loads, stores and atomics across the scale-up pod. AMD packages those memory transactions into Ethernet frames and routes them through AVGO Tomahawk 6 switches, with at most one switch hop required to reach any GPU memory in the rack. Once remote memory is exported and imported into another GPU’s virtual address space, software can access it much like local memory, although with a different performance profile. DMA engines inside the GPUs handle much of the data movement without tying up compute resources.

**3. This is likely AMD’s last fully electrical scale-up generation**
**

AMD said Helios is near the practical limit for an all-electrical scale-up fabric. Electrical connectivity forces GPUs, switches and cabling into a very dense physical configuration because reach is limited. AMD expects to begin introducing optical scale-up in the following generation, which should loosen the coupling between the compute topology and rack layout. Optics would allow more physical separation and packaging flexibility, rather than requiring every accelerator and switch to remain packed into one tightly constrained electrical domain.

**4. Helios assumes direct liquid cooling, but AMD does not expect a major further jump in rack density**
**

More than 85% of Helios heat is removed through direct liquid cooling, with liquid-cooled compute and switch trays connected through quick disconnects. Asked how rack power evolves, AMD said different data-center operators have different preferred operating ranges, but it does not expect a substantial increase in power density from current levels. The move to optical scale-up should reduce the need to keep pushing physical density simply to preserve signal reach, giving system builders more flexibility in how power and cooling infrastructure are arranged.

**5. The Pensando AI NIC is designed around post-deployment programmability**
**

Each GPU can connect directly to multiple 800G Pensando AI NICs, providing up to roughly 2.4 Tb/s of scale-out networking per GPU. AMD deliberately avoided a fixed-function networking design because AI transports, congestion-control methods and telemetry requirements are changing too quickly. The NIC uses programmable packet-processing and DMA pipelines for virtualization, encryption, RDMA transport, load balancing and telemetry. Protocols and congestion algorithms can continue evolving through firmware after the system has already shipped.

**6. Multipathing, congestion control and telemetry are becoming part of the accelerator architecture**
**

AMD described spreading one GPU-to-GPU flow across as many as 128 or 256 paths to improve network utilization and avoid low-entropy traffic patterns. The programmable transport can support selective retransmission, remove degraded paths, probe for recovery and implement customer-controlled source routing. Congestion control can react to RTT, packet loss or ECN markers on every packet rather than waiting for a host CPU or driver. Hardware telemetry can track statistics by interface, queue pair or path and stream only the events customers choose to monitor.

**7. Full-rack confidential computing and failure isolation are built into the design**
**

AMD said Helios supports confidential computing across the rack using packet-level AES-GCM encryption, device attestation, secure partitioning and protected execution environments. The rack can be divided into virtual pods that provide performance, security and fault isolation. The scale-up fabric is designed to survive the loss of an individual link, switch ASIC or complete switch tray. Traffic is rerouted and lost packets retransmitted automatically, while failed switch trays can be replaced online and brought back into service without shutting down the entire rack.

**8. AMD is using Ethernet’s ecosystem while retaining a proprietary transaction layer**
**

AMD’s argument for Ethernet is that it provides mature switches, cables, connectors, management tools and a large supplier ecosystem. The memory transaction protocol above Ethernet is still customized for this use case, with lightweight headers, dynamic packet packing and end-to-end retransmission. AMD distinguished this approach from the consortium-developed UALink protocol: both carry memory transactions, but AMD’s design packetizes them into Ethernet frames and transports them reliably end to end, while UALink uses a different hop-by-hop model.

## **NVDA Vera CPU for agentic AI**

**Speakers: Jonathan Evans and Polychronis Sakalakis, NVDA**

Summary: NVDA designed Vera around the view that agentic AI makes the CPU a critical part of the inference loop again. The architecture prioritizes high single-thread performance, predictable throughput under load, massive coherent bandwidth and rack-scale confidential computing.

**1. Agentic AI turns one query into a full-system workload**
**

NVDA said a single agent task can involve 20, 50 or more chained LLM calls, along with orchestration, tool use, browser execution, security, storage and memory management. Much of that orchestration runs on the CPU, while the GPU handles token generation. The repeated back-and-forth means CPU latency compounds across the full task and can leave an expensive GPU waiting. NVDA therefore views agentic AI as fundamentally different from a chatbot request: it simultaneously stresses CPU, GPU, networking, memory, storage and security.

**2. NVDA says throughput per megawatt is now the key AI-factory metric**
**

NVDA framed AI-factory performance around user interactivity and tokens generated per megawatt, arguing that tokens ultimately determine revenue while power is the binding infrastructure constraint. The company said Vera Rubin delivers up to roughly 30x greater throughput per megawatt than Blackwell on a long-context agentic coding workload. It also discussed a latency-focused companion accelerator that has shown approximately 4x higher token speed than the nearest alternative in third-party testing at context lengths above 100,000 tokens.

**3. Vera deliberately favors large, high-performance cores over maximum core count**
**

NVDA argued that cloud CPU design has focused too heavily on dollars per core, leaving single-thread performance relatively flat. Agentic workloads change the optimization target because the CPU sits in latency-sensitive loops and must keep Rubin GPUs supplied with work. Vera therefore uses fewer, much larger Olympus cores rather than a large number of smaller efficiency cores. NVDA’s goal is not simply high benchmark performance at low utilization, but strong per-thread performance while the entire CPU is loaded.

**4. Olympus is an unusually wide and deep CPU core**
**

The Olympus core includes a large neural branch predictor, support for two taken branches per cycle, a 108-byte instruction fetch path and decoding of more than 10 ARM instructions per cycle through fusion. The backend can hold more than 1,000 instructions and issue four loads, two stores and six vector operations per cycle. It also includes a 96 KB L1 data cache, large translation structures, aggressive prefetching and a dedicated graph prefetcher for the graph-heavy workloads NVDA expects in agentic systems.

**5. Spatial multithreading is designed to reduce noisy-neighbor effects**
**

Instead of conventional SMT, where two threads compete for many of the same structures, Olympus uses what NVDA calls spatial multithreading. Each thread receives largely separate pipelines, branch prediction, schedulers and register resources, while sharing selected expensive structures such as caches and floating-point units. NVDA said this provides more stable performance when a second thread becomes demanding, helping preserve service-level objectives. The design also reduces some of the side-channel and interference issues that can arise when unrelated threads share more of the same microarchitectural state.

**6. Vera uses one large compute die rather than distributing cores across chiplets**
**

Vera is built from six dies: one large compute die, four LPDDR5X memory dies and one system I/O die. NVDA chose a monolithic compute die to avoid the latency, power and bandwidth penalties of moving core-to-core and core-to-cache traffic across chiplet boundaries. All cores share a flat 164 MB last-level cache, while the internal coherency fabric provides up to 3.4 TB/s of bisection bandwidth. The separate memory dies connect through a low-latency NVLink C2C variant.

**7. Memory and I/O bandwidth are sized to keep Rubin continuously supplied**
**

Vera provides approximately 1.2 TB/s of LPDDR5X bandwidth, which NVDA said delivers much better bandwidth per watt than conventional server memory. The CPU also includes 96 lanes of PCIe 6.0 with CXL 3.1 support, totaling roughly 1.5 TB/s of bidirectional I/O bandwidth. Its second-generation NVLink C2C connection provides around 1.8 TB/s of raw CPU-GPU bandwidth. NVDA’s design objective is to prevent the CPU, memory or I/O subsystem from becoming the bottleneck for the Rubin accelerators attached to it.

**8. Confidential computing is implemented across the CPU, GPU and rack**
**

Vera Rubin is designed for rack-scale confidential computing rather than protection limited to one processor. The CPU and GPU can validate that requests belong to the correct VM, while each VM receives its own encryption context. Data is encrypted in memory and across links but can be processed in plaintext inside protected CPU and GPU domains. NVDA said the hardware works with its software security stack to protect models, agent context, user data and inference across the full AI factory. NVDA also cited a roughly 6x performance gain for Redpanda on selected streaming tests versus traditional CPUs.

## **INTC Diamond Rapids and the new fan-out Xeon fabric**

**Speaker: Krishna, INTC Fellow and chief architect for Xeon CPUs**

Summary: INTC introduced a new fan-out architecture that separates scalable compute building blocks from centralized memory and I/O hubs. Diamond Rapids is the first implementation, combining 256 P-cores, 1.28 GB of shared cache, 16 memory channels and 128 PCIe 6.0 lanes.

**1. The fan-out architecture separates compute from memory and I/O**
**

Each scalable compute building block contains multiple core chiplets stacked on a base die. The core chiplets hold the CPU cores and private caches, while the base die carries the shared last-level cache and interfaces to the centralized fabric hubs. INTC can vary both the number of core chiplets within a building block and the number of building blocks in the final package. Memory controllers, coherency, CXL expansion, I/O and accelerators are moved into separate fabric hubs rather than repeated inside each compute tile.

**2. Uniform memory latency was the central architectural priority**
**

Every compute building block connects directly to every memory and I/O fabric hub, avoiding intermediate hops through neighboring compute or I/O tiles. Transactions are distributed across the hubs by address, but all cores retain direct access to the full memory subsystem. INTC said this provides low and relatively uniform memory latency across the package, rather than creating cores that are physically closer to some memory channels than others. The same objective drove the decision to use the UCIe-S package connection rather than a denser but shorter-reach alternative.

**3. Diamond Rapids scales to 256 P-cores and 1.28 GB of shared cache**
**

The first implementation includes four compute building blocks containing a total of 16 compute chiplets. Across the package, Diamond Rapids supports up to 256 high-performance P-cores and approximately 1.28 GB of last-level cache. The memory system has 16 channels supporting DDR5-8000 and faster MRDIMM configurations, with peak bandwidth around 1.6 TB/s. I/O includes 128 lanes of PCIe 6.0 plus eight additional PCIe 4.0 lanes for system management and infrastructure.

**4. The package combines 3D hybrid bonding with disaggregated fabric hubs**
**

Four compute chiplets are hybrid-bonded onto each base die, creating a 3D fabric spanning routers in both layers. INTC said the compute chiplets use an Intel 18A-family process, while the base dies use Intel 3 with TSV support. The separate memory and I/O hubs are connected through UCIe-S. This construction lets INTC use advanced process technology primarily where it benefits CPU performance while placing cache, routing and I/O functions on more suitable process nodes.

**5. The memory hubs support both direct and hardware-managed CXL expansion**
**

Each unified memory fabric contains conventional memory controllers plus memory-value functions for CXL expansion. INTC supports a flat one-level model where expanded memory is directly visible to software and a hardware-managed two-level model that can use one memory tier as a cache for another. The same fabric hub also carries memory encryption, coherency agents and on-die snoop filters. INTC’s intent is to make large memory pools part of the native CPU fabric rather than an add-on hanging behind a separate subsystem.

**6. Power management is distributed by workload domain**
**

The hierarchical design creates separate voltage and frequency domains for cores, cache, memory and I/O. Hardware monitors I/O bandwidth, memory bandwidth and core sensitivity to memory latency, then adjusts the relevant domains independently. Diamond Rapids also supports prioritized cores, allowing software to identify a small group of CPU cores that feed an attached GPU and keep those cores at higher frequency and priority. The remaining CPU resources can operate at lower power without starving the accelerator on the critical path.

**7. APEX modernizes the x86 instruction set**
**

Diamond Rapids introduces Advanced Performance Extensions, or APEX, including additional general-purpose registers, three-operand instructions, flag separation and new conditional capabilities. INTC said these changes reduce the number of instructions required to perform the same work and have already been upstreamed into open-source and commercial compilers. The core also retains AMX and AVX acceleration, while the I/O hubs include QAT for compression and cryptography, DSA for data movement and IAA for analytics and scanning.

**8. Security and RAS are integrated into the shared fabric**
**

Diamond Rapids supports TDX for protected VMs, SGX for application-level enclaves and the broader Xeon security stack. INTC highlighted an on-die snoop filter that avoids prior trade-offs between memory integrity metadata and ECC capacity. Keeping this function on the package also lowers the latency required to acquire ownership of cache lines. The architecture includes I/O caching, memory encryption and distributed error telemetry, with centralized policy and local power or recovery decisions across the chiplets.

## **INTC Crescent Island for high-capacity agentic inference**

**Speakers: Sumit and Hongjie, INTC**

Summary: Crescent Island is INTC’s 350W inference GPU built around the new Xe3P architecture and unusually large LPDDR5X capacity. INTC is targeting agentic and sparse-model workloads that need much more memory capacity without the cost and power of HBM.

**1. Crescent Island offers up to 480 GB of LPDDR5X at roughly 350W**
**

INTC’s reference card has a lower standard capacity, while ODMs and system partners can configure Crescent Island with approximately 160 GB to 480 GB of LPDDR5X per GPU. The chip is optimized around a 350W envelope so it can run in relatively simple, passively air-cooled data-center systems. INTC chose LPDDR5X because it provides the density needed for large models and long contexts while preserving power efficiency and lowering cost versus HBM-based alternatives.

**2. Sparse models are decoupling memory capacity from active bandwidth demand**
**

INTC argued that newer sparse MoE models can become much larger in total parameter count without increasing the number of parameters activated for every token by the same amount. The full model, KV cache and multiple sessions still need to fit in memory, but only a subset of weights may be read during each inference step. That shifts the design requirement toward much higher capacity rather than scaling bandwidth linearly with total model size. Crescent Island is specifically designed around that emerging capacity-heavy workload profile.

**3. Xe3P increases matrix-compute density and broadens precision support**
**

Crescent Island is the first INTC GPU based on Xe3P, which is optimized more directly for AI inference than the prior Xe generations. The architecture expands the systolic depth from four to 16 and supports formats ranging from FP4 through FP64, including newer low-precision MX formats. The full GPU contains 32 Xe cores and 256 matrix engines organized into four compute slices, with a 32 MB L2 cache placed centrally to reduce access latency and improve power delivery and heat distribution.

**4. Large register files and shared memory support deeper kernel fusion**
**

INTC expanded the register and local-memory resources available to each core so inference kernels can fuse more operations without spilling intermediate data into slower memory. The vector and matrix pipelines share the same register file, helping combine matrix operations with softmax, normalization, activation and other elementwise work. Native sigmoid and related math support can remove polynomial approximations from inner loops. INTC also introduced a single-address queue intended to run prefill, decode and draft-model stages with less host synchronization.

**5. Idle power matters when large accelerator fleets are underutilized**
**

Crescent Island targets roughly 10W in a low-power idle mode, compared with a 350W active design point. INTC emphasized that large inference fleets do not keep every accelerator fully loaded at all times, making idle and partial-utilization efficiency meaningful at data-center scale. The chip uses distributed power management to follow the best voltage-frequency operating point and places compute around the central cache and fabric to improve both thermal dissipation and power delivery.

**6. The design supports standard PCIe scaling and multimodal inference**
**

Crescent Island connects to hosts and other accelerators through standard PCIe rather than requiring a proprietary rack fabric. The GPU also includes multiple video encode and decode engines so multimodal workloads can compress and decompress data on the fly rather than consume additional host resources or I/O bandwidth. INTC’s intended workloads include language models, reasoning models, vision and multimodal inference, with multiple GPUs connected within or across nodes through standard switching infrastructure.

**7. RAS extends from the execution cores through LPDDR memory**
**

INTC described ECC and parity protection across on-chip SRAMs, memory structures and the internal fabric. Fault isolation can occur down to the Xe-core level, while poison propagation helps prevent corrupted data from being silently consumed elsewhere in the system. LPDDR memory uses ECC across the data path, and the platform supports error injection and PCIe error reporting so failures can be tested and surfaced through system software.

**8. INTC is leaning on its existing Xe software ecosystem**
**

Crescent Island uses the same broad Xe software foundation that already supports INTC client and professional GPUs. INTC plans open-source support for leading inference and serving frameworks, day-zero model enablement and heterogeneous agent orchestration across CPUs, GPUs and other accelerators without requiring developers to rewrite agent code. The strategy is to compete through high capacity and ease of deployment rather than requiring customers to adopt an entirely new programming environment.

## **Waymo’s physical-AI and autonomous-driving compute roadmap**

**Speaker: Daniel, Waymo**

Summary: Waymo described how physical AI differs from data-center AI, with millisecond latency, constrained onboard compute and a much higher cost of failure. The keynote covered Waymo’s deployment scale, multimodal sensing, custom silicon, liquid-cooled redundant compute and the move toward larger foundation models in the vehicle.

**1. Waymo says autonomous driving has reached a deployment inflection point**
**

Waymo has now driven more than 200 million miles fully autonomously across 15 cities. The first 100 million autonomous miles took roughly 15 years, while the second 100 million took around seven months. The company also recently launched in four cities in one day, versus roughly eight years to reach its first four. Waymo said its vehicles have experienced around 13x fewer accidents than human-driven vehicles regardless of fault. The fleet currently numbers roughly 4,000 to 5,000 cars, with materially more vehicles expected over the next several years.

**2. Physical AI has a very different compute envelope from cloud AI**
**

Waymo wants to run the largest models possible, but all compute must fit into roughly a pizza-box-sized module inside the car. Decisions often need to occur within milliseconds, rather than the seconds or minutes acceptable for some cloud models. Errors also have physical consequences, so unusual artifacts or occasional reasoning failures are not acceptable. Driving decisions are made locally in the vehicle rather than sent to a data center, although connectivity remains important for operations, user features, software and fleet management.

**3. Waymo remains fully committed to multimodal sensing**
**

Waymo’s objective is a superhuman driver, not a system that merely matches human vision. Its vehicles combine cameras, lidar, radar and microphones because each modality covers weaknesses in the others. Cameras provide color and semantics but can struggle with darkness, glare or dust. Lidar adds precise geometry, while radar provides velocity and can remain useful in poor weather. Waymo showed cases where pedestrians were difficult or impossible to identify from cameras alone but became clear when lidar or radar was included.

**4. ML demand continues rising even as CPU and general-purpose GPU use flatten**
**

Waymo said CPU and GPGPU requirements have fluctuated and are relatively flatter than the continued growth in dedicated ML compute. Some planning and perception workloads that previously ran on CPUs or GPUs have migrated into learned models. More onboard ML capacity improves both driver performance and software-development velocity because model teams spend less time squeezing workloads into hard latency limits. Waymo uses a mix of partner silicon and custom chips, including lidar-processing and early sensor-fusion ASICs optimized for its own sensors and low-batch workloads.

**5. The onboard computer has used liquid cooling for more than a decade**
**

Waymo said its vehicles have used liquid cooling since roughly 2014. The system can tap into the vehicle’s existing thermal infrastructure, but automotive cooling is much harsher than chilled data-center water. A stationary vehicle in Phoenix can see coolant temperatures around 65°C after sitting in the sun. The compute module uses a central cold plate with mirrored hardware on both sides. Under normal operation both sides contribute, but the vehicle can fail over to one side and continue safely in a degraded mode if power or hardware is lost.

**6. Waymo is adopting a “think fast, think slow” foundation-model architecture**
**

The fast path ingests sensor data and handles safety-critical decisions at very low latency. A slower and larger model performs deeper semantic reasoning for situations that do not require a response within a few milliseconds, such as deciding whether it is safe to drive through water from a broken hydrant. These encoders feed a decoder that produces actions and trajectories. Waymo trains larger models in the data center and distills them for onboard deployment, while openly asking how it could eventually run a trillion-parameter model within one vehicle.

**7. Simulation is essential for the long tail**
**

Waymo uses foundation and world models to simulate situations that are rare or difficult to stage safely, including aircraft landing on a road, escaped animals, snow on the Golden Gate Bridge and secondary freeway collisions. The system models other agents, sensor outputs and physical properties such as road friction and visibility. Before a software release reaches vehicles, Waymo runs billions of simulated miles alongside closed-course and real-world testing. The goal is not to pre-program every scenario, but to build models that can reason through events the company never explicitly anticipated.

**8. Custom silicon is already a technical win and becomes an economic win at scale**
**

Waymo said custom chips are expensive and require long-term investment in architecture, compilers and model optimization. The company believes the current technical benefits are already meaningful because custom silicon lowers latency, improves sensor quality and lets software development begin earlier against a stable target. The economic case strengthens as the fleet expands. Waymo is now focused on integrating its pizza-box-sized computer toward a laptop-sized form factor, increasing reliability and supporting new operating domains including snow, London and Tokyo.

## **IBM’s dual-ISA Z/ARM processor and next-generation AI accelerator**

**Speaker: Christian, IBM**

Summary: IBM introduced a single enterprise CPU core that natively executes both the Z and ARM64 instruction sets, with each thread able to switch modes in nanoseconds. IBM also detailed a new HBM3E accelerator designed for agentic and generative-AI inference on mainframes.

**1. One physical core can execute both Z and ARM64 software**
**

IBM described the processor as the first commercial dual-ISA enterprise CPU in which both architectures are first-class citizens. It does not place separate Z and ARM cores on the same die. Instead, each hardware thread can independently switch between Z and ARM64 mode within nanoseconds. The two threads in one SMT core can also run different ISAs simultaneously while competing for shared core resources. IBM added a dedicated instruction to trigger the mode change and has already upstreamed the required Linux kernel patches.

**2. IBM claims the highest base frequency of any server processor**
**

The 2nm chip runs at a base frequency of 5.7 GHz or higher and contains 11 cores with two-way SMT. Each core has 36 MB of L2 cache, which contributes to a 432 MB virtual L3 across the chip. Multiple processors can form a system-level virtual L4 of roughly 3.5 GB. IBM said all of these performance characteristics apply equally to Z and ARM software, including the mainframe’s branch prediction, translation structures, cache hierarchy and reliability mechanisms.

**3. ARM operating systems and applications run without an IBM-specific port**
**

IBM’s objective is participation in the broad ARM software ecosystem rather than creating a special mainframe variant of ARM. The core implements ARMv9.3 features, including SVE and SVE2, and is designed for ARM SystemReady compliance. ARM software sees a normal little-endian ARM64 environment and can run unmodified. IBM said no other ISA made sense for this ecosystem-expansion effort because ARM provided the breadth of applications and partnership required to bring modern software closer to existing mainframe workloads and data.

**4. Most execution hardware is shared, but decode and architectural state are expanded**
**

IBM reused existing Z execution structures whenever the semantics aligned and added dedicated logic where ARM differed. Addition, memory access and many vector operations can share the same underlying machinery, while ARM required separate decode, translation and exception behavior. IBM added support for 2,792 ARM instructions and 239 system registers, more than doubling the preexisting Z instruction count. The core remains mostly big-endian internally, with the ARM little-endian view handled through memory data paths and architectural controls.

**5. Z and ARM virtual machines can coexist in the same hypervisor**
**

IBM’s mainframes traditionally support hardware partitions plus a second virtualization layer for large numbers of operating systems. The new design lets KVM and OpenShift run both s390x and ARM64 Linux images inside the same hypervisor. Whenever KVM schedules a virtual CPU, the physical hardware thread switches into the appropriate ISA mode. IBM said the nanosecond-scale transition adds virtually no overhead, allowing ARM applications to coexist directly beside core Z transaction-processing workloads.

**6. Mainframe availability mechanisms now apply to ARM workloads**
**

IBM said the platform is engineered for nine nines of application availability, equivalent to roughly 30 milliseconds of downtime per year. Arrays, datapaths and control structures include pervasive error detection. A core can roll back to its last completed architectural checkpoint after a transient failure, or transfer that checkpointed state to a spare core if the fault persists. Processor modules and memory components can also be replaced while the rest of the system remains operational. ARM software inherits the same recovery, isolation and serviceability mechanisms as traditional Z workloads.

**7. The new AI accelerator shifts sharply toward memory bandwidth**
**

IBM’s next PCIe-attached accelerator includes 16 active AI cores plus one redundant core and adds FP4, MXFP4 and other lower-precision formats. FP8 throughput improves by up to 4x versus the prior design, but the larger change is memory. The accelerator carries 96 GB of HBM3E and more than 4 TB/s of bandwidth, which IBM described as a 20x increase. PCIe 6.0 x16 and peer-to-peer connectivity support sharded models, large KV caches and direct communication with other accelerators or memory.

**8. IBM is treating AI as mission-critical enterprise infrastructure**
**

IBM’s design protects models, enterprise data, software and intermediate state at rest, in transit and in use. Confidential computing secures the host environment, secure pass-through extends protection to the accelerator, and quantum-safe secure boot validates critical firmware. The accelerator also includes redundant compute capacity and integrates with the mainframe’s serviceability and operational controls. IBM’s argument is that enterprise AI must provide security, resilience, governance, availability and auditability rather than focusing only on raw inference performance.

## **Waymo’s sensor-fusion processor ASIC**

**Speakers: Sabrash and Peter, Waymo**

Summary: Waymo detailed a custom ASIC for the low-latency “think fast” portion of its autonomous-driving foundation model. The chip combines a custom ML accelerator, camera processing, radar/lidar compute and software-managed memory around the requirements of Waymo’s own sensors.

**1. The ASIC is built specifically for low-batch, hard real-time inference**
**

Waymo divides its onboard model into a fast sensor-fusion path and a slower semantic-reasoning path. This ASIC accelerates the fast encoder, where high-resolution camera, lidar and radar data must be processed within milliseconds. Because Waymo designs both sensors and algorithms, it can co-design the full path from photons through sensor readout and into the ML model. The chip is PCIe attached and can be deployed in multiples, allowing Waymo to balance manufacturing yield against the number and complexity of sensors in a particular vehicle.

**2. Waymo prioritizes precision and achieved latency over headline TOPS**
**

The custom ML engine provides roughly 160 TOPS of dense INT8 performance, with FP16 at half rate and FP32 at quarter rate. It also supports mixed-precision computation and structured 50% sparsity at approximately twice the dense rate. Waymo retained relatively high-precision formats because its input is raw sensor data and small losses in detail can hurt model quality. The company emphasized achieved latency and useful performance per theoretical TOP rather than simply maximizing the peak arithmetic number on the specification sheet.

**3. Two accelerator cores can operate independently or combine on one inference**
**

Each accelerator core contains an 8-by-2 grid of processing elements, and the ASIC contains two such cores. The cores can run separate inference tasks with performance isolation or cooperate on one larger latency-critical model. The operating mode is selected dynamically at runtime. Waymo also supports preemption, allowing an inference context to be paused and restored. One use is running software-test workloads during otherwise unused periods without delaying the safety-critical processing path.

**4. The processing elements expose compute, SRAM and communication directly to software**
**

Each processing element contains software-managed SRAM, matrix/vector compute and direct connections to neighboring elements through a bidirectional mesh. A wider ring distributes weights and activations across the core. DMA engines independently move data over the ring and mesh, allowing weight loading, computation and partial-sum reduction to overlap. Waymo chose a matrix-vector primitive because it can map efficiently onto matrix multiplication, convolution, reductions and elementwise operations while retaining flexibility across the mixture of transformer and convolutional models used in sensor fusion.

**5. Static model shapes let Waymo replace complex hardware scheduling with mega-instructions**
**

Waymo’s sensor-fusion models have predictable shapes and control flow, allowing the chip to simplify dynamic scheduling. Software programs hardware threads through “mega-instructions” that describe roughly 1,000 cycles of iteration, synchronization and tensor movement. FIFO and counting-semaphore primitives coordinate those threads. The compiler builds the complete model graph ahead of time into one large mega-kernel, including cross-device synchronization and shared buffers, avoiding repeated host launches and reducing latency variability.

**6. LPDDR5X and scratchpad memory were sufficient for this workload**
**

Waymo chose in-package LPDDR5X rather than HBM because the sensor-fusion encoder does not require the bandwidth of the larger reasoning and generative models elsewhere in the stack. Large on-chip SRAM keeps activations resident, leaving most DRAM bandwidth available for weights. Separate scratchpad memory holds high-bandwidth, low-footprint intermediate radar and lidar data so those stages do not overwhelm the external-memory subsystem. Waymo said the memory architecture is designed for deterministic real-time traffic rather than maximizing general-purpose throughput.

**7. The camera pipeline is tightly coupled to simulation and replay**
**

The custom camera path handles sensor errors, converts raw images into RGB and YUV, performs temporal denoising, creates image pyramids and extracts dynamically selected patches for ML models. It also supports high dynamic range for cases such as bright headlights or driving out of a dark tunnel. Waymo deliberately applies lossy image compression inline before perception so recorded sensor data can later be replayed with the same compression artifacts seen by the vehicle, improving simulation fidelity and reproducibility.

**8. Radar and lidar run on a programmable GPU rather than a dedicated DSP**
**

Radar processing includes multidimensional FFTs and irregular memory patterns, while lidar workloads process large point clouds in parallel. Waymo uses an embedded GPGPU for both rather than a DSP, saying the GPU provides a more familiar programming environment and enough flexibility for evolving algorithms. The company reported that targeted sensor-fusion models outperform an industry-standard off-the-shelf accelerator on achieved performance efficiency after compiler optimization and model co-design. Non-target model families also improved, suggesting some of the software optimizations generalize beyond the original ASIC workload.

## **Fujitsu Monaka: a 2nm, 3D-stacked Arm CPU for AI and HPC**

**Speaker: Fujitsu Monaka architecture team**

Summary: Fujitsu presented Monaka as a 144-core Arm data-center CPU optimized for performance per watt, sovereign computing and mixed AI/HPC workloads. The design stacks 2nm compute dies on a 5nm I/O die through face-to-face hybrid bonding.

**1. Fujitsu uses 2nm silicon for less than 30% of total chip area**
**

Monaka separates the functions that benefit most from leading-edge logic from those that do not scale as effectively. The compute dies use a TSMC 2nm process, while the larger I/O and cache-related structures use a 5nm process. Fujitsu said less than 30% of total silicon area is built on the expensive 2nm node. The approach is intended to optimize cost, reduce development risk and accelerate time to market while still capturing the frequency and power benefits of leading-edge transistors for the CPU cores.

**2. Monaka combines 144 cores with 256-bit SVE2 execution**
**

The CPU supports 144 Arm cores per package and uses 256-bit SVE2 units designed for AI and HPC workloads. Fujitsu also described 12-channel DDR5 memory, PCIe 6.0-class connectivity and confidential-computing support. The microarchitecture includes wide instruction and load paths, advanced branch prediction, large translation structures and reliability protection through ECC, parity and hardware recovery from transient faults. Fujitsu is targeting both conventional CPU workloads and orchestration or preprocessing around AI accelerators.

**3. The compute die sits on top to shorten the thermal path**
**

Fujitsu places the power-hungry compute dies on the upper side of the 3D stack, closer to the cooling solution. The compute and I/O dies are connected face to face through hybrid bonding, reducing latency between the CPU cores, cache and I/O structures. The lower die also contains integrated capacitive structures that support cleaner and more efficient power delivery into the compute layer. The architecture attempts to solve thermal, latency and power-delivery problems together rather than treating 3D stacking purely as a density technique.

**4. Ultra-low-voltage operation is intended to deliver another process-generation-equivalent power gain**
**

Fujitsu said Monaka can operate at a voltage approximately 30% below a normal design point, with around a 50% reduction in power under those conditions. Because dynamic power falls with the square of voltage, Fujitsu views low-voltage operation as providing an energy benefit comparable to moving an additional process generation beyond 2nm. Achieving stable operation required custom SRAM and circuit design, as conventional memory structures become increasingly unreliable when voltage is pushed far below standard operating ranges.

**5. Several custom mechanisms reduce unnecessary register-file activity**
**

Fujitsu described bypassing large vector register files when operands are already available in temporary buffers, supplying common values such as zeros or ones from smaller structures and avoiding upper-lane writes when the data width is below 256 bits. These techniques target the energy consumed by repeatedly reading and writing wide vector registers. The CPU also supports full-width unaligned loads and mechanisms that combine consecutive memory elements, helping sustain the throughput required to keep the 256-bit SVE2 pipelines utilized.

**6. The memory hierarchy can be configured around different NUMA trade-offs**
**

Monaka supports multiple NUMA organizations, ranging from more, smaller memory-locality domains to a single large domain. Each region can have dedicated last-level cache and DDR channels, while finer cache partitions localize traffic further. Fujitsu also supports memory-bandwidth and cache-quality-of-service controls for many independent partitions. The intention is to let cloud, HPC and large-memory workloads choose different balances between memory latency, aggregate bandwidth and ease of software placement.

**7. Fujitsu offers separate air- and liquid-cooled operating points**
**

The air-cooled configuration targets a 350W TDP and approximately 2.1 GHz base frequency, while a liquid-cooled version reaches about 500W and 2.9 GHz. Fujitsu said lightly threaded workloads can boost above those base frequencies when power headroom is available. The two SKUs are intended to span conventional data-center environments and higher-performance HPC installations without requiring a completely different architecture.

**8. Monaka is part of a longer Fujitsu/NVDA roadmap**
**

Fujitsu said production deployment is planned for 2027. A follow-on processor targeted around 2029 has already been selected for Japan’s FugakuNEXT supercomputer project. Fujitsu also referenced a partnership with NVDA and a future tightly connected CPU-GPU architecture. Alongside performance per watt, the company emphasized Arm software compatibility, confidential computing and sovereign control over data and infrastructure as central Monaka design goals.

## **INTC Wildcat Lake: bringing Core Ultra architecture into entry PCs**

**Speaker: INTC Core Series 3 lead SoC architect**

Summary: Wildcat Lake, sold as INTC Core Series 3, brings newer Core Ultra CPU, GPU, NPU and connectivity IP into the entry-PC segment. INTC focused on right-sizing the silicon and using an organic UCIe-based package to reach lower system cost.

**1. INTC chose a new architecture rather than recycling an older entry product**
**

Instead of refreshing another Intel 7-based Raptor Lake design or moving an older premium chip downmarket, INTC brought much of the newer Core Ultra architecture directly into entry PCs. That delivered newer CPU cores, Xe3 graphics, an NPU, Thunderbolt, Wi-Fi 7 and Bluetooth 6. INTC cited around 2.7x higher AI performance, 2.1x higher creativity and productivity performance and up to 64% lower processor power versus the prior entry platform.

**2. The product was aggressively right-sized for the entry segment**
**

INTC cut the graphics configuration in half, reduced the NPU, lowered the P-core count and halved the memory subsystem. It removed the integrated camera pipeline, trimmed media functionality, reduced concurrent display support and limited maximum display output to the levels entry-PC customers actually requested. Those changes reduced the compute-die area by more than 30%. On the I/O side, INTC removed or reduced selected Thunderbolt, PCIe, display and audio resources, shrinking that die by roughly another 15%.

**3. INTC removed ray tracing but restored matrix hardware when AI demand accelerated**
**

Ray-tracing hardware was removed because Wildcat Lake is not aimed at gaming systems. INTC initially considered removing the GPU systolic arrays as well to save additional area and power, but reversed that decision as local AI became more important. Keeping the matrix hardware brought total SoC AI performance back to roughly 40 TOPS. INTC also retained its latest high-performance P-cores because a small number of fast cores are useful for orchestration and tool use around local AI workloads.

**4. UCIe made a lower-cost organic two-die package possible**
**

Premium Core Ultra products use INTC Foveros packaging with a very fine-pitch base die. For the narrower entry configuration, INTC decided a monolithic chip or organic two-die MCP would be more affordable. UCIe enabled the MCP approach. The organic package uses a much wider pitch than Foveros and required controllers, buffers and packetization, increasing the die-to-die interface area by roughly 70%. INTC said the larger interface was still far cheaper than retaining the base die, extra assembly steps and associated yield costs.

**5. The UCIe links were deliberately limited to reduce validation and power risk**
**

INTC used 16-wide links running at 8 GT/s and right-sized display and mainband bandwidth around entry-system requirements. Hundreds of sideband signals were packetized onto narrow links, but INTC included spare direct pins in case a signal proved too timing-sensitive after silicon arrived. The 8 GT/s rate kept bit-error risk low enough to avoid forward-error correction and retry. INTC described cross-die reset, power management and sideband timing as one of the largest validation efforts in the entire program.

**6. Buffering display traffic allows the UCIe link to enter much lower-power states**
**

Connected-standby systems may continually send small amounts of display data even when little else is active. Rather than leaving the die-to-die link awake for a trickle of traffic, Wildcat Lake buffers display updates and transmits them in bursts. That allows the UCIe interface to spend more time in a link state with roughly 8x lower power. INTC said this capability was close to a requirement for meeting the battery-life target of the entry platform.

**7. Platform-level savings mattered as much as processor cost**
**

Wildcat Lake moves from a 50-by-25 mm package to approximately 35-by-25 mm and uses a unique lower-cost platform. A narrower but faster DRAM arrangement reduced OEM bill of materials, while integrated Wi-Fi and Bluetooth avoided discrete components. INTC retained dedicated low-power rails for efficiency cores and graphics even though they added some platform cost, because those rails preserved much of the battery-life advantage inherited from Core Ultra. The product is primarily targeted around a 15W operating envelope.

**8. INTC says the product has more than 70 design wins**
**

The company said Core Series 3 has secured more than 70 system designs and highlighted that the organic MCP trade-off remained clearly favorable after silicon and manufacturing costs were fully included. INTC also emphasized the software cost of changing hardware configurations: removing or modifying features can create additional driver, validation and application work even when the silicon decision is correct. The product was designed for cost recovery through multiple frequency and partially enabled configurations without creating unwanted numbers of OEM SKUs.

## **BOS Semiconductors’ scalable automotive AI accelerator**

**Speaker: K. Ming, BOS Semiconductors**

Summary: BOS presented Eagle-N, a Samsung Foundry 5nm automotive AI accelerator that scales from roughly 190 to 500 TOPS and can reach around 1,000 TOPS through chiplet integration. The design emphasizes UCIe, virtualization, memory bandwidth and automotive functional safety.

**1. Eagle-N scales from 190 to 500 TOPS, with a path to roughly 1,000 TOPS**
**

BOS uses product harvesting and configuration options to offer several performance points from one physical design. Multiple Eagle-N chiplets can be connected through UCIe to double total performance toward approximately 1,000 TOPS. The accelerator is intended for ADAS, in-vehicle infotainment and other on-device AI workloads, either as the primary accelerator or as an add-on that allows an existing automotive SoC to run newer models. BOS said the chip supports integer, floating-point and MX-style low-precision formats.

**2. The design combines Samsung 5nm, UCIe and 12-channel LPDDR5X**
**

Eagle-N is fabricated on Samsung Foundry’s 5nm automotive process. It includes UCIe connectivity, PCIe 5.0 and an ARM host CPU alongside the custom NPU. The memory system uses 12 LPDDR5X channels for approximately 204 GB/s of bandwidth, with support for lower channel-count configurations. BOS emphasized that many AI models are limited more by memory bandwidth than by peak arithmetic, making the memory controller and on-chip interconnect major determinants of realized performance.

**3. A wide bidirectional mesh connects compute, dispatch and data movement**
**

The NPU uses multiple compute clusters connected through a 256-byte bidirectional on-chip mesh. Dedicated dispatch engines prefetch data from DRAM or the host and queue outputs, hiding external-memory latency from the tensor cores. Data-movement engines operate independently from matrix and special-function compute so transfers can overlap with execution. Each cluster also carries shared SRAM for weights, activations and intermediate results, reducing the amount of traffic that has to leave the NPU.

**4. Hardware virtualization creates independent accelerator partitions**
**

Eagle-N can divide the NPU into physically separated partitions with independent compute clusters, dispatch engines, data-movement resources, clocks and resets. Each partition also receives its own memory-protection and address-translation structures. BOS said this allows multiple inference workloads or operating environments to run simultaneously without interfering with one another, an important requirement when one automotive SoC supports ADAS, infotainment and other workloads with different safety and availability requirements.

**5. BOS is designing around an open chiplet operating model**
**

BOS discussed the Open Chiplet Architecture, a layered standard covering physical links, transport, system services and software. UCIe provides the high-bandwidth data path, while additional mechanisms cover boot, discovery, address translation, interrupts, time synchronization, trace, debug and safety messaging. A primary chiplet can initialize secondary chiplets and manage shared system functions. BOS’s objective is to connect multiple Eagle-N devices or pair Eagle-N with a different ADAS chiplet without creating a one-off proprietary integration each time.

**6. The safety island is isolated from the main AI domain**
**

BOS described a main compute domain designed to automotive safety requirements and a more highly protected safety island with dedicated power, clock, reset, memory and internal buses. The safety island monitors process conditions, clocks, power and temperature and can remain operational if the main domain fails. The fault-tolerant time interval is set at 32 milliseconds, aligned with a typical video frame period. The same control structure can coordinate self-test and safety reporting across multiple chiplets.

**7. Address placement and DRAM interleaving are optimized at the architecture level**
**

Because the 12 LPDDR channels are physically distributed around the package, BOS separates traffic between the left and right sides early in the address path and then interleaves accesses across the individual channels. A hash-based mapping spreads traffic to avoid hot spots. BOS said memory placement had to be considered during the initial architecture rather than after the NPU was completed, reflecting how strongly modern inference performance depends on package-level memory topology.

**8. Customer readiness drove BOS toward ARM rather than RISC-V**
**

Asked why Eagle-N uses an ARM CPU, BOS said many automotive customers still view the RISC-V ecosystem as insufficiently mature for their current deployment needs. Customer expectations and existing software therefore favored ARM despite the additional licensing cost. BOS is working with external software partners on the higher-level AI stack while developing lower-level firmware, drivers and hardware-specific optimization internally. Early silicon is already being used for software development and system validation.