Hot Chips 讨论强化AI内存紧缺与HBM架构演进，但未形成单一可交易新增催化

Some great stuff so far at Hot Chips…Takeaways from the key talks below…

## **Feeding AI’s voracious hunger for data**

**Speaker: Jim Handy, Objective Analysis**

Summary: Handy framed the memory cycle around hyperscaler AI spending, HBM’s heavy wafer requirements and limited spare DRAM capacity. He also covered NAND/HDD spillover, pricing, efficiency gains and processing closer to memory.

**1. Hyperscaler capex remains the core demand driver**
**

Handy said the biggest force in memory is the sharp acceleration in hyperscaler AI capex. Spending had historically grown steadily before breaking higher as the major platforms began competing to build AI infrastructure. That flows directly into memory because nearly every major AI accelerator, including NVDA GPUs, GOOGL TPUs and custom silicon, uses HBM. Even some networking components now use HBM, creating a demand category that barely existed several years ago.

**2. HBM is creating a much broader DRAM supply problem**
**

HBM produces roughly one-third as many gigabytes per wafer as standard DDR because of its larger die size, TSVs and supporting circuitry. That means rapidly rising HBM demand consumes enormous wafer capacity. DRAM suppliers had also gone more than a decade without major capacity additions because process shrinks were enough to satisfy historical bit growth. New fabs can take around two years even under aggressive timelines, leaving the industry unable to respond quickly to the current demand shock.

**3. Tight supply is flowing directly into pricing and memory-vendor revenue**
**

DRAM and NAND are commodities, so shortages let suppliers shift capacity toward customers willing to pay more. Handy noted that spot prices have risen roughly 7x from prior levels, though spot is an imperfect industry indicator. Both DRAM and NAND price per gigabyte have increased sharply, driving unusually strong revenue growth at Samsung, SK hynix, MU and major NAND suppliers.

**4. AI infrastructure is also tightening NAND, SSD and HDD markets**
**

META tested placing lower-cost QLC SSDs between TLC SSDs and hard drives and found it improved performance, prompting broader hyperscaler SSD adoption. At the same time, AI infrastructure increased HDD demand enough to create shortages. QLC SSDs then began filling both this new storage tier and, in some cases, replacing unavailable hard drives at much higher cost. That helped push NAND into shortage as well, benefiting Kioxia and SNDK.

**5. Better algorithms do not necessarily mean lower hardware spending**
**

Handy pushed back on the idea that every model-efficiency breakthrough reduces memory or infrastructure demand. When a new technique cuts memory requirements materially, hyperscalers can simply use the same infrastructure budget to process more tokens. If an algorithm improves efficiency by 6x, the result may be 6x more output rather than lower capex. Efficiency changes the productivity of AI infrastructure without necessarily reducing the dollars spent on it.

**6. The HBM base die is becoming an increasingly attractive place for processing**
**

HBM already includes a logic base die beneath the DRAM stack, making it a natural place to add functionality without creating an entirely new memory architecture. Handy expects increasingly sophisticated uses of this logic, particularly where processing closer to memory can reduce data movement. Fully custom memory remains harder economically because customers expect memory to stay cheap, while using the base die avoids the huge development cost of a fully custom DRAM design.

**7. Software remains the largest barrier for alternative memory architectures**
**

The software ecosystem for processing-in-memory and other alternative architectures remains weak. NVDA’s advantage came partly from investing early and aggressively in a complete software stack around its hardware. New memory-centric architectures either need to fit into existing programming environments or build something similarly usable. Without that, even technically strong products can struggle to reach enough volume to drive costs down. Handy cited INTC’s Optane as an example of interesting technology that never achieved sufficient scale.

**8. The memory wall is increasingly about cost as well as bandwidth**
**

The traditional memory wall is the performance penalty from constantly moving data between memory and compute. Handy believes a second wall has emerged because memory itself has become extremely expensive. Processing nearer to the data helps both problems by reducing traffic across the interface. Rather than moving an entire dataset back to the processor, smarter memory could perform an operation locally and return only the required result.

## **Evolving memory architectures for AI**

**Speaker: Ragu, Micron HBM Design Architecture**

Summary: MU focused on why HBM’s bandwidth advantage increasingly comes with penalties in silicon consumption, thermals, packaging complexity and reliability. It also covered stack-height limits, workload-specific HBM and future D2D interfaces.

**1. HBM’s capacity displacement versus DDR is getting worse**
**

For the same memory capacity, an HBM die is much larger than DDR because more silicon is devoted to parallel data paths, TSVs and power delivery. MU has previously quantified HBM3E at roughly 3x the silicon of DDR5 for the same bits. Ragu said that ratio is not improving and is likely getting worse as bandwidth increases. More parallelism means larger dies and more supporting circuitry, while taller stacks create additional yield challenges.

**2. Memory can represent roughly 90% of the silicon in a high-end AI package**
**

In a representative GPU package with four 12-high HBM stacks, total memory silicon can be roughly 8x the amount of GPU silicon. That means around 90% of the silicon in the system-in-package is memory-related. The reliability implications are substantial because one failing DRAM die can affect the entire package. Ragu described HBM as one of MU’s most cross-functionally complex products, spanning process, packaging, design, test and system architecture.

**3. Thermals are now a first-order architectural constraint**
**

The HBM base die is typically one of the hottest areas because it contains the highest-speed D2D links and increasingly complex logic. Heat is generated at the bottom while cooling sits above the DRAM stack, forcing heat through every layer. Each additional DRAM die adds thermal resistance, while more base-die functionality adds heat. MU said HBM is increasingly being architected around thermal limits from the start, with localized power density and hotspots becoming as important as total power.

**4. HBM has already been a meaningful source of training interruptions**
**

Ragu cited META’s published Llama 3 work, where roughly 17% of unintended training interruptions were attributed to HBM. Reliability becomes harder as the amount of memory silicon and number of dies per package rise. HBM3/HBM3E added metadata for CRC and system-level ECC, alongside on-die symbol-based ECC and Reed-Solomon protection. MU expects RAS mechanisms to keep advancing as stacks get taller and interfaces get faster.

**5. Sixteen-high looks achievable, but the path beyond that is less clear**
**

MU sees a good technical path to 16-high HBM. Beyond that, die thinning, mechanical stress and thermal resistance become much harder. JEDEC has discussed 20-high stacks, but Ragu said substantial work remains after 16. Signaling distance itself is not the primary issue because the entire stack remains below roughly one millimeter; the tougher constraints are TSV density, power density, thermals, mechanical integrity and manufacturing yield.

**6. HBM’s bandwidth scaling comes from extreme parallelism**
**

HBM uses similar DRAM cell technology to conventional memory but accesses far more banks in parallel. HBM3E had 128 banks per DRAM die, while HBM4 moves to 256. The external interface also doubles from roughly 1,000 I/O lines to 2,000 within about the same shoreline. That has helped HBM scale from 128 GB/s and 1 GB in HBM1 to more than 2.8 TB/s and over 24 GB per cube in HBM4, at the cost of much higher silicon intensity.

**7. Future HBM may become more workload specific**
**

HBM and GPUs historically served as relatively broad, one-size-fits-all products. MU now sees growing interest in customized memory architectures for specific AI workloads. Different applications may prefer different bank structures, channel counts, access granularities or bandwidth/capacity balances. Ragu expects the processor and memory paired with it to become increasingly co-optimized as AI workloads segment.

**8. High-speed interfaces and packaging are becoming part of the core memory architecture**
**

MU expects continued innovation in memory-specific D2D interfaces, interposers and eventually optics. HBM bandwidth only matters if data can move efficiently into the processor. Packaging is also shifting from micro-bumps and thermal-compression bonding toward fusion and hybrid bonding at single-digit-micron pitches, which can increase data density while lowering thermal resistance.

## **Samsung on the evolution of the HBM base die**

**Speaker: Sanghan, Samsung Memory Business**

Summary: Samsung laid out a three-stage roadmap that turns the HBM base die from a communications layer into a customized AI platform, eventually culminating in vertically stacking HBM directly on the XPU through ZHBM.

**1. ZHBM could materially change the power and bandwidth equation**
**

Samsung’s long-term ZHBM architecture eliminates the traditional 2.5D interposer and vertically integrates HBM directly on top of the XPU. Removing the lateral PHY/D2D path lets I/O and TSV structures spread across the footprint rather than clustering at the edge. Samsung is targeting roughly 0.5 pJ/bit, ~70% lower total DRAM power versus HBM5 and >2.3x higher DRAM bandwidth. Absolute DRAM power could fall by >100W, freeing that power for GPU compute. Thermal limits currently point to roughly four-high stacks rather than 12-high or 16-high.

**2. Moving the memory controller into HBM could reclaim 5–10% of XPU area**
**

Samsung sees the memory controller as one of the clearest functions to move from the XPU into the custom-HBM base die. The controller can consume roughly 5–10% of XPU silicon area, and Samsung estimates reclaiming that space for compute could potentially improve performance by 10–20%. It said most customers are actively exploring this architecture. The embedded controller can also use SRAM as a fine-grained repair resource for failing DRAM cells.

**3. Custom HBM could become a gateway to a second memory tier**
**

Samsung believes capacity is becoming nearly as important as bandwidth as context windows and KV caches expand. Its Phase 2 roadmap uses unused shoreline around the custom-HBM base die to connect directly to a second memory tier through a dedicated controller and PHY. Samsung expects better latency and bandwidth than PCIe-based expansion and said this second tier could be LPDDR or even HBF.

**4. Attention is the first major compute block Samsung wants to move into HBM**
**

Samsung does not think it makes sense to place large amounts of dense compute inside the base die because of thermal limits. Instead, it wants to selectively move memory-bound work closer to memory. For LLM inference, attention is an attractive function to offload into advanced HBM, while compute-heavy prefill and FFN remain on the XPU. The goal is to cut interposer traffic, D2D bandwidth and data-movement power without turning HBM into a full accelerator.

**5. Proprietary HBM interfaces may be favored over UCIe**
**

Replacing the traditional HBM PHY with a smaller specialized D2D interface can reduce footprint, shorten the channel and improve energy efficiency. The downside is much higher power density in a smaller area, which Samsung said was a major challenge in custom HBM4. Its Heat Path Block can reduce peak temperature by >35% when covering enough of the hot interface region. Samsung has evaluated UCIe but currently sees proprietary HBM-vendor interfaces as more attractive because UCIe is larger and consumes more power.

**6. Advanced-node base-die logic is becoming mandatory**
**

HBM energy efficiency improves every generation, but total power still rises because bandwidth scales faster. With HBM4 doubling I/O count and increasing speed, Samsung said a conventional DRAM process could leave the base die consuming more power than the DRAM core stack itself. Moving the base die to advanced logic is therefore becoming essential, while also freeing area for controllers, telemetry, test logic and other SoC-like functionality.

**7. Custom HBM allows richer telemetry, testing and repair**
**

Samsung plans to use additional base-die logic for voltage, process and aging telemetry and has integrated on-die automatic test functionality that can generate its own test patterns. A custom memory controller can therefore test DRAM more extensively from wafer level through the completed system-in-package and use SRAM resources to repair some defects discovered later in manufacturing.

**8. ZHBM requires packaging and DRAM/SoC design to converge**
**

ZHBM requires wafer-level integration and hybrid copper bonding at sub-six-micron pitches. Samsung also believes the historical separation between DRAM and SoC design has to disappear. Once the HBM base die contains custom logic, controllers and potentially compute, the memory and XPU need to be co-designed around routing, thermals, power density and physical interconnect from the start.

## **Advanced packaging for HBM**

**Speaker: Jasik Lee, SK hynix USA**

Summary: SK hynix focused on the packaging needed to scale HBM toward 16-high and eventually 20-high stacks, with die warpage, inter-die spacing, hotspots and customer-package integration becoming increasingly important constraints.

**1. Hybrid bonding becomes increasingly attractive around 20-high**
**

SK hynix expects HBM eventually to move from micro-bump stacking to hybrid bonding, although Lee does not expect it for HBM4E. Removing the gaps between DRAM dies allows the silicon itself to become thicker within the same package-height limit. In a hypothetical 20-high stack, SK hynix estimates die thickness could increase roughly 20–24%, improving manufacturing margin and thermals. It also sees around a 35% improvement in thermal conductivity at 20-high.

**2. Sixteen-high already requires substantially thinner dies and smaller gaps**
**

In its 16-high HBM3E test vehicle, SK hynix still had to make individual DRAM dies roughly 10% thinner than 12-high and reduce inter-die gaps by about 50%, even after increasing total cube thickness from 720 to 775 microns. That makes die warpage and gap filling much harder. Higher pin speeds also increase power density around the PHY, adding localized hotspots to the broader stack-level thermal problem.

**3. SK hynix’s molded process offers a productivity advantage**
**

SK hynix’s process places the full DRAM stack, joins it during a single reflow and then fills the gaps with molding material. Compared with bonding each die individually, this improves productivity and allows materials with relatively good thermal conductivity between dies. The downside is greater sensitivity to warpage because individual dies can be around 50 microns thick or thinner, while ever-smaller gaps become harder to fill reliably.

**4. Hybrid bonding does not solve localized base-die hotspots**
**

Even if hybrid bonding improves overall thermal conductivity, the base die still contains small regions with extremely high power density. SK hynix is developing IHBM, which adds silicon heat blocks above hotspot regions to improve local heat extraction. Lee said this requires co-optimization from the beginning of the design and cannot simply be added late in a product cycle.

**5. HBM suppliers increasingly need to co-design with customer packaging**
**

Advanced HBM experiences much more of the customer’s subsequent packaging process than traditional memory. Hybrid-bonded HBM can be among the first components attached to an interposer, exposing it to the thermal and mechanical stress of later integration. Customers are also evaluating more interposers, bridges and alternative packaging technologies, requiring deeper collaboration around process flows, materials and mechanical stress.

**6. Training and inference may ultimately use different memory architectures**
**

SK hynix does not assume every AI workload should simply use taller HBM. Training often needs both more bandwidth and more capacity, supporting continued stack-height increases. Inference may use a smaller pool of very-high-bandwidth memory and place less bandwidth-sensitive data in LPDDR or another lower-cost tier. Lee said the industry still needs to determine the optimal architecture rather than assuming training and inference converge on the same solution.

**7. HBM4 is approaching practical limits on total cube thickness**
**

Through HBM3E, standard HBM thickness was roughly 720 microns. HBM4 moves to around 775 microns, which SK hynix described as close to the practical limit under current packaging approaches. Future scaling therefore cannot rely indefinitely on making the package taller and will increasingly depend on thinner dies, smaller gaps and eventually hybrid bonding.

**8. Interposer routing could become another scaling constraint**
**

Higher signal speeds require tighter bump pitches, more shielding and potentially more metal layers in the interposer or bridge. Those layers are much thicker than in conventional logic, so adding more can materially change wafer thickness and mechanical behavior. SK hynix said it is not yet clear how far the industry can push routing-layer counts before warpage and manufacturability become limiting.

## **Raptor: a 3D-DRAM accelerator for generative inference**

**Speakers: Ayush Ankit, META and formerly d-Matrix; Sudip Hoja, d-Matrix**

Summary: d-Matrix argued that conventional HBM faces fundamental power and package-edge limits as inference bandwidth requirements rise. Its Raptor architecture stacks custom DRAM directly beneath compute and reports 100-TB/s measured bandwidth at much lower data-movement energy.

**1. HBM faces both a bandwidth wall and a power wall**
**

d-Matrix used Kimi K3 as an example of a model approaching three trillion parameters, requiring roughly three terabytes for weights, while long-context inference can add another terabyte of KV cache. HBM offers far more capacity than SRAM but becomes difficult to scale toward SRAM-like bandwidth around 100 TB/s. Pin speed, I/O width and the number of HBM stacks improve relatively slowly, while package edge limits how many interfaces can attach to compute. At current HBM energy per bit, d-Matrix estimates 100 TB/s would require roughly two kilowatts just for memory movement.

**2. Raptor reports 100 TB/s at a measured 0.37 pJ/bit**
**

Raptor uses a TSMC 4nm compute die stacked face-to-face with custom DRAM at roughly 36-micron pitch, with DRAM banks physically aligned to compute engines. d-Matrix compared this with HBM4 at roughly 2.4 pJ/bit just to move data into the base die. Its measured result is 0.37 pJ/bit, or roughly 6–7x better energy efficiency, enabling 100 TB/s in a single-high stack. Even then, full-bandwidth I/O still consumes around 300W.

**3. Decode-heavy and agentic inference are the primary targets**
**

d-Matrix separates inference into compute-heavy prefill and bandwidth-heavy decode. Prefill processes many tokens together and keeps compute highly utilized, while decode repeatedly streams weights and KV-cache data for one or a few tokens. Agentic workloads combine both phases repeatedly. Across the workloads analyzed, a meaningful portion of wall-clock time was spent in decode attention and decode MLP, making those phases particularly suited to very-high-bandwidth 3D DRAM.

**4. A rack-scale deployment can hold a roughly three-trillion-parameter model**
**

The major trade-off versus HBM is lower memory capacity per device. d-Matrix argues rack-scale aggregation can compensate. A representative configuration of around 72 cards with single-high 3D DRAM can hold a Kimi K3-class model while leaving capacity for KV cache and batching. Larger models could extend across multiple racks or use disaggregated memory.

**5. Thermal behavior is manageable at one-high but difficult for multi-high stacks**
**

Raptor puts the logic die on top so it can sit directly beneath a cold plate. The DRAM is designed to operate around 105°C and refreshes roughly 8x more frequently than standard DRAM. The current design is only one-high. d-Matrix’s simulations suggest four DRAM layers on top of a GPU dissipating more than roughly 1.3 W/mm² could push DRAM temperatures above 140°C. Logic-on-top is better for cooling; logic-on-bottom is better for power delivery.

**6. Faster refresh can be handled with relatively little bandwidth loss**
**

d-Matrix uses very small microbanks, allowing 8x faster refresh to consume only around 1.37% of memory bandwidth. Reliability is handled with Reed-Solomon coding plus CRC, while yield is improved through roughly 8–9% spare banks that can replace failing banks without sacrificing the memory channel.

**7. Density is around half HBM4 today, but much of the gap may be process related**
**

Raptor’s 3D DRAM currently delivers around half HBM4’s capacity per square millimeter. Some of the penalty is structural, but d-Matrix says a meaningful portion comes from using a less advanced DRAM process. A leading DRAM node could narrow the density gap substantially. The company reported roughly 3,000 tokens/sec/user on GLM 5.2 and 988 on the larger model using its single-high implementation.

## **High-bandwidth flash in AI compute**

**Speakers: Anurag Agarwal, Oxmiq Labs; Radha Krishna Giduturi, Praxmudi**

Summary: The presentation evaluated HBF as a high-capacity AI inference tier. Simulations show it can work well in specific low-bandwidth or capacity-constrained situations, but HBM remains much more economical for high-throughput rack-scale inference.

**1. At rack scale, HBM is roughly 6–8x better on cost per token**
**

The speakers modeled a 72-accelerator rack using comparable-cost HBM, HBF and mixed configurations. All-HBF provides roughly 14x more capacity but only about 0.6x the bandwidth. On dollars per million decode tokens at interactive response rates, HBM wins across essentially the entire rack-scale curve, with roughly 6–8x better economics. At the single-box level, however, one HBF accelerator may hold a trillion-parameter model that could require multiple HBM GPUs simply for capacity.

**2. HBF is fundamentally a capacity tier rather than a bandwidth tier**
**

HBF has been positioned as offering roughly 8–16x more capacity at the same price as HBM. The trade-off is much lower bandwidth per gigabyte: grade-two HBF provides around 25x less. That makes simple $/GB comparisons misleading for inference. What matters is tokens per second at the desired interactivity, and cheap capacity can still become expensive per token when bandwidth is the bottleneck.

**3. HBF wins within a relatively narrow workload zone**
**

For a trillion-parameter model with short context, one HBF GPU can hold and serve the workload efficiently. As context expands toward one million tokens, the footprint can rise from roughly 600 GB to around 2.6 TB. HBF may still have enough capacity, but often not enough bandwidth for desired batch sizes and interactivity. In the modeled scenarios, HBF works best at lower batch sizes and other low-bandwidth-demand points.

**4. Mixed HBM/HBF caching becomes ineffective as more experts activate**
**

One approach keeps hot MoE experts in HBM and colder experts in HBF. The problem is that each new query can activate different experts, and at modest batch sizes the system can quickly read more than half the model. At that point the HBM cache loses much of its value. More sophisticated scheduling could improve reuse by grouping similar queries, but that requires explicit workload-aware orchestration.

**5. Software integration may be a larger obstacle than the hardware**
**

HBF is not a drop-in DRAM replacement. Peak reads may require at least 64 KB accesses across channels, while writes can occur at roughly 1 MB granularity. A mixed HBM/HBF system therefore needs explicit movement between tiers and potentially separate memory-management logic. Software also has to manage retention and lifecycle functions that DRAM handles transparently. Existing inference frameworks such as vLLM were built around DRAM-like memory and would require new allocators and transfer scheduling.

**6. KV cache and MoE expert offload are the simplest initial use cases**
**

The most practical model is to use HBF as a backing tier rather than compute directly from it. MoE expert weights or KV-cache data can reside in HBF and move asynchronously into HBM through DMA as needed. Existing compute kernels can stay largely unchanged because execution still happens from HBM, while the inference framework manages residency and prefetching.

**7. Sparse attention may be HBF’s strongest architectural use case**
**

Sparse-attention models maintain large KV caches but read only a small top-K subset for each token, creating the high-capacity/low-bandwidth pattern HBF is designed for. The complication is that top-K generates scattered reads while flash prefers large sequential transfers, so software or data layout may need to change. HBF’s excess capacity could also allow more MoE experts to be replicated locally, reducing network traffic in some lower-batch workloads.

**8. Endurance, maintenance and power remain open questions**
**

The HBF spec does not yet define a firm write-endurance target. Decode generates relatively few writes per token and software can aggregate them, but endurance still needs active management. Retention maintenance may also require periods where the device is unavailable. Energy efficiency is not formally specified; the presenters said industry indications suggest HBF may be somewhat worse than HBM, though not dramatically. Their performance results were analytical simulations rather than measured silicon.

## **RISC-V standards, profiles and the path to server-class silicon**

**Speaker: Krste Asanović, RISC-V International and SiFive**

Summary: Asanović explained how RISC-V is using profiles and platform standards to move from a highly configurable ISA into a binary-compatible application and server ecosystem. He also covered AI matrix extensions, security and higher-performance silicon.

**1. RVA23 is intended to create a genuinely compatible application ecosystem**
**

RISC-V supports everything from tiny embedded cores to servers, which creates many possible extension combinations. RVA23 narrows that flexibility at the application-processor level by defining a rich mandatory feature set. Software compiled for RVA23 should run across compliant CPUs from different vendors without separate builds. Asanović said multiple vendors are bringing RVA23 server-class silicon to market this year.

**2. Profiles solve the hardware/software chicken-and-egg problem**
**

Software vendors typically target features already present across a broad installed base, while hardware vendors do not want to spend silicon area on features software will not use. Profiles create an agreed roadmap: hardware vendors commit to required capabilities and software can assume they exist. RVA23 makes major features such as vectors and virtualization mandatory, while crypto and other functions are organized through curated options.

**3. Server standards now extend well beyond the CPU ISA**
**

The broader RISC-V server-platform specification covers the boot environment, UEFI, ACPI, I/O MMU, interrupts and other system components expected by modern operating systems. The goal is straightforward: download a standard Linux distribution and boot it on a compliant RISC-V server without vendor-specific workarounds.

**4. RISC-V is developing several approaches to AI matrix acceleration**
**

Four matrix approaches are under development because implementations have different requirements. The simplest uses batch dot products on existing vector registers, while others reinterpret vector registers as matrix tiles or add dedicated accumulator/matrix state. RISC-V can also combine scalar, vector and matrix execution within the same core. Asanović expects most of the matrix-extension work to reach ratification over roughly the next 12–18 months.

**5. RISC-V supports both standardized and highly customized processors**
**

Vendors can use a minimal base ISA for embedded systems or a rich profile such as RVA23 for general-purpose processors. RISC-V also reserves encoding space for vendor-specific extensions without interfering with standard software. That lets accelerator and SoC designers add specialized operations while still using the common compiler, operating-system and software ecosystem.

**6. Security standards are being built around multiple isolated domains**
**

RISC-V’s security work includes physical memory protection, supervisor-level protection, bus-level isolation through RISC-V Worlds and a confidential-computing model using supervisor domains. Asanović compared RISC-V Worlds conceptually with TrustZone, but with support for multiple mutually distrustful domains rather than only secure and non-secure worlds.

**7. High-performance RISC-V is increasingly an ecosystem-investment question**
**

Asanović argued there is no fundamental ISA barrier preventing RISC-V from reaching high performance. Historically, the constraint has been design investment, leading-edge process access and demand for high-end implementations. As the ecosystem matures and RVA23 servers reach market, a larger installed base can justify more software and hardware investment.

## **Ubuntu and the RISC-V software ecosystem**

**Speaker: Gordon Markus, Canonical**

Summary: Canonical described progress toward making RISC-V a first-class Ubuntu architecture. Ubuntu 26.04 LTS already uses RVA23 as its baseline, roughly 95% of the normal package archive is available, and Canonical is expanding native build capacity and infrastructure software.

**1. Ubuntu 26.04 LTS has already moved to RVA23**
**

Canonical first tested RVA23 in Ubuntu 25.10, rebuilding roughly 30,000 open-source packages against the new profile. After that succeeded, Ubuntu 26.04 LTS adopted RVA23 as its official RISC-V baseline. Canonical moved ahead of broad hardware availability because a stable profile lets software and semiconductor roadmaps intersect before volume silicon arrives.

**2. Roughly 95% of the normal Linux software archive is already available**
**

The Debian package archive available on RISC-V is now close to 95% of what users can access on the most widely deployed architectures. The remaining gap is not mainly a portability problem. Many packages have not completed equivalent automated testing because Canonical has been building and testing largely through emulation, causing some suites to run extremely slowly or time out.

**3. Native RISC-V build infrastructure is now a major priority**
**

Canonical is adding native RISC-V systems inside its own data centers to replace much of the emulated build and test environment. Comprehensive testing through emulation can take months, and many apparent failures are simply timeouts. Markus expects native builders to bring package testing and pass rates much closer to established architectures.

**4. Kubernetes and cloud infrastructure are the next major software layers**
**

Canonical plans to enable Kubernetes on RISC-V so server-class hardware can support standard cloud-native environments as it becomes broadly available. Similar work is planned for MicroCloud and eventually OpenStack. Canonical can already operate small RISC-V MicroCloud clusters in its labs.

**5. Canonical expects ISA baselines to keep evolving**
**

RVA23 is already more feature rich than the minimum baseline Canonical targets for several older architectures, but the company does not expect one baseline to remain static forever. Canonical already supports architecture variants elsewhere and could use the same approach for RISC-V as features such as matrix extensions mature and become widely adopted.

**6. Multi-architecture software practices have reduced the porting burden**
**

Many major open-source projects already support multiple architectures. RISE and other ecosystem groups are funding RISC-V build farms, GitHub runners and foundational software such as LLVM, Go, Python and Rust. Markus also said AI-assisted coding is making some porting work easier because much of it is mechanical rather than a fundamental software redesign.

**7. Predictable standards are the key requirement from hardware vendors**
**

Canonical’s main request from RISC-V is stability and roadmap visibility. Linux distributions have to maintain software and security support for long periods, so vendors need to know which hardware capabilities will exist and when. Markus said RVA23 and the server-platform spec have materially improved that predictability.

## **NVDA on bringing RISC-V into CUDA and NVLink Fusion**

**Speaker: Franz, NVDA**

Summary: NVDA described what third-party RISC-V application processors need to support to participate in CUDA and NVLink Fusion systems. NVDA now has multiple RVA23 processors in its labs and is publicly demonstrating CUDA on RISC-V hardware.

**1. NVDA now has multiple RVA23 processors in its labs**
**

A year ago, NVDA’s biggest practical problem bringing CUDA to RISC-V was the lack of suitable RVA23 hardware. That has changed quickly: NVDA now has at least three RVA23 processors in its labs, and SiFive is demonstrating NVDA’s CUDA implementation during Hot Chips. Franz described it as the first public demonstration of CUDA running on RISC-V.

**2. NVLink Fusion creates a path for custom RISC-V host CPUs**
**

NVLink Fusion lets customers use a custom CPU or accelerator while retaining much of NVDA’s rack-scale architecture. A RISC-V CPU could therefore replace NVDA’s ARM-based host processor in a specialized system. The key requirement is integrating NVDA’s C2C network, which provides up to roughly 5x PCIe bandwidth and coherence so CPUs and GPUs can access memory across the system.

**3. CUDA relies first on the standard RISC-V server profile and platform**
**

NVDA does not want a separate RISC-V specification for CUDA. The starting point is compliance with the RVA23 profile and RISC-V server-platform spec, which provide binary compatibility, vectors, virtualization, firmware interfaces, ACPI and security. ACPI was particularly important to NVDA because CUDA needs standardized hardware discovery, power management and thermal management.

**4. CUDA adds only a short list of extra hardware requirements**
**

NVDA’s CUDA-specific requirements are only around two pages. One major requirement is PCIe cache coherency so software does not need explicit CPU-cache flushes and invalidations around GPU DMA. NVDA also requires peer-to-peer PCIe communication so GPUs can exchange data directly instead of routing it through CPU memory.

**5. Agentic AI is making the host CPU more important**
**

NVDA’s Vera CPU is heavily customized for agentic AI because these workloads place more demands on CPU-side memory, storage and orchestration. The CPU has to feed GPUs at very high speed while handling communication, synchronization and non-accelerated portions of the application. A custom RISC-V CPU could be optimized differently for another workload while still connecting to NVDA GPUs and networking through NVLink Fusion.

**6. RISC-V’s main advantage is the ability to specialize**
**

Franz was explicit that RISC-V is not attractive because x86 or ARM are inadequate; NVDA itself uses ARM extensively. The advantage is that many vendors can implement and customize RISC-V for important niches that may not justify a specialized processor from the largest incumbent CPU suppliers.

**7. Ecosystem readiness is materially better than a year ago**
**

RISC-V has moved from standards work into usable application-processor silicon. NVDA now has hardware for the CUDA port, Canonical supports RVA23 and the server-platform spec has matured. Franz’s message was that RISC-V vendors should take advantage of existing ecosystems rather than trying to recreate an entire AI or server stack themselves.

## **RISC-V for automotive and zonal architectures**

**Speaker: Thomas Rucker, Infineon**

Summary: Infineon discussed how RISC-V could fit into the shift from domain-based automotive electronics toward zonal controllers and centralized vehicle computing. Adoption still depends heavily on deterministic real-time behavior, safety, qualified tools and migration paths.

**1. Zonal architectures consolidate functions that historically lived in separate ECUs**
**

Traditional vehicle architectures often assign individual functions or domains to separate electronic control units, creating large numbers of ECUs, communication overhead and expensive wiring. Zonal architectures instead consolidate functions based on physical location in the vehicle. OEMs are unlikely to move directly to one centralized car computer because existing software has been qualified over many years, so Infineon expects a multi-stage transition.

**2. The shift toward zones is primarily being driven by automakers**
**

Rucker emphasized that zonal architecture is not simply a semiconductor-industry push toward larger SoCs. OEMs are driving it because the old model of mapping individual vehicle functions onto separate boxes increases cost, wiring and integration complexity. Consolidation reduces hardware but forces one controller to host workloads with very different real-time, safety and security requirements.

**3. Zone controllers need heterogeneous compute rather than one performance profile**
**

A zone controller may simultaneously handle real-time vehicle control, gateway functions, power management, predictive maintenance, audio and small AI workloads such as keyword recognition. Some tasks require predictable worst-case execution times and very low interrupt latency, while others can optimize for average performance. Infineon therefore expects a range from simpler MCU-class devices to “super-integrators” combining real-time MCU functions with application-processor capabilities.

**4. RVA23 works for central computers, but automotive still needs an MCU profile**
**

Infineon believes RVA23 is a suitable foundation for central car computers. Zone controllers, endpoints and many domain controllers need a microcontroller-focused profile, RVM, which is not yet finalized. Likely requirements include privileged execution, physical memory protection, non-maskable interrupts, atomics, cache-management instructions and strong code density.

**5. The processor profile alone is not enough**
**

Automotive also needs a standardized subsystem around the CPU, including interrupt routing, debug and trace, mixed-criticality isolation and fault reporting. Higher-end versions may need real-time virtualization, but usually with only a small number of statically scheduled VMs because timing must remain deterministic and certifiable.

**6. Safety-qualified toolchains remain a major gating item**
**

RISC-V adoption will depend on qualified compilers, timing-analysis tools and development environments that support automotive safety certification. Worst-case execution-time analysis is especially important for real-time vehicle functions. Customers also want credible migration paths from existing architectures and confidence that established software can move without excessive rewriting.

**7. Automotive optimization extends well beyond the ISA**
**

Customers care about memory, connectivity, accelerators, temperature range, reliability, power and total system cost, not just CPU architecture. RISC-V can reduce dependence on proprietary CPU IP and increase competition, but vendors still need processors optimized specifically for automotive environments and a broad, long-lived product portfolio.

**8. Ethernet is likely to become more important inside vehicles**
**

Infineon expects Ethernet to play a growing role between zone controllers and endpoints, although legacy automotive buses will remain for some time. Rucker’s broader point was that RISC-V adoption will happen alongside a larger redesign of vehicle electronics rather than as a simple CPU swap inside otherwise unchanged ECUs.