XTHE

Samsung Electronics brought together global experts for a panel called In Tech, We Trust: Rethinking Security and Privacy in the AI Age ” during its tech forum at CES 2026. The event, held at the Wynn in Las Vegas, focused on how trust is becoming a key factor in how people use and accept AI as it becomes a bigger part of everyday life.  

Making Invisible Intelligence Feel Trustworthy 

As AI increasingly anticipates needs, curates routines, and operates autonomously across devices, panelists Allie K. Miller (CEO of Open Machine), Amy Webb (CEO of The Future Now Strategy Group), Zach Kass (Global AI Advisor at ZKAI Advisory and former Head of Go-to-Market at OpenAI), and Shin Baik (AI Platform Center Group Head at Samsung Electronics) emphasized that trust must be earned not through promises but via consistent, understandable behavior. Each expert contributed their view on how transparency and reliability are critical for earning user trust.  

During the session, Samsung explained its trust-by-design approach, stressing that AI should be predictable, transparent, and easy for users to control. Allie Miller said when it comes to AI, users are looking for openness and control. They want to be leaders in their own personalized experiences, to understand whether an AI model is running locally or in the cloud, to know that their data is secure, and to clearly see what is powered by AI and what is not. That level of visibility builds confidence in the provider’s head. There is a responsibility to show up for users by designing personalized experiences around the core components of trust, clarity, security, and accountability.  

Samsung also pointed out that on-device AI keeps personal data on the device whenever possible. Cloud-based AI is used only when more speed or scale is needed, so users get flexibility without sacrificing privacy.  

Security for an AI-Driven World 

The panel discussed how security needs to change as AI spreads across phones, TVs, and home appliances. Samsung presented its Knox security platform, which now protects billions of devices through chip-level security and the Knox Matrix, a system that enables devices to authenticate and protect one another.  

Trust in AI starts with security, thus proven, not promised. Shin Baik said that, for more than a decade, Samsung Knox has provided a deeply embedded security platform that protects sensitive data at every layer. But trust does go beyond a single device  it requires an ecosystem that protects itself. Samsung Knox devices continuously authenticate and monitor one another, so each device acts as a shield for the rest, creating a resilient, secure environment users can rely on.  

Across Industry, Look At The Future Of Trust. 

Shin Baik emphasized that trust grows when AI behaves predictably and securely across devices, arguing that users need visible signals of control rather than black-box systems. Samsung pointed to its alliances with industry leaders such as Google and Microsoft as a way to strengthen shared security research, interoperability, and ecosystem-wide protection, while Allie Miller pointed out the value of transparency for users, including clear visibility into where AI models run, how data is used, and explicit labels that show what is powered by AI and what is not. Meanwhile, Zach Kass added that while misinformation and misuse present real challenges, for every risk there is also a countermeasure, and technology itself will play a critical part in mitigating AI’s downsides.  

Amy Webb evaluated the relationship between trust and consumer purchasing habits. I don’t think they’re making decisions based solely on trust. She said people aren’t paying for trust. They don’t buy things because of trust. They buy things because of convenience. So if the AI piece of this hooks people in, it makes their lives easier and more convenient.  

As AI becomes part of daily life, the panel agreed on one point. The technologies people trust most will be those that prioritize security, transparency, and user choice from the start.

Source: Samsung Explores How Trust, Security and Privacy Shape the Future of AI at CES 2026 

Apple’s OpenELM 2.0 models work with the MLX framework to deliver fast, private, local retrieval-augmented generation (RAG) on Apple Silicon in optimized setups. The back-end processes over 150 tokens/sec by using layer-wise scaling and unified memory. This configuration delivers secure, high-speed AI inference directly on your device, eliminating the need for the cloud.  

Key Aspects Of The OpenELM And MLX Stack 

• High-speed local inference: The MLX framework is designed for Apple Silicon and efficiently runs models, processing over 150 tokens/sec in some cases for real-time responses.  

• Privacy-focused RAG: Build local RAG servers using the MLX framework to keep sensitive data on your MacBook and protect privacy.  

• OpenELM architecture: Layer-wise scaling improves the accuracy of OpenELM models by optimizing parameter distribution across transformer layers.  

• Optimized Deployment: MLX provides libraries like MLX-LM for fast model deployment with minimal code and supports fine-tuning for on-device machine learning.  

1. Index documents by splitting them into small text chunks.  

2. Generate embeddings: Use MLX to turn chunks into numerical representations that help models interpret meaning.  

3. Store in vector database: Store embeddings, numerical text representations, locally in a vector database, a system for storing and searching embeddings efficiently.  

4. Retrieve and generate: Locate the best context for your queries, then use Apple or Open ELM models to generate responses.  

Note: these results are based on ideal conditions and specific Apple Silicon hardware.  

On-device AI has long promised a local-first approach, but hardware limits and slow speeds have held it back. With the release of Apple’s Open ELM 2.0 MLX framework, things are changing as Apple pairs the open-source, efficient language model Open ELM with the MLX array framework. Apple now delivers private, high-speed local RAG (retrieval-augmented generation) at 150 tokens per second on everyday consumer hardware.  

For developers and privacy-focused businesses, this update is more than just faster for performance. It changes how sensitive data from medical records to proprietary code is handled and searched, keeping everything on the device and protected by private RAG.  

The Architecture of OpenELM 2.0: Layer-wise Scaling Reimagined 

The key innovation in OpenELM 2.0 is its new layer-wise scaling approach. Instead of using the same layer sizes throughout, it spreads parameters unevenly. Layers near the input and output have different sizes and headcounts, allowing the model to use its resources more efficiently.  

When used with the MLX framework, Apple’s open-source library for Apple Silicon, Open ELM 2.0, leverages the unified memory architecture. Here, the CPU and GPU share a single fast memory pool, eliminating the usual PCIe bottleneck found with separate GPUs. Thanks to 4-bit quantization with MLX-LM, the model fits easily in a base MacBook AS memory living room for the vector database needed for RAG.  

Enabling 150 Token/Sec Performance: Speculative Decoding And MLX Kernels. 

OpenELM 2.0 reaches 150 tokens/sec, which feels almost instant to users, by using two main updates: optimized Metal kernels for Group Query Attention (GQA) and RMSNorm. These kernels reduce how often the processor needs to access memory, which is usually the main slowdown for large language models (LLMs).  

For speculative generation, OpenELM 2.0 uses a draft-and-verify method: a model like the 270M version predicts tokens first, and then the larger 3B model checks them in parallel. This lets the system generate several tokens at once, reaching speeds of 150 tokens/sec.  

The speed is especially important for local RAG workflows. In these pipelines, the model needs to read and summarize the context it finds before answering. High throughput means that, even with extensive documentation, the wait for the first token is barely noticeable.  

The Private RAG Advantage: Security at the Edge 

The private int private 150 token/SEC local RAG is more than just a label. It’s a technical guarantee based on the local-first design. In typical RAG setups, you send data, such as health records or company spreadsheets, to the cloud for processing.  

With OpenELM 2.0 and MLX, the whole process stays on your device.  

• Local embedding: data is converted into vectors using MLX-optimized embedding models, such as Hugging Face’s Sentence Transformers, and the results are saved locally in an encrypted format as microservices.  

• Local inference: the open ELM 2.0 model searches the local store and creates responses using the device’s GPU. This reduces ongoing subscription costs and keeps the system running even when offline. For fields like healthcare or law, this is the only practical way to use generative AI every day.  

Developer Implementation: The MLX LM Ecosystem 

Apple has made it easy for engineers to get started. The Mix LM package lets you add OpenELM 2.0 to a Swift or Python project and handles converting and quantizing Hugging Face rates with just one CLI command or a few lines of Python code. It uses modern containerization to manage memory limits, allowing the system to adjust its GPU cache for devices ranging from iPhones with 8 GB of RAM to M3 Max workstations with 128 GB.  

Final Thoughts 

The Apple Open ELM 2.0 MLX Framework signals a new era for local machine learning with a private 150-token/SEC local RAG. Apple allows that retina-level AI, which delivers instant responses, to run without a huge server farm. As the open-source community continues to improve these models and the MLX framework gains more support, cloud-only LLMs will lose their edge for anyone who cares about both privacy and performance. Local elements are not just here they’re running 150 tokens per second. 

Source: OpenELM: An Efficient Language Model Family with Open Training and Inference Framework 

Today, Microsoft released its March 2026 Patch Tuesday updates addressing 79 security flaws. This includes two zero-day vulnerabilities that have been publicly disclosed.  

Transitioning from the initial overview, Dispatch Tuesday also fixes three serious vulnerabilities. Two are remote code execution issues, and one is an information disclosure flaw.  

Here is a breakdown of the number of bugs in each vulnerability category. In cybersecurity, a vulnerability is a weakness that could allow unauthorized actions in a system.  

• 46 elevation of privilege vulnerabilities.  

• Two security features bypass vulnerabilities  

• Eighteen, remote code execution on the line of notice.  

• Information Disclosure Vulnerabilities  

• Four denial-of-service vulnerabilities.  

• Four spoofing vulnerabilities.  

Bleeping Computer only counts Patch Tuesday security updates released by Microsoft on the day itself. This means the total does not include 9 Microsoft Edge flaws or issues in Marina Payment Orchestrator Service, Azure, and Microsoft Devices Pricing Program that were fixed earlier this month. Thank you.  

Two Zero-Day Vulnerabilities and Microsoft Office Flaws 

This month’s Patch Tuesday addresses two publicly disclosed zero-day vulnerabilities with no reports of active exploitation.  

Microsoft defines a zero-day flaw as one that is either publicly disclosed or actively exploited before an official fix is available.  

The two publicly disclosed zero-day vulnerabilities are:  

CVE-2026-21262 – SQL Server Elevation of Privilege Vulnerability  

Microsoft has fixed this publicly disclosed SQL Server flaw.  

Improper access control in SQL Server allows an authorized attacker to elevate their privileges on a network, Microsoft explains.  

Microsoft credited Erland Sommarskog for discovering this flaw. He told Bleeping Computer that it was first disclosed in the “Packaging Permissions in Stored Procedures” article.   

CVE-2026-2612. net denial of service vulnerability.  

An “Out of bounds read” in .NET allows an unauthorized attacker to deny service over a network, Microsoft explains.  

An anonymous researcher is credited with finding this flaw.  

Microsoft has also addressed two remote code execution bugs (CVE-2026-26110 and CVE-2026-26113) in Microsoft Office. Remote code execution (RCE) refers to bugs that could allow attackers to run programs on your computer from a distance. These can be exploited through the preview pane, so users should update the application as soon as possible.  

One remarkable issue is the Microsoft Excel Information Disclosure flaw (CVE-2026-26144), which could allow attackers to steal data using Microsoft Copilot. An information disclosure occurs when unauthorized people can access confidential data.   

An attacker who successfully exploited this vulnerability could cause Copilot Agent Mode to exfiltrate data via unintended network egress, enabling a zero-click information disclosure attack, Microsoft explains.  

Turning to Azure Cloud Services, Microsoft has also fixed a vulnerability in Azure Container Instances (ACI) that could have allowed users to access information belonging to other Azure customers.  

Microsoft did not share technical details about the vulnerability. However, Palo Alto Networks researchers say attackers could have used the bug to run code on other users’ containers, steal sensitive data like crypto secrets, or even install crypto-mining malware.   

Microsoft said it notified customers who might have been affected through service health notifications in the Azure portal. The company added that anyone who did not receive a notification does not need to take any action.  

There is no indication that any customer data was accessed as a result of this vulnerability. Out of an abundance of caution, Microsoft sent notifications to customers potentially affected by the researcher’s activities, advising them to revoke any privileged credentials deployed to the platform before August 31, 2021, Microsoft said in a statement.  

Microsoft recommends that all Azure customers regularly rotate their privileged credentials as a precaution. Credential rotation involves frequently changing passwords or secure keys to reduce the risk of theft.  

Palo Alto Networks researchers say the issue known as Azure escape could allow attackers to compromise Kubernetes clusters that host ACI. This would give them full control over other Azure customers’ containers.  

ACI was designed to prevent attacks from malicious neighboring containers, such as cross-account or cross-tenant attacks. However, it used an older version of runc (the standard container runtime), which was vulnerable to several container escape flaws.  

The researchers used a modified proof-of-concept code for CVE-2019-5736 to escape from a container and establish a root reverse shell on the host system. They then found token permissions in the Kube system namespace that allowed them to run commands on any pod in the cluster and carry out the cross-account attack.  

A malicious Azure user could have compromised the multi-tenant Kubernetes cluster hosting ACI. As a cluster administrator, an attacker could execute commands in other customers’ containers, exfiltrate secrets and private images deployed to the platform, or deploy crypto miners. A sophisticated adversary would further investigate the detection protocols protecting ACI to avoid detection, Palo Alto said.  

ACI, launched in July 2017, is a container-as-a-service (CaaS) that runs on multi-tenant clusters such as Kubernetes and Service Fabric. It lets users deploy containers without managing the underlying infrastructure.  

In a related development, two weeks ago, Microsoft fixed a similar problem in Azure Cosmos DB. In that case, users could access other customers’ databases with full administrative rights, potentially gaining full control of them.

Source: Microsoft March 2026 Patch Tuesday fixes 2 zero-days, 79 flaws 

Microsoft Warns of Information Leak Flaw in Azure Container Instances

Silicon Motion has launched the SM8008A PCIe Gen5 x4 NVMe SSD controller. This controller delivers high performance and sets industry-leading energy efficiency for data-center boot drives and other enterprise storage.  

With the growth of AI and cloud services, data centers are adding more servers than ever. Each server requires a system drive to run its software. In large-scale data centers, the energy used by these systems accumulates and can account for a significant portion of total power consumption.  

Silicon Motion SM8008 Features And Support 

The SM8008 directly tackles the challenge of balancing high performance with low power draw in modern data centers. Built with TSMC’s 6nm process, it reaches up to 14 GB/s in sequential speed, delivers over 2.3 million random IOPS for 4 KB tasks, and keeps power consumption under 5 watts. With 8 NAND channels and support for ONFI and DDR5.0 interfaces up to 3600 MT/s, the S8 offers robust and efficient storage management.  

The controller uses a PCIe Gen5 x4 interface and runs on the NVMe 2.0a protocol, delivering the required performance for new server platforms. Depending on the SSD configuration, it supports up to 16 TB of storage.  

To help data centers cut energy costs when running thousands of drives, the controller supports DDR4-3200 or LPDDR4-3200 DRAM featuring built-in error correction. This approach reduces power consumption while maintaining reliability, making it cost-effective for enterprise environments.  

Silicon Motion SM-80008 Security 

With built-in security features like TCG, Profile 2.0, encryption, and hardware support for AES-256, SHA-512, and RSA-3072, the SM-8008 helps enterprises ensure robust protection for critical data while supporting secure boot and firmware integrity.  

The controller also incorporates security schemes such as DICE and SPDM and already meets the CNSA 2.0 cryptographic standard. As a result, it will satisfy new requirements expected for some government and business users starting in 2027.  

Silicon Motion SM8008 Storage Capabilities 

For storage, the controller uses Silicon Motion, NAMD command technology, and a strong LDPC error-correction engine. These features protect data and help the drive last longer, even with TLC and QLC memory. The controller’s hardware also supports NAMD processing methods that increase speed and throughput.  

The SM8008 enables flexible deployment in demanding environments by supporting standards used in large-scale data centers and enterprises. Its compatibility with the NVMe 2.0a protocol and the OCP Hyperscale NVMe Boot SSD Specification version 1.0 ensures seamless interoperability. The controllers’ support for multiple SSD form factors such as M.2, U.2, E1.S, and E3.S allows vendors to streamline server and storage architectures for diverse applications.  

Silicon Motion SM8008 Specifications 

Category	Details
Host Interface	PCIe Gen5 x4
Specifications.	NVMe 2.0A protocol support   OCP Hyperscale NVMe Boot SSD Specifications Version 1.0 Compliant (Partial)
NAND Interface	8 NAND channels supporting ONFI and Toggle DDR 5.0 up to 3600 MT/s.
DRAM Interface	DDR4-3200 and LPDDR4-3200 with inline PCC support.
Performance.	Up to 14 GB sequential performance with active power under 5 W
Security features.	TCG Opal 2.0 AES-256 SHA2-512 RSA-3072 B DICE SPDM Secure Boot   NAND, common maximizing enterprise performance of next generation NAND geometries with LDPC error correction and endurance extension for QLC and beyond.
Enterprise features.	Supports NVMe Management Interface Basic Management Command (NVMe-M1V1.2)   Supporting advanced data placement of images, SR‑IOV with 64 virtual functions.   CNSA 2.0 support   Up to 16 TB

Silicon Motion SM8008, Availability 

Interest in controllers built specifically for boot storage is growing as hyperscale infrastructure evolves. Early deployments of the SM8008 are underway with companies such as ATP and Exascend integrating the controller into new enterprise SSD platforms for large server environments. Silicon Motion’s boot storage lineup now includes SLTA, PCIe Gen3/Gen4 controllers, PCIe NVMe BGA SSDs, and the SM8008 Gen5 controller, underscoring its focus on dedicated enterprise boot storage.t.  

About Silicon Motion 

Silicon Motion Technology Corporation is a global leader in NAND flash controllers for solid-state storage devices. The company ships more SSD controllers than any other supplier worldwide for servers, PCs, and other devices. It also dominates as the leading merchant provider of eMMC and UFS, as well as embedded storage controllers for smartphones, IoT products, and automotive applications.  

Silicon Motion delivers custom high-efficiency solutions for hyperscale data centers, industrial systems, and automotive SSDs. The company designs its controllers for advanced AI, cloud, and enterprise storage platforms, ensuring high performance, low power use, and reliable operation.  

Many of the world’s NAND flash vendors, data center and enterprise storage providers, storage device makers, and top OEMs rely on Silicon Motion’s controller technologies to achieve innovative, high-quality storage solutions. www.siliconmotion.com

Source: Silicon Motion Launches SM8008, a Purpose-Built PCIe Gen5 Controller for Enterprise Boot Drives and Ultra-Low-Power Storage 

Samsung has begun shipping its 6th-generation HBM4 memory designed for AI computing. These 10 nm-plus 1C chips built on a 4nm logic base die reach speeds up to 11.7 Gbps—well above the industry standard. Positioned to support advanced AI platforms such as NVIDIA’s upcoming Vera Rubin chips, this release marks a significant advance in memory technology.  

Main details of Samsung’s HBM4: 

• Performance: Achieves 11.7 Gbps per pin, with potential for 13 Gbps. This is 46% faster than the industry standard for HBM4 (8 Gbps per pin).  

• Capacity: 12-layer strikes offer 24GB to 36GB. Semicolon 16-layer strikes may reach 48GB.  

• This is a 6th‑Gen 10nm‑class DRAM core and a 4nm logic‑based die. Bandwidth per stack is 2.7× that of HBM3E, up to 3.3 TB/s.  

• Efficiency: power is 40 times better, thermal resistance is up 10%, and heat dissipation is up 30%.  

• Manufacturing: The first shipments have gone out to customers who will use them in advanced accelerators, marking a milestone in adoption.  

Samsung aims to lead the AI memory market with HBM4, having solved earlier production issues to meet high demand. This strategic focus sets the context for its recent progress.  

Samsung is expected to officially launch HBM4 at NVIDIA GTC 2026 in March, following the successful completion of NVIDIA’s final tests and as part of the major Rubin AI platform announcement.  

Having passed NVIDIA’s verification, Samsung’s HBM4 will debut at NVIDIA GTC 2026 for the Vera Rubin AI platform.  

Over the past few years, Samsung has experienced issues with its HBM memory division. During this period, its South Korean competitor SK Hynix became NVIDIA’s exclusive supplier of HBM3 and HBM3E memory, highlighting a shift in competitive dynamics. In response, Samsung has overhauled its HBM and semiconductor divisions and is now seeing results from those changes.  

NVIDIA will reportedly use its first allotment of HBM4 memory from Samsung’s Vera Rubin, as Samsung’s new HBM4 memory is the best of the HBM4 offerings compared to rivals SK Hynix and US-based Micron. Samsung’s new HBM4 memory is rated above 11 Gbps, well above JEDEC standards for HBM4, and was pushed and requested by NVIDIA at those higher pin speeds.  

Samsung and NVIDIA are collaborating on HBM4. Reflecting on this, Samsung stated in a press release that its cutting-edge HBM solutions’ high bandwidth and energy efficiency should help accelerate future AI development and support the manufacturing infrastructure built on these technologies.  

Samsung employs 6th-generation 10 nm-class DRAM and a 4 nm logic-based die with HBM4 speeds up to 11 Gbps. They plan to continue advancing their memory and foundry services to support global AI expansion.  

Samsung announced mass production of HBM4 and initial customer shipments, becoming the industry’s first to do so.  

Samsung used its cutting-edge 6th-generation 10nm DRAM processor (1C) to achieve stable yields and top performance right from the start of mass production. This was done smoothly, without any extra redesigns, to achieve stable yields and top performance right from the start of mass production. This was done smoothly without needing any extra redesigns.  

Instead of taking the conventional path of using proven designs, Samsung took the leap and adopted the most advanced nodes, such as 1C DRAM and a 4nm logic process, for HBM4, said Sang Joon Hwang, Executive Vice President and Head of Memory Development at Samsung Electronics. By leveraging our process, competitiveness, and design optimization, we can deliver significant performance gains, enabling us to meet our customers’ escalating demands for higher performance when they need it.  

Raising the Standard for Effectiveness and Efficiency 

Samsung’s HBM4 runs at a steady 11.7 Gbps, about 46% faster than the industry-standard 8 Gbps. This is a 1.22x increase over HBM3E’s maximum of 9.6 Gbps. HBM4 can potentially reach up to 13 Gbps, helping to reduce data bottlenecks as AI models grow.  

The memory bandwidth per stack is now 2.7x that of HBM3 (1.2 TB/s), reaching up to 3.3 TB/s. HBM4 comes in capacities ranging from 24 GB to 36 GB and will expand to 48 GB with 16-layer stacking, further exceeding HBM3E’s previous capacities.  

To manage higher power and heat, Samsung added a low-power design to the core die. HBM4 is 40% more efficient, offers 10% better thermal resistance, and provides 30% better heat dissipation than HBM3E.  

By delivering leading performance, energy efficiency, and reliability, Samsung’s HBM4 positions customers to maximize their GPU investments and meet growing data center demands with confidence.

Source: Samsung Ships Industry-First Commercial HBM4 With Ultimate Performance for AI Computing 

Cloud providers like Microsoft, CoreWeave, and Oracle Cloud Infrastructure are rolling out NVIDIA GB300 NGL72 systems for low-latency and non-context tasks, including agentic coding and coding assistants.  

Building on this hardware expansion, leading inference providers like Base 10, Deep Infra, Fireworks AI, and Together AI use the NVIDIA Blackwell platform to reduce token costs by up to 10 times. The new Blackwell Ultra platform advances these gains for agentic AI applications.   

AI agents and coding assistants have driven a significant increase in programming-related AI queries, rising from 11% to above 50%, according to the OpenRouters State of Inference report. As a result, these tools need low latency for instant responses and long context to understand and operate over entire codebases, enabling them to handle larger projects or complex workflows effectively.  

Semi-Analysis InferenceX data shows that NVIDIA’s software and Blackwell Ultra platform bring significant advances, with GB300 and NVL72 offering much higher throughput per MW and lower token costs than the Hooper platform.  

NVIDIA’s work in chip system design and software boosts performance for AI workloads, from authentic coding to interactive assistants, while also lowering costs at scale.  

GB300 NVL72 Delivers Up to 50x Better Performance for Low-Latency Workloads. 

Signal 65’s recent analysis shows that NVIDIA, JB200, NVL72, with advanced hardware and software design, deliver over 10 times more tokens per word and cut token costs to one-tenth compared to the HOPO platform. These games keep growing as technology improves.  

Current updates from the NVIDIA, TensorRT, LLM, Dynamo, Mooncake, and SGLang teams continue to improve Blackwell NVL72’s throughput for a mixture of X-Buds inference across all latency levels. For example, recent TensorRT LLM updates have made GB200 up to 5 times faster for low-latency tasks compared to 4 months ago.  

• Faster, more efficient GPU kernels help Blackpool fully utilize its computing power and increase throughput.  

• NVIDIA NVLink Symmetric memory allows GPUs (graphics processing units) to access each other’s memory directly, improving the efficiency of data exchange between processors.  

• Programmatic Dependent Launch releases in real time by starting the next kernel setup before the previous one finishes.  

With these software improvements, the GB300 NVL72 equipped with Blackwell Ultra GPU now achieves 50 times the throughput per megawatt of the Hopper platform.  

These improvements result in much lower costs across all latency levels, with the largest savings up to 35× lower per million tokens at low latency.  

The GB300NVL72 system and its software stack, including Dynamo and the TensorRT LLM, offer dynamically lower port token costs than the Hooper platform.  

For agent decoding and interactive assistant workloads, where every millisecond counts across multi-step workflows, this ongoing software optimization, paired with next-generation hardware, lets AI platforms scale real-time, interactive experiences to support far more users.  

GB300 NVL72 Deliver Superior Economics for Long-Context Workloads. 

Both GB200 and GB300 deliver low latency (quick response times), but GB300 and NBL72 are better suited for tasks that require processing large amounts of information at once (long-context tasks). For example, with large code inputs and outputs, GB300 NBL72 reduces token costs by up to 1.5× versus GB200 NBL72.  

Context grows as the agent reads in more of the code. This allows it to better understand the codebase, but it also requires much more computing power. Blackwell Ultra delivers 1.5× higher NVFP for compute performance and 2× faster attention processing, enabling the agent to efficiently understand entire code bases.  

Infrastructure For Agentic AI 

Leading cloud providers and AI innovators have already deployed NVIDIA GB200 NVL72 at scale. They are also deploying GB300 NBL72 in production. Microsoft, CoreWeave, and OCI use GB300 NBL72 for low-latency and long-context use cases, such as agentic coding and coding assistance, by reducing token costs. This GB300 NBL72 enables a new class of applications that can resume across massive codebases in real time. 

As inference moves to the center of AI production, long-context performance and token efficiency become critical. Said Chen Goldberg, Senior Vice President of Engineering at CoreWeave. Grace Blackwell NBL72 addresses that challenge directly, and CoreWeave’s AI cloud, including CKS and SUNK, is designed to translate GBL300 systems’ gains—building on the success of GB200 into predictable performance and cost efficiency. The result is better token economics and more usable inference for customers running workloads at scale.  

NVIDIA, Vera Rubin, NVL72 To Bring Next Generation Performance 

As adoption grows, ongoing software updates for NVIDIA BlackBand systems will continue to improve performance and reduce costs for all users.  

Going forward, the NVIDIA Rubin platform, which combines six new chips into a single AI supercomputer, will deliver even greater performance gains. For MOE inference, Rubin offers up to 10 times more throughput per megawatt than Blackwell, cutting costs to $110 per million tokens. Rubin can also train large MOE models with only a quarter as many GPUs as Blackwell. 

Source: New SemiAnalysis InferenceX Data Shows NVIDIA Blackwell Ultra Delivers up to 50x Better Performance