Tech News 2025 | AI Innovations, Product Launches & Trends

Home
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
About US
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
Tech Reviews
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
- AI
- Troubleshooting
- Smartwatches
- Smartphones
- Camera
- Gadgets
- Games
- Laptops
- Seasonal Sales
Phone Features
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
Buying Guides
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
Comparison
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
News
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
Contact
Agentic AI systems must use deep models capable of independently solving complex technical problems.

Multi-agent systems can produce up to 15 times more tokens than standard chats since they keep sending history, tool outputs, and reasoning steps at each step during long tasks. This context explosion can lead to the world rift, where agents slowly lose track of the main objective due to the need to use large reasoning models for every subtask, known as the thinking tax. Also, these applications are too costly and slow in real-world use.

Today, we are announcing Nemotron 3 Super to solve these problems. The new Super model has 120 billion total parameters, with 12 billion active at a time. It is designed for maximum effectiveness and precision in complex multi-agent tasks such as software development and security triage. This release follows our introduction of Nemotron 3 Nano in December.

Number 23 Super solves the thinking tax problem with its hybrid mixture-of-experts (MoE) design. It offers more than five times the throughput of the previous Nemotron Super. The model also handles context explosion with a built-in 1-million-token context window, providing agents with long-term memory for accurate reasoning. It is fully open, with open weights, datasets, and recipes, so developers can easily customize, optimize, and deploy it on their own systems.

WhatsApp's Nemotron 3 Super Apart

Nemotron 3 Super introduces design features to reduce trade-offs between effectiveness and correctness in large reasoning models.
- Latent MOE (a type of mixture-of-experts architecture that compresses hidden data representations) uses token compression to activate more experts (specialized submodels) per inference at the same computational cost.
- Multi-token prediction accelerates long-sequence generation by enabling single-step future-token prediction, thereby enabling speculative decoding.
- A Hybrid Mamba Transformer backbone means the model has two main types of layers. Mamba layers process long sequences efficiently, while Transformer layers are specialized for exact reasoning. This combination increases the model's speed and makes it four times more memory- and compute-efficient.
- Native NVFP4, used for pre-training, is a special low-memory format built for NVIDIA Blackwell chips. It reduces memory usage and speeds inference (model output) by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining high accuracy.
- After initial training, the model uses reinforcement learning (AI learns by trial and error) across 21 environments, running on NVIDIA Nemotron Gem and Nemotron RL and accumulating 1.2 million simulated experiences (environment rollouts).
These advantages combine to create a model well-suited for long-running autonomous agents on Pinchbench, an innovative benchmark for evaluating how well LLMs perform as the brain of an open-cloud agent. Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.

Delving Deeply Into The Architecture

Hybrid Mamba Transformer MOE Backbone

Super uses the same blended approach as nano, but it operates on a much larger scale to understand how its architecture supports this. Consider the way its backbone combines three types of layers.

Mamba-2 layers handle most sequence processing. These State-Space Models operate in linear time with respect to sequence length, enabling the practical use of a 1M Token Context Window. Mamba layers efficiently manage memory when processing large code lengths, extended chat histories, or many documents.

Transformer attention layers are inserted at key points because SSMs may struggle to locate specific facts in a known context. These attention layers preserve the ability to retrieve targeted information within large inputs.

MOE layers increase the number of effective parameters without needing heavy computation. Only some experts are used for each token, which keeps latency low and throughput high. This is important when many agents run concurrently in a shared system.

Latent MOE

In a typical MOE setup, tokens are routed from the full hidden dimension (all the internal data space) to the different experts (specialized sub-networks). This process can slow down computation, increase costs, and limit the number of experts the model can use.

Supersuper uses latent MOE (working in a reduced dimension before routing). Here, token embeddings (numerical summaries of tokens) are compressed into a simpler, smaller space. The experts perform their tasks in this compressed space, and then their results are expanded to match the full model size. This approach has practical effects:

More experts can be used at the same cost per computer (i.e., the same amount of computer resources). Compression enables the model designer to support four times as many experts without requiring more computation.

Finer-grained specialization, when more experts are available, the model can afford highly specialized routing, for example, activating distinct experts for Python syntax versus SQL logic only when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and dialogic reasoning within a few terms.

Multi-Token Prediction (MTP)

Standard language models are trained to predict one token at a time. A fundamentally myopic objective, Super is trained with MTP, where specialized prediction heads simultaneously forecast seven future tokens at each position.

This has two concrete benefits:

This leads to better reasoning during training. Predicting multiple future tokens helps the model learn longer patterns and logical connections, rather than just guessing the next word. The model is trained to predict entire sequences, which improves performance on tasks that require step-by-step logic.

MTP also speeds up inference by predicting multiple future tokens in a single pass. This reduces the time required to generate long outputs. The MTP heads make draft predictions that can be checked in parallel, enabling up to 3x faster generation for tasks such as code and tool calls without requiring a separate draft. Both benefits come from a single design. Shared weights across all MTP heads limit the number of extra parameters and stabilize training by aligning heads, keeping draft predictions more consistent across longer sequences.

Native NVFP4 Pre-Training

Most quantized models are first trained in full precision and then compressed, which usually results in some loss of accuracy. Super does things differently. Most floating-point operations during pre-training use NVFP4 and NVIDIA's 4-bit floating-point format. This format, optimized for Blackwell, greatly reduces memory usage and speeds up inference compared to FP8 while maintaining high accuracy.

By training natively in reduced precision (only using 4-bit math from the beginning), the model learns to be accurate even with small numbers, meaning it stays stable and effective while using much less memory.

Training Super: A Three-Stage Process
1. Supervised fine-tuning is a phase in which the model is further trained on specific examples relevant to the tasks it will face, so it learns to act appropriately for those tasks.
1. Reinforcement learning is used to further improve model behavior by letting it learn from actual outcomes in test scenarios.
Pre-training: Super is pre-trained on 25 trillion tokens using the NVFP4 format, learning to be accurate with four-bit math throughout pre-training. The data includes 10 trillion unique curated tokens focused on reasoning and coding.

Supervised Fine-Tuning: Before Reinforcement Training, Super is being fine-tuned with about 7 million supervised samples. These come from a larger set of 40 million samples that include reasoning, following instructions, coding safety, and multi-step agent tasks. This stage lays the foundation for the behavior RL will employ. The model teams learn to give correct responses across different tasks, so RL starts from a stable base rather than a raw, pre-trained model.

Multi-environment reinforcement learning makes the model more agent-like by training it across many environments (such as Nemotron, GitHub, NMEDIAS, and the Open RL Library). These scenarios test tasks such as tool use, coding, and complex planning, creating the main dataset for reinforcement training.

This type of reinforcement learning step helps the model work reliably in multi-step workflows, reduces reasoning errors, and manages the organized tasks often found in agent pipelines.

The Super + Nano Deployment Pattern

Nemotron 3 Nano works well for processing specific targeted steps in an agent-like workflow; however, as multi-agent applications become more complex and involve several steps, a more powerful model is needed for better planning and reasoning. For example, imagine an agent that needs to choose among tools to create a presentation with 10 high-quality slides.

Nemotron 3 Super is a great fit for these situations in software development. For example, Nemotron 3 Nano can handle single merge requests, while Nemotron 3 Super can handle more complex coding tasks that require a deeper understanding of the codebase. For expert-level coding, proprietary models are best.

Building With Super's Open Resources

Nemotron 3 Super is fully open source, including model weights,datasets, and architectural recipes, enabling developers to tailor, refine, and deploy the model for privacy and security.

Get Started

With these capabilities in mind, getting started is straightforward. Start using Nemotron 3 Super today. Deploy it on your preferred platform, whether on a workstation or in the cloud. To experience its capabilities, sign up for a pro subscription on Perplexity. Access it directly by your API, use OpenRouter, or visit build.NVIDIA.COM. Take the next step. Explore Nemotron 3 Super Now!

Source: Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning

NVIDIA Nemotron 3 Architecture Reveals Hybrid Model Built for Long Context AI Agents

Harish Shenoy
March 15, 2026
12 min read
267

Agentic AI systems must use deep models capable of independently solving complex technical problems. Multi-agent systems can produce up to 15 times more tokens than... Details

GitHub Agent HQ Internal Leak Reveals Microsoft’s New AI Army Command Center

Harish Shenoy
March 15, 2026
15 min read
274

At Universe 2025, GitHub unveiled Agent HQ, a platform that serves as machine control for AI coding assistance from vendors like OpenAI, Anthropic, Google, and... Details

Tesla Optimus AI Stack Adds Vision-Language Navigation For Humanoid Autonomy

Harish Shenoy
March 15, 2026
9 min read
330

Tesla has advanced general-purpose robotics. The latest Optimus AI update adds vision-language navigation, enabling the robot to reason. By merging language understanding with spatial cognition,... Details

NVIDIA Isaac SDK Update Adds On-Device Reinforcement Learning For Factory Robots

Harish Shenoy
March 15, 2026
9 min read
248

The global manufacturing sector is navigating a seismic shift in which yesterday's static automation can no longer keep pace with the race for competitiveness. As... Details

Microsoft Windows HotPatching Expands to AI Clusters for Zero-Downtime Security

Harish Shenoy
March 15, 2026
7 min read
342

Windows auto-patch will enable hot-patch security updates by default to accelerate device security. This change in default behavior will apply to all eligible devices managed... Details

AWS Nitro Update Introduces Enclave-Level Isolation for Sovereign AI Inference.

Harish Shenoy
March 15, 2026
12 min read
289

AWS has brought out major updates to the Nitro system, making it even more secure for AI inference. With improved enclave-level isolation, organizations can process... Details

Android 17 Secure Companion API Standardizes Biometric Authentication for AI Wearables

Harish Shenoy
March 15, 2026
8 min read
260

Mobile technology is changing fast as the line between smartphones and wearables blurs. In early 2026, Google introduced the Android 17 secure companion API to unify wearable... Details

Apple Metal 4 API Documentation Confirms Native Neural Scheduling For Mac GPUs

Harish Shenoy
March 15, 2026
6 min read
332

Apple Metal 4 now provides native low-latency neural scheduling for Mac GPUs, enabling developers to integrate AI features more efficiently. Developers can now run Core... Details

Samsung 2nm Exynos 2600 Hits Volume Production With New GAA Transistor Yields

Harish Shenoy
March 14, 2026
8 min read
406

Reports from late 2025 and early 2026 show that Samsung is mass-producing the Exynos 2600 with its 2nd-generation 2NM gate-tolerant (GAA) transistor technology. Wells have improved to 50-60%.... Details

Anthropic Claude SDK 4.2 Adds Native Multi-agent Routing For Enterprise AI

Harish Shenoy
March 14, 2026
10 min read
304

Anthropic has updated its agentic development capabilities with the transition from the Claude code SDK to the Claude agent SDK in early 2026, providing native,... Details

Prev.
1
…
141
142
143
144
145
…
186
Next

Latest post

Which Top Books Did Amazon’s Editors Name as Best of 2026?

When Can You Download App Free New App Award Winners?

How Can Your Home Fire TV Stream Every World Cup Match?

Popular Posts

Best Budget Smartphones 2026: Affordable Phones That Impress (4122)

Best Business Laptops 2025 (3732)

The Future Is Calling: Top Upcoming Smartphones of 2026 You’ll Want to Wait For (3156)

DSLR vs Mirrorless: Which Is Better for Photography Beginners? (2535)

NIST Update Signals Fast Track for Post-Quantum Standards (2320)

Stay Connected

Category: News