Nemotron-3-Nano-30B-A3B

Nemotron 3 Nano 30B-A3B

Sparse twice over: six experts of 128, and six attention layers of 52.

The Layer Composition

The architecture, and it is the reason everything else on this page works.

Six attention layers out of fifty-two. Under twelve percent.

Mamba-2 carries sequence state through a recurrent update at constant cost, regardless of how much input arrived. No key-value cache that grows with length. Forty-six of fifty-two layers behave that way.

The six attention layers use grouped-query attention with two groups, which compresses even that small cache further.

Why Long Context Is Cheap Here

The comparison worth making explicitly, because a large window means different things on different architectures.

On a conventional transformer, every layer builds a KV cache that grows linearly with input. At a few hundred thousand tokens that cache becomes the dominant memory cost — which is why large transformer models advertise long windows and serve a fraction of them in practice.

Here, forty-six layers build nothing. Only six contribute to the cache, and those six use grouped-query attention with two groups.

NVIDIA's own reference command reaches up to a million tokens on a single GPU — tensor parallelism of one, with an FP8 KV cache. The architecture supports considerably more than most deployments configure.

Which makes 262,144 tokens comfortable rather than strained. On a conventional model of this total size, a quarter-million-token window is the memory budget's limit. Here it sits well inside what the design allows — no scaling factor to tune, no trade-off where enabling long context degrades short prompts.

Independent evaluation reports roughly 86.3% on RULER for long-sequence comprehension. That matters because holding a long window and retrieving accurately across it are separate achievements.

Six Experts of 128

The other axis of sparsity, and NVIDIA describe it precisely.

A granular MoE architecture with a learnt MLP router activating 6 of 128 experts.

Ten percent of the parameters, under five percent of the expert pool.

"Granular" is the operative word. Many small experts with several selected per token, rather than few large experts with one or two. That granularity lets the router compose specialists rather than pick one — a different bet from the ultra-sparse designs elsewhere in this catalogue.

And the previous generation is the comparison NVIDIA draw: better accuracy while activating less than half the parameters per forward pass.

Throughput: Up to 3.3×

The claim, with its condition.

Up to 3.3 times higher inference throughput than similarly-sized open models, while also being more accurate on popular benchmarks.

Measured at 8K in, 16K out, on a single H200 — a specific configuration rather than a general assertion, which makes it checkable.

Where the speed comes from. Two mechanisms multiplying: 3.2 billion active parameters instead of thirty-one, and forty-six layers that never touch a growing cache. Neither alone would produce that figure.

What it means for an agent loop. A hundred sequential steps at 3.3× is the difference between a workflow that runs while you wait and one you come back to.

Fully Open

NVIDIA released more than weights, and they list it.

Both checkpoints — the pre-trained base and the post-trained final model.

The training recipe.

The data.

The code.

A technical report documenting the architecture, the training, and the evaluation.

Why that combination is rare. Open weights let you run and fine-tune a model. Open data and an open recipe let you audit what went into it and rebuild from the same foundation. For regulated work, for research, and for anyone whose organisation asks what a model was trained on, that is a different category of answer.

And the base checkpoint has been used. A sibling model post-trained from this same base reached gold-medal performance at both the 2025 International Mathematical Olympiad and the International Olympiad in Informatics — which says something about what the foundation supports that no benchmark table does.

Training

Publishing the learning-rate schedule and batch size is not something most releases do. It is the kind of detail that matters only if you intend to train something yourself — and NVIDIA released the recipe precisely so that you can.

Specifications

Languages: English and coding languages primarily, with Spanish, French, German, Japanese, and Italian also supported.

Published precisions: BF16 and FP8 from NVIDIA directly, plus community GGUF builds.

Capabilities

Reasoning Is a Flag, and Turning It Off Costs Something

The model responds by first generating a reasoning trace and then concluding with a final response, configured through a flag in the chat template rather than a request parameter.

NVIDIA state the trade honestly:

If the user prefers the model to provide its final answer without intermediate reasoning traces, it can be configured to do so, albeit with a slight decrease in accuracy for harder prompts that require reasoning.

Read the qualifier. The cost is on harder prompts that require reasoning — not on everything.

Which sorts the decision cleanly. Classification, routing, extraction, and formatting lose nothing by disabling it; they have one correct answer and no reasoning to do. Analysis, mathematics, and multi-step problems pay for it.

Through an API the flag is handled for you. It matters when self-hosting.

Using Nemotron 3 Nano on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: nvidia/Nemotron-3-Nano-30B-A3B

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="nvidia/Nemotron-3-Nano-30B-A3B",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "nvidia/Nemotron-3-Nano-30B-A3B",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Nemotron-3-Nano-30B-A3B",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

An Agent Loop, Where the Throughput Shows

The workload NVIDIA name as the design target — agentic reasoning.

import json
import time

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read a file relative to the repository root.",
            "parameters": {
                "type": "object",
                "properties": {"path": {"type": "string"}},
                "required": ["path"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "run_tests",
            "description": "Run the test suite and return pass/fail counts with failure output.",
            "parameters": {"type": "object", "properties": {}},
        },
    },
]


def read_file(path: str) -> dict:
    """Replace with your real, sandboxed file access."""
    raise NotImplementedError


def run_tests() -> dict:
    """Replace with your real, sandboxed test runner."""
    raise NotImplementedError


HANDLERS = {"read_file": read_file, "run_tests": run_tests}

session = [
    {"role": "user", "content": "The DZD invoice test fails on totals ending in .005. Find the cause and fix it."}
]

CEILING = 80
start = time.monotonic()

for step in range(CEILING):
    response = client.chat.completions.create(
        model="nvidia/Nemotron-3-Nano-30B-A3B",
        messages=session,
        tools=TOOLS,
        max_tokens=16384,
    )

    message = response.choices[0].message
    session.append(message)

    if not message.tool_calls:
        print(message.content)
        break

    for call in message.tool_calls:
        handler = HANDLERS.get(call.function.name)
        if handler is None:
            outcome = {"error": "unknown tool", "name": call.function.name}
        else:
            try:
                outcome = handler(**json.loads(call.function.arguments or "{}"))
            except Exception as exc:
                # Failures return as data — large-scale RL on diverse environments trained for this.
                outcome = {"error": type(exc).__name__, "detail": str(exc)}

        session.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(outcome)})
else:
    print(f"Stopped at the {CEILING}-step ceiling.")

print(f"{(time.monotonic() - start) / 60:.1f} min · {step + 1} steps")

Time the whole loop. A throughput claim is about tokens per second; what decides whether an agent is usable is wall-clock time across a full session, and those are different measurements.

An eighty-step ceiling is affordable here in a way it would not be on a model with ten times the active parameters. That is the throughput figure expressed as a design decision rather than a benchmark.

Return tool errors as data. Post-training included large-scale reinforcement learning on diverse environments — recovering from a failed call is behaviour the training produced, and an exception discards it.

Filling the Window

Where the Mamba-2 layers earn their place.

from pathlib import Path

REPO = Path("src")

sources = "\n\n".join(
    f"=== {path.relative_to(REPO.parent)} ===\n{path.read_text(encoding='utf-8')}"
    for path in sorted(REPO.rglob("*.py"))
)

response = client.chat.completions.create(
    model="nvidia/Nemotron-3-Nano-30B-A3B",
    messages=[
        {
            "role": "system",
            "content": (
                "You are auditing a codebase. Identify every path where a database write can occur "
                "outside a transaction. Name the file, the function, and the call chain that reaches "
                "it. Report nothing you cannot trace."
            ),
        },
        {"role": "user", "content": sources},
    ],
    max_tokens=32768,
)

print(f"input: {response.usage.prompt_tokens:,} of 262,144")

Asking for the call chain rather than the line is what uses a large window rather than a search. A write outside a transaction is easy to grep for; the path reaching it from three modules away needed the whole subsystem in one context.

262,144 tokens holds a substantial codebase — a subsystem, a service, a document set. And unlike most models at that window, filling it here costs a small model's memory rather than a large one's.

Measure recall at your working length. Six attention layers of fifty-two is a genuinely different retrieval mechanism from a conventional transformer — the RULER result suggests it holds, and whether it holds on your data is an empirical question with a cheap answer.

Reading the Reasoning

message = response.choices[0].message

trace = getattr(message, "reasoning_content", None)
if trace:
    logger.debug("reasoning: %d characters", len(trace))

print(message.content)

Keep the fields apart in both directions. Merging the trace into the answer breaks structured-output parsing and puts a working draft in front of readers expecting a conclusion.

Check finish_reason. Reasoning shares the output budget, and a truncated response can contain a complete trace and no answer at all.

⚠️ Self-Hosting: Three Things That Will Stop You

NVIDIA's reference vLLM command carries configuration that is not optional, and two pieces of it are unusual.

The reasoning parser is a plugin file

--reasoning-parser-plugin nano_v3_reasoning_parser.py
--reasoning-parser nano_v3

Not a built-in parser — an external Python file you supply alongside the launch command.

That is uncommon, and it fails in a specific way: without the plugin, reasoning traces leak into the visible answer rather than arriving in their own field.

The tool-call parser is Qwen's

--tool-call-parser qwen3_coder

An NVIDIA model using a Qwen parser. Surprising, and correct — tool-call formats converge across model families more than model names suggest.

Using the wrong value produces tool calls that arrive as prose, which looks like the model ignoring your tools rather than a configuration error.

Two environment variables

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
VLLM_USE_FLASHINFER_MOE_FP8=1

The first permits configurations beyond vLLM's default length check.

The second selects the optimised FP8 MoE kernel on the quantised build.

And the rest of the configuration

--trust-remote-code — the architecture is custom.

--kv-cache-dtype fp8 — halving the cache for the six layers that have one.

--tensor-parallel-size 1 — a single GPU, even at very long context.

--enable-auto-tool-choice for tool use.

Where It Fits

Agentic reasoning, which NVIDIA name as the design target and where throughput compounds across steps.

Long context on modest hardware — 262,144 tokens at a small model's memory cost, because the architecture rather than the budget sets the ceiling.

High-throughput production traffic, where 3.2 billion active parameters and 3.3× throughput decide the economics.

Multi-document and subsystem-scale analysis, with strong published long-context retrieval.

Retrieval-augmented systems and chatbots, both named in the intended use.

Regulated and audited environments, where open weights, open data, an open recipe, and a technical report together answer questions most models leave open.

Continued training, given the published base checkpoint and recipe — and a sibling post-trained from it reached olympiad gold.

Not for vision. Text only.

Not for peak capability. The Super and Ultra tiers in this family are where that lives.

Practical Notes

Disable reasoning for mechanical work — NVIDIA say the cost falls on hard prompts, not all prompts.

Time whole agent sessions, not single requests.

Return tool errors as data.

Check finish_reason — reasoning shares the budget.

Measure long-context recall at your real working length.

If self-hosting: the reasoning-parser plugin file, qwen3_coder for tool calls, both environment variables, --trust-remote-code, and an FP8 KV cache.

Consider the base checkpoint if you intend to post-train rather than only run inference.

Limitations

Text only. No image, audio, or video input.

3.2 billion active parameters is the compute ceiling per token. This is an efficiency model, and the larger tiers in this family exist for work that reaches it.

31.6 billion parameters must be loaded even though 3.2 billion run per token.

262,144-token window. Generous, and less than the architecture supports — the constraint is the deployment rather than the design.

Disabling reasoning costs accuracy on hard prompts, and NVIDIA say so rather than leaving it implicit.

The reasoning parser is an external plugin file. Without it, traces leak into the answer.

The tool-call parser is named for a different model family. Easy to get wrong, and the failure is silent.

A custom architecture. trust_remote_code is required, and framework support is newer than for mainstream designs.

Six languages beyond English and code.

Throughput figures are measured at a specific configuration — 8K in, 16K out, single H200. Your shape will differ.

Reasoning traces are working notes. Treat the final response as the output and the trace as debugging material.