Mistral-Nemo-Instruct-2407

Mistral Nemo Instruct 2407

Built jointly by Mistral AI and NVIDIA. A drop-in replacement for Mistral 7B with thirty-two times the context.

The 128k Window Came From Theta

The architectural detail that explains the headline capability, and it is a single number.

Rotary position embeddings, theta = 1,000,000.

The conventional value is 10,000. This model uses a hundred times that.

What theta does. Rotary embeddings encode a token's position by rotating its representation at a frequency derived from theta. A small theta rotates quickly — positions become distinguishable over short spans and start repeating over long ones. A large theta rotates slowly, which extends how far the model can tell positions apart before the encoding wraps.

Why it matters that this was trained in. A great many long-context models reach their advertised window by scaling positional encoding at inference — widening a window the model never saw during training. This one was trained with a 128k context window, with theta set for that range from the start.

The practical difference: behaviour at token 100,000 is behaviour the model was trained to produce, not behaviour extrapolated into a region it has never been.

Architecture

Thirty-two query heads against eight key-value heads. A 4:1 grouped-query ratio, cutting the key-value cache to a quarter of what full multi-head attention would need.

That ratio is what makes the long window affordable. A 128k context with full multi-head attention would need four times the cache memory for the same model — the difference between running on accessible hardware and not.

The 131,072-token vocabulary is the multilingual half of the design. A larger vocabulary means fewer tokens per word in non-English text, which costs less context and preserves more meaning per token. On a model trained across nine languages, that is not a detail — it is why the multilingual numbers hold up.

A Drop-In Replacement

An unusual claim, stated directly in the model card: drop-in replacement of Mistral 7B.

What that means in practice. Same interface, same prompt format, same integration. Change the model identifier and you get:

Thirty-two times the context — 128k against 4k.

A substantially more capable model, described as significantly outperforming existing models of similar or smaller size.

Nine languages trained in rather than incidentally present.

And no code to change. For anyone maintaining an integration built against the earlier model, that is the rare upgrade with no migration cost attached.

Benchmark Results

Published in the model card.

English

Multilingual MMLU

Reading the Numbers Honestly

Three things the table says that are worth acting on.

NaturalQuestions at 31.2% is the honest one

Far below everything else, and it is the benchmark that matters most for a common use case.

NaturalQuestions measures open-domain factual recall — real questions, answered from memory, with no document supplied. 31.2% means the model gets roughly two-thirds of them wrong.

And TriviaQA at 73.8% is not a contradiction. TriviaQA questions are more heavily represented in training data; NaturalQuestions are real search queries, which are stranger, more specific, and less likely to have been memorised.

What to do about it. Ground factual work. Retrieval, documents in the prompt, or a search tool — anything that moves the model from recalling to reading. A 128k window exists precisely to make that easy, and reading is where this model performs closest to a much larger one.

Do not use it as a knowledge base. That is the honest reading of 31.2%.

TruthfulQA at 50.3% is a second warning

That benchmark measures resistance to repeating common misconceptions. Half is a coin flip.

In practice: a confidently stated plausible-sounding claim from this model deserves checking, especially where a popular misconception exists on the topic.

Multilingual holds within five to nine points

English MMLU is 68.0%. The eight other languages range from 59.0% to 64.6%.

That gap is narrow by the standards of the field. A five-point drop for Spanish and a nine-point drop for Japanese, against a model where multilingual data was a large proportion of training rather than an afterthought.

Spanish, Portuguese, German, and French cluster above 62% — close enough to English that a product in those languages does not need a different model.

Specifications

Jointly trained — an unusual arrangement, and the reason the model is also published through NVIDIA's own NeMo framework under a separate identifier.

Apache 2.0 — commercial use, modification, and redistribution without conditions.

Capabilities

It Answers Directly

Worth stating explicitly, because most of this catalogue now reasons by default.

No thinking mode. No effort parameter. No reasoning_content field to read or replay.

Two consequences, and both are advantages in the right place.

max_tokens means what it says. The ceiling covers the answer alone — nothing shares it, nothing is consumed by a trace you did not request. On a reasoning model, a budget of 512 tokens can produce nothing at all; here it produces five hundred tokens of answer.

Latency tracks input and output length, not how hard the model judged the question. On an interactive endpoint, that predictability is frequently worth more than depth.

Using Mistral Nemo on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: mistralai/Mistral-Nemo-Instruct-2407

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Nemo-Instruct-2407",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "mistralai/Mistral-Nemo-Instruct-2407",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistralai/Mistral-Nemo-Instruct-2407",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Grounding It Properly

The configuration the 31.2% on NaturalQuestions argues for.

GROUNDED = (
    "Answer only from the material provided below. Quote the passage supporting each "
    "statement. Where the material does not contain the answer, say so plainly and stop — "
    "do not fill the gap from general knowledge. 'The source does not address this' is a "
    "complete and correct answer."
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Nemo-Instruct-2407",
    messages=[
        {"role": "system", "content": GROUNDED},
        {"role": "user", "content": f"{documents}\n\nQuestion: {question}"},
    ],
    max_tokens=4096,
    temperature=0.2,
)

The instruction against filling gaps is the operative one. A model that scores 31.2% on open-domain recall will produce a plausible answer from memory if you let it, and that answer will read exactly like a grounded one.

Permitting "the source does not address this" turns a weakness into a boundary. Without that permission, the model has no acceptable way to say it does not know.

Multilingual Work

SYSTEM = (
    "Respond in the same language as the user's message. Use the register and conventions "
    "of that language rather than translating English phrasing."
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Nemo-Instruct-2407",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Bonjour, ma commande n'est toujours pas arrivée. Que dois-je faire ?"},
    ],
    max_tokens=1024,
)

Nine languages were part of training rather than incidental, and the MMLU spread — 59% to 64.6% against 68% English — is narrow enough that a French or Spanish product does not need a different model.

Instructing against translated phrasing is worth the line. A model can answer correctly in French while sounding like English rendered into French, and that difference is what a native reader notices first.

Structured Extraction

import json
import re

response = client.chat.completions.create(
    model="mistralai/Mistral-Nemo-Instruct-2407",
    messages=[
        {
            "role": "system",
            "content": (
                "Return a single JSON object with keys: order_id, issue_type, amount, currency. "
                "Use null for anything the message does not state. Reply with JSON only, no prose."
            ),
        },
        {"role": "user", "content": ticket},
    ],
    max_tokens=1024,
    temperature=0.1,
)

raw = response.choices[0].message.content
cleaned = re.sub(r"^```(?:json)?|```$", "", raw, flags=re.MULTILINE).strip()

try:
    data = json.loads(cleaned)
except json.JSONDecodeError as exc:
    raise ValueError(f"model did not return parseable JSON: {cleaned[:400]}") from exc

Low temperature for a task with one correct answer, and a parse that raises rather than defaulting to an empty object. On a pipeline running at volume, a silent {} is a wrong answer wearing the costume of no answer.

Long-Document Work

Where the trained 128k window earns its place.

from pathlib import Path

contract = Path("supplier_agreement.txt").read_text(encoding="utf-8")

response = client.chat.completions.create(
    model="mistralai/Mistral-Nemo-Instruct-2407",
    messages=[
        {
            "role": "system",
            "content": (
                "Work only from the document supplied. Quote the clause behind every statement. "
                "Where the document does not address something you would expect it to, name the "
                "gap instead of filling it."
            ),
        },
        {"role": "user", "content": f"{contract}\n\nWhat happens to our obligations if the vendor is acquired?"},
    ],
    max_tokens=8192,
    temperature=0.3,
)

Naming gaps rather than filling them is the instruction that earns its place on document work. The absence of a clause is frequently the finding.

Three Inference Frameworks

Unusual breadth, and the joint development explains it.

mistral_inference — Mistral's own, and the recommended path for this model.

transformers — the standard route, with code snippets in the model card.

NVIDIA NeMo — published separately under NVIDIA's own identifier, a direct consequence of the models having been trained together.

Through an API this is handled for you. It matters for self-hosting, and the NeMo path is worth knowing about if you are already on NVIDIA's stack.

Where It Fits

Upgrading an existing Mistral 7B integration, where drop-in compatibility means no migration work at all.

Long-document processing, with a window trained rather than extrapolated.

Multilingual products across nine languages with a narrow quality gap.

Interactive assistants, where no reasoning pass means predictable latency and a straightforward token budget.

Grounded retrieval systems, where the model reads rather than recalls — which is where it performs best relative to its size.

High-volume production work under Apache 2.0 with no licensing conditions.

Not as a knowledge source. 31.2% on open-domain factual recall is the number to remember.

Not for deep multi-step reasoning. There is no thinking mode, and the reasoning models elsewhere in this catalogue are built for that.

Not for vision. Text only.

Practical Notes

Ground factual work. This model reads far better than it recalls.

Permit "I don't know" explicitly — otherwise the model has no acceptable way to express uncertainty.

Lower the temperature for extraction and classification.

Size max_tokens to the answer; nothing else consumes it.

Instruct against translated phrasing on multilingual output.

Use the full 128k window with confidence — it was trained, not scaled.

Consider the NeMo path if you self-host on NVIDIA's stack.

Limitations

Weak open-domain factual recall. 31.2% on NaturalQuestions. Ground anything factual.

TruthfulQA at 50.3%. Common misconceptions are a real failure mode; verify plausible-sounding claims on topics where one exists.

No reasoning capability. Multi-step logic and complex analysis belong on a model built for them.

Text only. No image, audio, or video input, and no image generation.

Nine documented languages. Others are outside the stated coverage.

A July 2024 model. Mature, widely supported, and stable — and the field has moved considerably since. Choose it for reliability and licence, not for frontier capability.

Multilingual performance trails English by five to nine points on MMLU. Narrow, and not zero.

Confident answers with no visible reasoning. There is less signal about where the model was uncertain, which makes grounding and citation requirements more important rather than less.