Mistral-Small-24B-Instruct-2501

Mistral Small 3 (2501)

Twenty-four billion parameters, a 32k window, and a model card that tells you how to prompt it.

Knowledge-Dense

Mistral's own word for what this model is, and it names the design goal precisely.

It fits on a single RTX 4090, or a 32GB MacBook, once quantized.

Knowledge density is a different objective from capability. A larger model knows more in absolute terms; this one was built to know as much as possible per gigabyte — which is what decides whether it runs on hardware you already own.

The stated use cases follow directly from that:

Fast-response conversational agents — where latency is the product. Low-latency function calling — where each tool call is a round trip. Subject-matter experts via fine-tuning — where you specialise a small model rather than prompt a large one. Local inference for hobbyists and organisations handling sensitive data — where the model has to run inside infrastructure the data cannot leave.

That last one is the case open weights exist for, and it is not available on a closed model at any price.

The System Prompt Is the Control Surface

The model card lists strong adherence and support for system prompts as a key feature — and then does something unusual: it publishes an example and explains what each part is for.

Two instructions in it are worth copying directly.

It tells the model what it does not know

Your knowledge base was last updated on 2023-10-01.
The current date is 2025-01-30.
When you're not sure about some information, you say that you don't have
the information and don't make up anything.

That first line is the knowledge cutoff, stated in the prompt. Mistral's approach is to tell the model what it does not know rather than hope it works it out — and a model with a fixed cutoff and no sense of today's date will reason about "recent" and "currently" against the wrong anchor, with full confidence.

Inject the current date. One line, computed at request time, and it removes a category of quiet error.

And permit "I don't have that information" explicitly. A model with no acceptable way to express uncertainty will produce something plausible instead. This is the instruction that gives it an alternative.

It tells the model to ask rather than guess

The card's own examples:

"What are some good restaurants around me?" → "Where are you?"

"When is the next flight to Tokyo" → "Where do you travel from?"

Clarification instead of a confident answer to an unanswerable question.

Why this belongs in your system prompt rather than in your product design. Users write ambiguous questions constantly. Without this instruction, the model resolves the ambiguity itself — picking a city, assuming a departure point — and answers a question nobody asked. With it, the ambiguity comes back to the person who can resolve it.

Specifications

Languages: English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Russian, Korean.

The Tekken tokenizer with a 131k vocabulary is the multilingual half of the design. A larger vocabulary means fewer tokens per word in non-English text — which costs less context and preserves more meaning per token. On a model with a 32k window, that efficiency is not a detail.

Capabilities

Text only. This is the first generation in the line; vision arrived in the release after it, alongside a 128k window.

The Benchmarks Came With a Caveat

Mistral published two kinds of evaluation, and they were unusually candid about the relationship between them.

A third-party human evaluation. Over a thousand proprietary coding and generalist prompts, with external evaluators picking their preferred response from anonymised generations.

And the admission printed alongside it:

We are aware that in some cases the benchmarks on human judgement starkly differ from publicly available benchmarks.

They say so, then defend the methodology — extra caution in verifying a fair evaluation, and confidence that the results are valid.

Why that admission is worth more than the numbers. Human preference and automated benchmarks measure different things. A model can score well on multiple-choice questions and produce answers people dislike, or the reverse. Publishing the divergence rather than the favourable half is the honest presentation.

One further note from the card: all benchmark accuracy came through the same internal evaluation pipeline, so figures may vary slightly from other published sources. That is normal, stated, and a reason to measure on your own prompts rather than reconcile two tables.

Using Mistral Small 3 on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: mistralai/Mistral-Small-24B-Instruct-2501

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-24B-Instruct-2501",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "mistralai/Mistral-Small-24B-Instruct-2501",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistralai/Mistral-Small-24B-Instruct-2501",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

The System Prompt Mistral Recommends

Built from the card's own example, with the date computed at request time.

from datetime import date

SYSTEM = f"""You are a helpful assistant.

Your knowledge base was last updated on 2023-10-01. The current date is {date.today().isoformat()}.
When you're not sure about some information, you say that you don't have the information and don't
make up anything.

If the user's question is not clear, ambiguous, or does not provide enough context for you to
accurately answer the question, you do not try to answer it right away and you rather ask the user
to clarify their request."""

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-24B-Instruct-2501",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": user_message},
    ],
    max_tokens=2048,
)

Three instructions doing three separate jobs.

The cutoff and the current date stop the model reasoning about "recently" against a two-year-old anchor.

Permission to say "I don't have that" gives it an acceptable alternative to invention.

The clarification instruction sends ambiguity back to the person who can resolve it instead of having the model guess.

Adapt the first line for your domain — a support assistant should also be told what it can and cannot commit to on the company's behalf.

Grounding Against a 2023 Cutoff

The constraint that decides how you use this model for anything factual.

GROUNDED = f"""Answer only from the material provided below. Quote the passage supporting each
statement. Where the material does not contain the answer, say so plainly and stop — do not fill the
gap from general knowledge.

Your training data ends in October 2023. The current date is {date.today().isoformat()}. Treat
anything you recall about events, versions, prices, or people as potentially out of date."""

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-24B-Instruct-2501",
    messages=[
        {"role": "system", "content": GROUNDED},
        {"role": "user", "content": f"{documents}\n\nQuestion: {question}"},
    ],
    max_tokens=4096,
    temperature=0.2,
)

A cutoff of October 2023 is a long way back. Library versions, API changes, product names, regulations, and people in roles have all moved since — and the model will answer about them confidently.

Retrieval is the fix. Documents in the prompt move the model from recalling to reading, and reading is where a 24-billion-parameter model performs closest to a much larger one.

Low-Latency Function Calling

One of the three named use cases, and the configuration that suits it.

import json

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "check_stock",
            "description": "Return current stock level and warehouse location for a SKU.",
            "parameters": {
                "type": "object",
                "properties": {"sku": {"type": "string"}},
                "required": ["sku"],
            },
        },
    },
]


def check_stock(sku: str) -> dict:
    """Replace with your real inventory system."""
    raise NotImplementedError


HANDLERS = {"check_stock": check_stock}

thread = [
    {
        "role": "system",
        "content": (
            "You are an inventory assistant. Look up facts with the tools provided rather than "
            "assuming them. If the user's request does not identify a specific product, ask which "
            "one before calling anything."
        ),
    },
    {"role": "user", "content": "Is the black one in stock in Algiers?"},
]

CEILING = 8

for step in range(CEILING):
    response = client.chat.completions.create(
        model="mistralai/Mistral-Small-24B-Instruct-2501",
        messages=thread,
        tools=TOOLS,
        max_tokens=2048,
    )

    message = response.choices[0].message
    thread.append(message)

    if not message.tool_calls:
        print(message.content)
        break

    for call in message.tool_calls:
        handler = HANDLERS.get(call.function.name)
        if handler is None:
            outcome = {"error": "unknown tool", "name": call.function.name}
        else:
            try:
                outcome = handler(**json.loads(call.function.arguments or "{}"))
            except Exception as exc:
                outcome = {"error": type(exc).__name__, "detail": str(exc)}

        thread.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(outcome)})
else:
    print(f"Stopped at the {CEILING}-step ceiling.")

The user message is deliberately ambiguous — "the black one" identifies nothing. The system prompt instructs the model to ask rather than pick a SKU, which is the behaviour the card's own example was demonstrating.

Without that instruction, a model this instruction-responsive will choose a product and look it up. The tool call succeeds, the answer is confident, and it is about the wrong item.

Validate arguments before dispatch regardless. A required field arriving empty should fail in your validation rather than in your inventory system.

Structured Extraction

import json
import re

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-24B-Instruct-2501",
    messages=[
        {
            "role": "system",
            "content": (
                "Return a single JSON object with keys: order_id, issue_type, amount, currency. "
                "Use null for anything the message does not state. Reply with JSON only, no prose."
            ),
        },
        {"role": "user", "content": ticket},
    ],
    max_tokens=1024,
    temperature=0.1,
)

raw = response.choices[0].message.content
cleaned = re.sub(r"^```(?:json)?|```$", "", raw, flags=re.MULTILINE).strip()

try:
    data = json.loads(cleaned)
except json.JSONDecodeError as exc:
    raise ValueError(f"model did not return parseable JSON: {cleaned[:400]}") from exc

Low temperature for a task with one correct answer, and a parse that raises rather than defaulting to an empty object. On a pipeline running at volume, a silent {} is a wrong answer wearing the costume of no answer.

Fine-Tuning as a Subject-Matter Expert

Named as one of the three primary use cases, and the reason the licence matters.

Prompting adapts a model without training it, and costs input tokens on every call. A long system prompt with examples, repeated across a million requests, is a permanent tax. Fine-tuning moves that behaviour into the weights — free thereafter, with the prompt shrinking to the actual input.

Worth it when the task is narrow, its definition is stable, you have a few hundred to a few thousand good examples, and volume is high enough that per-call savings compound.

Not worth it when the definition changes often, the examples are few, or a larger model simply solves the problem.

At 24 billion parameters under Apache 2.0, this sits in the range where fine-tuning is affordable on accessible hardware — and where the resulting model is yours to deploy commercially without conditions.

Where It Fits

Local and on-device deployment — a single 4090 or a 32GB Mac, quantized, which is the design point rather than a stretch.

Air-gapped and sensitive-data environments, named directly in the card's intended use.

Fast conversational agents, where latency and consistency outrank depth.

Low-latency tool calling, where each call is a round trip and the model's speed is the product.

Fine-tuned specialisation, with Apache 2.0 removing the licensing question entirely.

Grounded retrieval systems, where the model reads rather than recalls.

Not for long documents. 32k is the first-generation window; later releases in this line reach four times further.

Not for images. Text only — vision arrived in the next release.

Not as a knowledge source. October 2023 is a long way back.

Practical Notes

Always send a system prompt. The card treats it as the main control surface, and publishes an example worth copying.

Inject today's date. One computed line, and it removes a class of quiet error.

Permit "I don't have that information" explicitly.

Instruct the model to ask for clarification on ambiguous requests rather than resolving them itself.

Ground anything factual. The cutoff is October 2023.

Lower the temperature for extraction and classification.

Size max_tokens to the answer — nothing else consumes it.

Validate tool-call arguments before dispatch.

Limitations

32k context. The first-generation window; later releases in this line reach 128k.

Text only. No image input — vision came in the following release.

Knowledge ends October 2023. A long way back, and the model will answer about later events with full confidence.

No reasoning capability. Multi-step logic and complex analysis belong on a model built for them.

Ten documented languages. Others are outside the stated coverage.

Benchmark figures came through Mistral's own evaluation pipeline and may differ from other published sources — stated on the card.

Human and automated evaluations diverge, which Mistral acknowledge directly. Measure on your own prompts rather than reconciling two tables.

A base variant exists under a nearly identical name and does not follow instructions.

Confident answers with no visible reasoning. There is less signal about where the model was uncertain — which makes the system prompt's uncertainty and clarification instructions more important rather than less.