Mistral-Small-3.2-24B-Instruct-2506

Mistral Small 3.2 24B Instruct

A minor update that fixed three things. One of them is measured, and it is the one that matters most in production.

Infinite Generations, Halved

The headline number, and it describes a failure mode rather than a capability.

Small 3.2 reduces infinite generations by 2× on challenging, long and repetitive prompts.

What an infinite generation is. The model starts repeating — a sentence, a list item, a phrase — and does not stop until it hits the token ceiling. The response is unusable, the tokens are billed, and the request took as long as the ceiling allowed.

Why it is worse than a wrong answer. A wrong answer fails visibly. A repetition loop consumes the full output budget, adds maximum latency, and produces something that looks like output until someone reads past the first paragraph.

Halving it is a reliability improvement, not a quality one — and on a model running at volume, reliability is the number that decides whether a pipeline needs a retry layer.

Temperature 0.15

Mistral's explicit recommendation, and unusually low.

Most models suggest 0.7 for general use. This card recommends 0.15.

The connection to the repetition fix is direct. Sampling variance is one of the mechanisms that sends a model into a loop — a slightly wrong token choice leads to a slightly wrong continuation, and the model settles into a pattern it cannot leave. Low temperature narrows the sampling distribution and removes much of that path.

Take the recommendation seriously rather than treating it as a default to override. A value carried across from another model reintroduces the behaviour this release was tuned to reduce.

The Function-Calling Template Was Rebuilt

The third named improvement, and the one with the least visible surface.

The function-calling template is more robust — a change to how tool definitions and calls are formatted, not to how the model reasons about them.

What a fragile template produces: tool calls that arrive as prose, malformed argument JSON, calls to functions that were not offered, and arguments missing required fields. All of them fail in your code rather than in the model's.

If tool calling was unreliable on the previous release, this is the fix.

Through an API the template is applied for you. It matters when self-hosting, where the correct tool-call parser has to be configured explicitly.

Use a System Prompt

Stated directly on the model card, with unusual insistence: make sure to add a system prompt to best tailor the model to your needs.

Mistral ship one. A SYSTEM_PROMPT.txt file lives in the repository, recommended for general assistant use.

Why the card bothers to say it. This model responds strongly to instruction — "will follow your instructions down to the last letter" is the card's own phrasing. A model that precise with no system prompt is precise about nothing in particular.

Write your own for a specific task. Role, constraints, output format, and what to do when the answer is not available. That last one matters more than it looks: a model with no permitted way to say "I don't know" will invent something instead.

Specifications

Languages: English, French, German, Spanish, Portuguese, Italian, Japanese, Korean, Russian, Chinese, Arabic, Persian, Indonesian, Malay, Nepali, Polish, Romanian, Serbian, Swedish, Turkish, Ukrainian, Vietnamese, Hindi, Bengali.

Arabic is in that list — and it is one of relatively few open models at this size where it was part of training rather than incidental.

A base variant exists under a nearly identical name. Pre-trained only; it does not follow instructions.

Capabilities

Chat completions only. The model is served through the chat completions interface; a responses-style endpoint is not supported for it.

It Answers Directly

No thinking mode, no effort parameter, no reasoning_content field.

max_tokens covers the answer alone — nothing shares it, nothing is consumed by a trace you did not request. On a reasoning model a budget of 512 can produce nothing; here it produces five hundred tokens of answer.

Latency tracks input and output length, not how hard the model judged the question. On an interactive endpoint that predictability is frequently worth more than depth.

Using Mistral Small 3.2 on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: mistralai/Mistral-Small-3.2-24B-Instruct-2506

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "mistralai/Mistral-Small-3.2-24B-Instruct-2506",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistralai/Mistral-Small-3.2-24B-Instruct-2506",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

At Mistral's Recommended Settings

A system prompt and a low temperature — the two things the card asks for.

SYSTEM = (
    "You are a support assistant for an Algerian e-commerce platform. Answer in the language "
    "the customer writes in, using that language's own register rather than translated English "
    "phrasing. Where you do not have the information needed, say so plainly and offer to escalate "
    "— do not guess at policy, prices, or delivery times."
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": message},
    ],
    temperature=0.15,
    max_tokens=2048,
)

Both settings do work. The system prompt is what the card insists on; 0.15 is what it recommends.

The permission to say "I don't have that" is the load-bearing clause. A model described as following instructions down to the last letter will follow an instruction to answer — even when it has nothing to answer with. Giving it an acceptable alternative is how you get that behaviour instead.

Watching for Truncation

On a model whose headline fix is repetition, the check that confirms it is worth having.

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
    messages=messages,
    temperature=0.15,
    max_tokens=4096,
)

choice = response.choices[0]

if choice.finish_reason == "length":
    logger.warning(
        "hit the output ceiling: %d tokens — check for repetition",
        response.usage.completion_tokens,
    )

print(choice.message.content)

finish_reason == "length" on a task that should have finished is the signal for a repetition loop. The fix halved the rate; it did not remove it.

Log the case rather than retrying blindly. If it recurs on the same prompt, the prompt is the problem — long, repetitive input is exactly the condition the card names.

Reading an Image

Vision is a Pixtral-style encoder, and the model handles interleaved text and images natively.

import base64

with open("invoice.png", "rb") as handle:
    encoded = base64.b64encode(handle.read()).decode("utf-8")

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}},
                {
                    "type": "text",
                    "text": (
                        "List every line item with its quantity and amount, and report the currency "
                        "exactly as printed. Mark anything you cannot read cleanly as unreadable "
                        "rather than reconstructing it."
                    ),
                },
            ],
        }
    ],
    temperature=0.15,
    max_tokens=4096,
)

Send pages at full resolution. Downscaling before upload discards detail the encoder would have used.

The number of images per prompt is set at serving time, so a request with many images may be rejected depending on how the model is deployed. Test with your real image count before building a batch path.

Structured Output

response_format accepts a JSON schema, which is the more reliable route than asking for JSON in prose.

import json

SCHEMA = {
    "type": "json_schema",
    "json_schema": {
        "name": "order_issue",
        "strict": True,
        "schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": ["string", "null"]},
                "issue_type": {
                    "type": "string",
                    "enum": ["not_delivered", "damaged", "wrong_item", "late", "other"],
                },
                "amount": {"type": ["number", "null"]},
                "currency": {"type": ["string", "null"]},
            },
            "required": ["order_id", "issue_type", "amount", "currency"],
            "additionalProperties": False,
        },
    },
}


def extract(message: str) -> dict:
    """Pull typed fields from a customer message."""
    response = client.chat.completions.create(
        model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
        messages=[
            {
                "role": "system",
                "content": "Extract only what the message states. Use null for anything absent — never infer.",
            },
            {"role": "user", "content": message},
        ],
        response_format=SCHEMA,
        temperature=0.15,
        max_tokens=512,
    )
    return json.loads(response.choices[0].message.content)

Every field nullable, and an explicit instruction against inference. A schema forbidding null invites the model to produce a value where the source has none — and on a pipeline running at volume, that error is invisible until someone acts on it.

Tool Calling

The template was rebuilt in this release, which makes this the version to use if tool calls were unreliable before.

import json

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "lookup_order",
            "description": "Return order status, line items, and delivery events for an order ID.",
            "parameters": {
                "type": "object",
                "properties": {"order_id": {"type": "string"}},
                "required": ["order_id"],
            },
        },
    },
]


def lookup_order(order_id: str) -> dict:
    """Replace with your real data access layer."""
    raise NotImplementedError


HANDLERS = {"lookup_order": lookup_order}

thread = [
    {"role": "system", "content": "You are a support agent. Look up facts rather than assuming them."},
    {"role": "user", "content": "Order 48213 shows delivered but nothing arrived. What happened?"},
]

CEILING = 10

for step in range(CEILING):
    response = client.chat.completions.create(
        model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
        messages=thread,
        tools=TOOLS,
        temperature=0.15,
        max_tokens=4096,
    )

    message = response.choices[0].message
    thread.append(message)

    if not message.tool_calls:
        print(message.content)
        break

    for call in message.tool_calls:
        handler = HANDLERS.get(call.function.name)
        if handler is None:
            outcome = {"error": "unknown tool", "name": call.function.name}
        else:
            try:
                outcome = handler(**json.loads(call.function.arguments or "{}"))
            except Exception as exc:
                outcome = {"error": type(exc).__name__, "detail": str(exc)}

        thread.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(outcome)})
else:
    print(f"Stopped at the {CEILING}-step ceiling.")

Validate arguments before dispatch. A more robust template reduces malformed calls; it does not guarantee them, and a missing required field should fail in your validation rather than in your database layer.

The system prompt instruction to look things up rather than assume is worth including. A model this instruction-responsive will answer from its own guess if nothing tells it not to.

Multilingual Work

Twenty-four languages, including Arabic.

SYSTEM = (
    "Respond in the same language as the user's message, using that language's own register "
    "and conventions rather than translating English phrasing."
)

The instruction against translated phrasing earns its line. A model can answer correctly in French or Arabic while sounding like English rendered into it — and that difference is what a native reader notices before anything else.

Self-Hosting

~55 GB of GPU RAM in bf16 or fp16. Two cards at 40 GB, or one at 80 GB.

vLLM is the recommended framework, with Mistral's own tokenizer, config, and load formats specified rather than the defaults — and mistral_common 1.6.2 or later installed alongside.

The tool-call parser must be set to mistral, with automatic tool choice enabled. Omit it and tool calls arrive as prose.

Images per prompt are limited at launch, so the ceiling is a deployment decision rather than a model property.

Quantised builds are published by the community across GGUF, AWQ, and NVFP4 formats — the last targeting FP4 tensor cores on Blackwell hardware.

A ready-to-run Docker image exists if you would rather not assemble the configuration yourself.

Where It Fits

High-volume production traffic, where the repetition fix is a reliability improvement on every call.

Support and assistant products across twenty-four languages, with Arabic among them.

Document and image reading — invoices, forms, screenshots, charts — through the Pixtral-style encoder.

Tool-driven workflows, with a rebuilt calling template.

Structured extraction, with JSON schema support and a low recommended temperature.

Self-hosted deployment under Apache 2.0, on two cards or one large one.

Not for deep multi-step reasoning. There is no thinking mode, and the reasoning models elsewhere in this catalogue are built for it.

Not for image generation. It reads images; it does not produce them.

Practical Notes

Use temperature 0.15. It is the card's recommendation and it connects to the repetition fix.

Always send a system prompt. The card insists on it, and a ready-made one ships in the repository.

Give the model a permitted way to say it does not know.

Check finish_reason — a length stop on a short task is a repetition signal.

Send images at full resolution, and verify your image count against the serving limit.

Validate tool-call arguments before dispatch.

Instruct against translated phrasing on multilingual output.

Read the model suffix — a base variant exists that does not follow instructions.

Limitations

A minor update, and the card says so. Three named improvements; everything else matches or slightly improves on the previous release.

Repetition is reduced, not eliminated. Halved on the hardest prompts, which means it still happens.

No reasoning capability. Multi-step logic and complex analysis belong on a model built for them.

16,384-token output ceiling — roughly an eighth of the context window.

Chat completions only. A responses-style endpoint is not supported for this model.

Twenty-four documented languages. Others are outside the stated coverage.

Text output only. It reads images; it does not generate them.

Images per prompt are capped at serving time, not by the model — verify the limit on your path.

A base variant shares almost the same name and does not follow instructions.

Confident answers with no visible reasoning. There is less signal about where the model was uncertain, which makes grounding and explicit "I don't know" permission more important rather than less.