Qwen2.5-72B-Instruct

Qwen2.5-72B-Instruct

Eighteen trillion training tokens, twenty-nine languages, and a claim about system prompts that most models do not make.

One Budget, Shared

The number that shapes every request, and it is a total rather than a limit on the input.

Input and output draw on the same 32,768. A prompt of 30,000 tokens leaves 2,768 for the answer, whatever max_tokens says.

And 8,192 is a ceiling on generation, not an allocation. You never get more than that in one response; you frequently get less, because the input took the room.

Which makes one calculation worth doing before every long request:

MAX_CONTEXT = 32_768
GENERATION_CEILING = 8_192
SAFETY_MARGIN = 512


def output_budget(estimated_input_tokens: int) -> int:
    """How many output tokens are actually available for this request."""
    remaining = MAX_CONTEXT - estimated_input_tokens - SAFETY_MARGIN

    if remaining <= 0:
        raise ValueError(
            f"input of ~{estimated_input_tokens:,} tokens leaves no room for a response "
            f"within {MAX_CONTEXT:,}"
        )

    return min(GENERATION_CEILING, remaining)

Leave a margin. Token estimates are estimates, and a request computing to exactly 32,768 is a request that fails.

The practical shape that follows: long input and long output are mutually exclusive here. A 24,000-token document can produce a 6,000-token analysis. A 6,000-token brief can produce an 8,000-token document. Both in one request cannot happen.

Plan which half of the budget your workload needs, and structure around it rather than discovering the constraint at the point where a response comes back truncated.

Where the Coding and Maths Came From

Qwen name the source rather than describing an outcome:

Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains.

Separate expert models, distilled into this one.

That is a different claim from "trained on more code." A specialist model trained on mathematics alone reaches places a general model does not, and transferring that into a general model is a distinct exercise from adding data to a general training run.

What it means for you. The coding and mathematics capability here has a lineage — it came from models built for those tasks, rather than emerging from scale.

Resilient to the Diversity of System Prompts

The claim worth reading twice, because it describes a failure mode most cards do not acknowledge:

More resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots.

What the failure looks like. A model tuned on a narrow range of system prompt shapes behaves well on prompts resembling its training and degrades on ones that do not — a very long system prompt, an unusually structured one, one written in a second language, one defining a persona rather than a task.

Resilience to that diversity is a property you notice only when you stop having the problem.

Where it matters most:

Multi-tenant applications, where each customer writes their own system prompt and none of them writes it the way you would.

Role-play and persona work, which Qwen name specifically — a character definition is a structurally different system prompt from a task instruction.

Condition-setting for chatbots — complex conditional behaviour expressed in the system turn rather than in code.

The practical consequence: you can write a long, unusual, domain-specific system prompt and expect it to be followed, rather than tuning its shape to what the model likes.

One caution that follows from the budget above. A long system prompt is input, and it is charged against the same 32,768 as everything else. On a multi-tenant deployment where customers write their own, that is worth measuring rather than assuming.

⚠️ 32,768 Is Native

Worth stating, because you are on the better side of a distinction that matters.

The model's native length is 32,768 tokens. That is what it was trained at.

A 131,072-token configuration exists through YaRN scaling, and Qwen attach a caveat to it: most serving frameworks implement static YaRN, where the scaling factor remains constant regardless of input length — which potentially impacts performance on shorter texts.

Qwen's own advice is to enable it only when processing long contexts is required.

At 32,768 you are running the model as trained. No scaling factor, no degradation on short prompts, no trade-off to manage — the window is the one the model learned.

Which is the right position for a mixed workload. A deployment that enables long context permanently to handle occasional long documents pays for it on every short request in between, and short requests are usually the majority.

Architecture

The QKV bias is the detail worth noticing. Most current architectures drop bias terms from the attention projections — they add parameters and are usually found not to help.

Qwen2.5 keeps them, and lists it as an architectural property rather than an omission. It is a deliberate choice in a generation where the convention went the other way.

Specifications

Languages include Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.

The licence is the family's exception. Qwen2.5 ships at seven sizes — 0.5B through 72B — and the smaller ones are permissively licensed. This one is not. Read the Tongyi Qianwen agreement against your deployment rather than assuming the family's usual terms.

Transformers below 4.37.0 raises an error.

Official quantisations include AWQ and GPTQ builds published by Qwen alongside the standard weights.

Capabilities

Structured Data, Both Directions

An improvement Qwen list in two halves, and they are separate capabilities.

Understanding structured data — tables specifically named. Reading a table and reasoning about it is a different skill from reading prose, and a model that flattens a table into a sentence has lost the structure that made it useful.

Generating structured outputs, especially JSON — named with that emphasis.

Both matter in a pipeline, and they are frequently the same pipeline: a table comes in, a JSON object goes out, and the model is the transformation between them.

And the shared budget shapes that pipeline. A large table consumes input; the JSON it produces consumes output. On a wide table with many rows, the two together reach 32,768 sooner than either alone suggests — which is why batching by row count rather than by file is the right unit.

It Answers Directly

No thinking mode, no effort parameter, no reasoning_content field.

max_tokens covers the answer alone — nothing shares it except the input. On a reasoning model part of that budget would go to a trace you did not ask for; here every token is answer.

Latency tracks input and output length, not how hard the model judged the question. On an interactive endpoint that predictability is frequently worth more than depth.

Using Qwen2.5-72B-Instruct on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: Qwen/Qwen2.5-72B-Instruct

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-72B-Instruct",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "Qwen/Qwen2.5-72B-Instruct",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-72B-Instruct",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Sizing a Request

The habit worth building into any path that handles variable-length input.

def ask(system: str, user: str, *, want: int = 4096) -> str:
    """Send a request with an output budget computed from what the input leaves."""
    # A rough estimate is enough; the safety margin absorbs the error.
    estimated_input = (len(system) + len(user)) // 3

    budget = output_budget(estimated_input)

    if budget < want:
        print(f"warning: wanted {want:,} output tokens, {budget:,} available")

    response = client.chat.completions.create(
        model="Qwen/Qwen2.5-72B-Instruct",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": user},
        ],
        max_tokens=min(want, budget),
        temperature=0.4,
    )

    choice = response.choices[0]

    if choice.finish_reason == "length":
        raise ValueError(
            f"truncated at {response.usage.completion_tokens:,} tokens — "
            f"input consumed {response.usage.prompt_tokens:,} of {MAX_CONTEXT:,}"
        )

    return choice.message.content

The warning line is the useful part. It tells you when the input has quietly eaten the budget you planned for the answer — which is the failure you would otherwise diagnose from a truncated response three weeks later.

And the exception carries both numbers. Knowing that the input took 29,000 tokens turns "the answer was cut off" into "the document was too long", which is a different fix.

An Unconventional System Prompt

The capability Qwen claim, used deliberately rather than avoided.

SYSTEM = """<role>
You are Karim, a support agent for an Algerian electronics retailer. You have worked there four
years. You are direct, mildly impatient with vague questions, and genuinely good at your job.
</role>

<knowledge>
You know the product catalogue and the returns policy. You do NOT know: stock levels, individual
order status, or anything about a specific customer's account. For those you use the tools provided.
</knowledge>

<rules>
- Never invent a delivery date, a price, or a stock figure.
- Never promise a refund or an exception. Escalate instead.
- If a customer is angry, acknowledge it once and move to the problem. Do not apologise repeatedly.
</rules>

<style>
Reply in the customer's language. Two to four sentences. No lists unless they asked for steps.
</style>"""

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-72B-Instruct",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": message},
    ],
    max_tokens=1024,
    temperature=0.5,
)

That prompt is structurally unusual — XML-ish sections, a persona with a history, a negative knowledge boundary, and a behavioural rule about apologising. On a model tuned narrowly, a shape like that produces inconsistent adherence.

Qwen's resilience claim is precisely about this. Write the prompt the way your application needs it rather than the way you guess the model prefers, and test whether it holds.

The negative knowledge boundary is the most valuable part. Telling a model what it does not know is more effective than telling it what it does, because the failure mode you are preventing is confident invention.

And keep it proportionate. A three-thousand-token system prompt on every turn is three thousand tokens of a 32,768 budget, before the conversation starts.

Tables In, JSON Out

The pipeline both structured-data improvements serve.

import json

SYSTEM = """You extract structured data from tables.

Return a single JSON array. Each element has keys: sku, description, quantity, unit_price, currency.

Rules:
- Use null for any cell that is empty or unreadable. Never infer a value from neighbouring rows.
- Preserve the currency exactly as written, including the symbol or code.
- If the table has merged cells or a structure you cannot resolve unambiguously, return an object
  with a single key "error" describing what was ambiguous.
- Reply with JSON only, no prose."""


def extract(table_text: str) -> list | dict:
    """Extract a table to JSON, sized against the shared budget."""
    estimated_input = (len(SYSTEM) + len(table_text)) // 3
    budget = output_budget(estimated_input)

    response = client.chat.completions.create(
        model="Qwen/Qwen2.5-72B-Instruct",
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": table_text},
        ],
        max_tokens=budget,
        temperature=0.1,
    )

    choice = response.choices[0]

    if choice.finish_reason == "length":
        raise ValueError("output truncated — split the table into fewer rows per request")

    raw = choice.message.content

    try:
        data = json.loads(raw)
    except json.JSONDecodeError as exc:
        raise ValueError(f"model did not return parseable JSON: {raw[:400]}") from exc

    if isinstance(data, dict) and "error" in data:
        raise ValueError(f"model could not resolve the table: {data['error']}")

    return data

JSON output is roughly as long as the table that produced it, sometimes longer once keys are repeated per row. On a shared budget that means a table filling half the window cannot produce a complete extraction — split by rows, not by file.

The ambiguity escape hatch is worth copying. A merged cell or an unclear header has no correct extraction, and a model with no way to say so will produce a plausible one instead.

Low temperature, because extraction has one correct answer and variance is noise you then have to reconcile.

Long-Form Generation

The workload Qwen tuned the generation ceiling for — with the budget constraint that shapes it.

from pathlib import Path

SECTIONS = ["Executive summary", "Findings", "Risk assessment", "Recommendations"]


def write_section(name: str, brief: str) -> str:
    """Generate one section, leaving room in the shared budget for the output."""
    prompt = f"Brief:\n\n{brief}\n\nWrite the '{name}' section."
    budget = output_budget((len(prompt)) // 3)

    response = client.chat.completions.create(
        model="Qwen/Qwen2.5-72B-Instruct",
        messages=[
            {
                "role": "system",
                "content": (
                    "Write only the section requested. Do not summarise other sections, add a "
                    "conclusion, or restate the brief."
                ),
            },
            {"role": "user", "content": prompt},
        ],
        max_tokens=budget,
        temperature=0.4,
    )

    choice = response.choices[0]

    if choice.finish_reason == "length":
        raise ValueError(f"section '{name}' hit its budget — narrow the brief or split the section")

    return f"## {name}\n\n{choice.message.content}"


brief = Path("brief.txt").read_text(encoding="utf-8")
document = "\n\n".join(write_section(name, brief) for name in SECTIONS)
Path("report.md").write_text(document, encoding="utf-8")

Qwen list generating long texts over 8K tokens among the improvements, so the ceiling is what the model was trained to fill rather than a limit it trails off before reaching.

The brief goes into every request, and it is charged every time. A short brief makes this comfortable; a twenty-thousand-token source does not — for that, summarise the source once and pass the summary to each section.

That trade is the whole design constraint. Full context per section, or long sections. Not both.

Multilingual Work

Twenty-nine languages, Arabic among them.

SYSTEM = (
    "Respond in the same language as the user's message, using that language's own register and "
    "conventions rather than translating English phrasing."
)

The instruction against translated phrasing earns its line. A model can answer correctly in Arabic or French while sounding like English rendered into it, and that difference is what a native reader notices before anything else.

And budget for it. Non-Latin scripts frequently tokenise less efficiently than English, so the same content in Arabic consumes more of the 32,768 than its English equivalent — which is worth measuring on your own text rather than estimating.

Where It Sits in the Family

Qwen2.5 spans seven sizes: 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B.

This is the top of the range, and two things distinguish it beyond capability.

The licence. The smaller sizes are permissively licensed; this one carries Qwen's own agreement.

The footprint. Seventy-two billion parameters is a multi-card deployment at full precision, and Qwen publish AWQ and GPTQ builds specifically to bring it down.

Which makes the comparison worth running. The 32B shares the architecture, the generation ceiling, and the language coverage. Twenty of your real prompts through both will tell you whether the gap justifies the difference — and on many workloads it does not.

Practical Notes

Compute the output budget from what the input leaves. 32,768 is a total.

Leave a safety margin — token estimates are estimates.

Keep system prompts proportionate; they are charged on every turn.

Write the system prompt your application needs, not the shape you guess the model prefers.

Tell the model what it does not know, explicitly.

Split extraction by rows rather than by file — JSON output is as long as the table.

Check finish_reason, and log both token counts when it fires.

Lower the temperature for extraction and classification.

Measure token consumption on non-Latin scripts rather than estimating from English.

Compare against the 32B on your own prompts before committing to the footprint.

Read the Tongyi Qianwen License — this size is the family's exception.

Limitations

32,768 tokens covers input and output together. Long input and long output compete for the same budget; you cannot have both in one request.

8,192-token generation ceiling, and you frequently get less because the input took the room.

No reasoning mode. Multi-step logic and complex analysis belong on a model built for them.

Text only. No image, audio, or video input.

A custom licence, unlike the smaller models in this family.

Transformers 4.37.0 or later is required for self-hosting.

Seventy-two billion parameters at full precision is a multi-card deployment; quantised builds are published for that reason.

A September 2024 model. Mature, widely deployed, well understood, and the field has moved considerably since. Choose it for stability, multilingual coverage, and predictable behaviour — not for frontier capability.

Confident answers with no visible reasoning. There is less signal about where the model was uncertain, which makes explicit permission to say "I don't know" more important rather than less.