Qwen3-235B-A22B-Instruct-2507

Qwen3-235B-A22B-Instruct-2507

A dual-mode model, split in two. This is the half that answers directly.

The Split

The original Qwen3-235B-A22B did something unusual for its generation: one model, two modes, switchable per request between thinking and direct answering.

This release ends that. Qwen's own framing on the card is precise:

the updated version of the Qwen3-235B-A22B non-thinking mode

One half of a dual-mode model, improved on its own. And a sibling — Thinking-2507 — did the same for the other half, with an increased thinking length and a recommendation to use it for highly complex reasoning.

Why split a model that already did both. Optimising for two behaviours means compromising on each. Three months spent improving direct answering alone produces a better direct-answering model than three months spent improving both.

What you lose is the switch. A pipeline that used one identifier for fast and slow paths now needs two.

What you gain is a model that is not compromised toward a behaviour you were not using.

The Upgrade Does Not Break Anything

A detail stated plainly on the card, and it is unusually considerate:

specifying enable_thinking=False is no longer required

Not "rejected." Not "returns an error." No longer required.

Which means existing code upgrades by changing the model string. A request that still sets the parameter is accepted and the parameter is ignored — the behaviour it asked for is the only behaviour available.

That is a small decision with a large effect: nobody has to audit their request builders before migrating.

Architecture

Sixty-four query heads against four key-value heads — a 16:1 grouped-query ratio, cutting the key-value cache to a sixteenth of what full multi-head attention would need.

Ninety-four layers is deep. Combined with 22 billion active parameters, this is a tall, sparse stack rather than a wide one.

Eight experts of 128 — roughly six percent of the pool per token.

And 262,144 tokens is native, not an extension applied at inference. The card lists it as the context length rather than as a YaRN-extended figure, and the reference deployment commands configure it directly.

What Three Months Bought

The card names five improvements, and the list is more specific than most.

General capabilities — instruction following, logical reasoning, text comprehension, mathematics, science, coding, and tool usage.

Long-tail knowledge coverage across multiple languages. The obscure fact in the less-common language — precisely where models are weakest and where improvement is hardest to fake.

Alignment with user preferences on subjective and open-ended tasks, for more helpful responses and higher-quality text generation.

256K long-context understanding, named as enhanced rather than merely present.

Read the second and third together. Long-tail knowledge and open-ended quality are the two axes that benchmarks measure worst and users notice first. A release that leads with them is a release tuned for use rather than for a leaderboard.

Specifications

An official FP8 checkpoint is published, using fine-grained FP8 quantisation with a block size of 128 — details in the quantization_config field of the model configuration.

Community builds cover AWQ, GPTQ Int4-Int8 mixed, NVFP4, and GGUF.

Capabilities

No Reasoning Trace, and What That Simplifies

Three consequences, all of them in your favour on the right workload.

max_tokens covers the answer alone. Nothing shares it. On a thinking model a budget of 2,048 can produce a full trace and no answer; here it produces two thousand tokens of answer.

Latency tracks input and output length, not how hard the model judged the question. For an interactive endpoint that predictability is frequently worth more than depth.

There is no reasoning_content field to read, replay, or keep separate. One less thing for a multi-turn loop to get wrong.

And one consequence against you. A genuinely hard problem gets a direct answer rather than a worked one. The Thinking sibling exists for those, and it is a different model rather than a parameter.

Using Qwen3-235B-A22B-Instruct-2507 on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: Qwen/Qwen3-235B-A22B-Instruct-2507

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-235B-A22B-Instruct-2507",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "Qwen/Qwen3-235B-A22B-Instruct-2507",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-235B-A22B-Instruct-2507",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Routing Between the Two Halves

The pattern the split creates, and the one worth building deliberately.

INSTRUCT = "Qwen/Qwen3-235B-A22B-Instruct-2507"
THINKING = "Qwen/Qwen3-235B-A22B-Thinking-2507"


def ask(prompt: str, *, reason: bool = False, max_tokens: int | None = None) -> str:
    """Route to the direct or the reasoning half of the family."""
    model = THINKING if reason else INSTRUCT

    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=max_tokens or (32768 if reason else 4096),
    )

    choice = response.choices[0]

    if choice.finish_reason == "length":
        raise ValueError(f"{model} hit the ceiling at {response.usage.completion_tokens:,} tokens")

    return choice.message.content


# Direct work — extraction, summarising, generation, tool orchestration.
ask(f"Summarise this support thread for a handover:\n\n{thread}", max_tokens=1024)

# Genuinely hard work — the other half of the family.
ask("Prove whether this retry schedule can starve a single tenant under sustained load.", reason=True)

Note the different default ceilings. Four thousand for a direct answer; thirty-two thousand for a reasoning one, where a trace shares the budget.

Where the old model gave you a parameter, you now have a router. More code, and a cleaner separation — the fast path never risks accidentally triggering a reasoning pass, and the slow path is built for depth rather than tuned to also be fast.

Long-Context Work

262,144 tokens, native — which makes this straightforward rather than experimental.

from pathlib import Path

corpus = "\n\n---\n\n".join(
    f"### {path.name}\n{path.read_text(encoding='utf-8')}"
    for path in sorted(Path("agreements").glob("*.txt"))
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-235B-A22B-Instruct-2507",
    messages=[
        {
            "role": "system",
            "content": (
                "Review this set of agreements. Identify every obligation stated in one document "
                "that contradicts an obligation in another. Quote both clauses and name both files. "
                "Report nothing you cannot quote."
            ),
        },
        {"role": "user", "content": corpus},
    ],
    max_tokens=16384,
)

print(f"input: {response.usage.prompt_tokens:,} of 262,144")

Native context means no YaRN decision. No scaling factor to tune, no trade-off where enabling long context degrades short prompts — the window is the window.

Cross-document contradiction is what justifies it. A conflict between the third document and the eleventh is invisible to a pipeline that reads them one at a time.

Tool Orchestration

Named among the improved capabilities, and a natural fit for a direct-answering model.

import json

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "query_database",
            "description": "Run a read-only SQL query and return rows as JSON.",
            "parameters": {
                "type": "object",
                "properties": {"sql": {"type": "string"}},
                "required": ["sql"],
            },
        },
    },
]


def query_database(sql: str) -> dict:
    """Replace with your real, read-only data layer."""
    raise NotImplementedError


HANDLERS = {"query_database": query_database}

thread = [
    {
        "role": "system",
        "content": (
            "You are a data analyst. Query for facts rather than assuming them. If a question is "
            "ambiguous about the time period or the metric, ask before querying."
        ),
    },
    {"role": "user", "content": "Which product categories lost margin last quarter?"},
]

CEILING = 20

for step in range(CEILING):
    response = client.chat.completions.create(
        model="Qwen/Qwen3-235B-A22B-Instruct-2507",
        messages=thread,
        tools=TOOLS,
        max_tokens=8192,
    )

    message = response.choices[0].message
    thread.append(message)

    if not message.tool_calls:
        print(message.content)
        break

    for call in message.tool_calls:
        handler = HANDLERS.get(call.function.name)
        if handler is None:
            outcome = {"error": "unknown tool", "name": call.function.name}
        else:
            try:
                outcome = handler(**json.loads(call.function.arguments or "{}"))
            except Exception as exc:
                outcome = {"error": type(exc).__name__, "detail": str(exc)}

        thread.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(outcome)})
else:
    print(f"Stopped at the {CEILING}-step ceiling.")

With no reasoning pass, each turn is fast — which is what makes a twenty-step loop practical in wall-clock terms.

The trade is that each step is a direct decision. Keep tool descriptions precise; a direct-answering model infers less about your intent than a reasoning one would.

And the clarification instruction matters for the same reason. "Last quarter" and "margin" are both ambiguous, and a model that answers directly will resolve them itself if nothing tells it to ask.

Self-Hosting

Reference commands ship for both SGLang and vLLM, configured at the full 262,144 context.

The FP8 checkpoint runs on half the tensor parallelism in Qwen's own example — four devices against eight for the bfloat16 weights. That is the practical argument for it.

Out-of-memory is the documented failure, and Qwen name the fix: reduce the context length, with 32,768 suggested as a fallback. Community quantisation configs default to 32,768 with the full figure commented out, which tells you how often that comes up.

Enable expert parallelism. The community reference commands set it explicitly on a 128-expert model.

A known FP8 issue: fine-grained FP8 in transformers has problems with distributed inference. Setting CUDA_LAUNCH_BLOCKING=1 is the documented workaround when multiple devices are involved.

Qwen-Agent is the recommended agent framework, and the reason is practical: it encapsulates tool-calling templates and parsers internally, which removes the class of silent failure where a parser mismatch turns tool calls into prose.

Local runtimes are supported — Ollama, LM Studio, MLX-LM, llama.cpp, and KTransformers — though 235 billion parameters is a serious proposition even quantised.

Where It Fits

High-throughput production traffic, where twenty-two billion active parameters and no reasoning pass keep both cost and latency predictable.

Interactive assistants, where consistent response time is the product.

Long-document and corpus work at 262,144 tokens, native and without a scaling decision.

Tool orchestration, with tool usage named among the improved capabilities.

Open-ended generation and subjective tasks, which the card names as specifically improved.

Multilingual work, with long-tail knowledge coverage across languages called out.

Not for the hardest reasoning. The Thinking sibling is a different model, and that is the point of the split.

Not for vision. Text only.

Practical Notes

Route hard problems to the Thinking sibling rather than prompting harder here.

Size max_tokens to the answer — nothing else consumes it.

Keep tool descriptions precise; a direct-answering model infers less.

Instruct the model to ask about ambiguity rather than resolving it silently.

Use the full native context without a YaRN decision — it is trained, not extended.

If self-hosting: expert parallelism on, FP8 for lower tensor parallelism, and CUDA_LAUNCH_BLOCKING=1 for distributed FP8 in transformers.

Reduce context length first when you hit out-of-memory. Qwen name it as the fix.

Existing enable_thinking=False code needs no change — the parameter is accepted and ignored.

Limitations

No thinking mode. No reasoning blocks, no effort parameter, no deliberation pass. That is the design, and a real constraint on hard problems.

The switch is gone. A pipeline that used one identifier for two modes now needs two identifiers.

Text only. No image, audio, or video input.

Twenty-two billion active parameters is the compute ceiling per token, whatever the 235 billion total suggests.

235 billion parameters must be loaded even though 22 billion run per token — the MoE saving is in compute, not memory.

Out-of-memory at full context is a documented failure, with reduced context length as the documented fix.

A known FP8 distributed-inference issue in transformers requires an environment-variable workaround.

Confident answers with no visible reasoning. There is less signal about where the model was uncertain, which makes grounding and explicit clarification instructions more important rather than less.