Qwen3.5-35B-A3B

Qwen3.5-35B-A3B

Thirty-five billion parameters. Three active. Forty layers, thirty of them linear.

The Layout

Published and confirmed across the model's quantised derivatives:

10 × (3 × (Gated DeltaNet → MoE) → 1 × (Full Attention → MoE))

Three quarters of the depth never builds a growing cache. Gated DeltaNet carries sequence state through a recurrent update at constant cost, regardless of how much input arrived. Ten full-attention layers provide the exact global retrieval that recurrent state cannot.

Nine experts of 256 — three and a half percent of the pool per token.

Two sparsity mechanisms stacked, and they multiply rather than overlap. Linear attention removes the quadratic cost of length; MoE removes the cost of width. Three billion active parameters against a 262,144-token window is what the combination produces.

Multimodal From the Start

A 27-block vision transformer, integrated rather than attached — and trained by early fusion on multimodal tokens rather than bolted onto a finished language model.

Text, images, and video, interleaved in a single prompt.

Qwen's claim is that early fusion achieves parity with the previous text generation while outperforming that generation's dedicated vision-language models on reasoning, coding, agents, and visual understanding.

That is a claim about breadth, and it is cheaply testable: point one workload you currently split across two models at this one and compare. If it holds, you maintain one integration instead of two.

Three Billion Active Changes the Shape of the Work

The number that decides what this model is for.

A 262,144-token window at three billion active parameters is an unusual combination. Long context is normally expensive because every token costs full compute; here it costs a small model's compute.

What that enables:

Document archives at volume, where per-page cost decides whether the corpus gets processed at all.

Long agent sessions, where accumulated history does not force compaction after twenty steps.

High-throughput production traffic, with multimodal input on the same path rather than routed to a second model.

What it does not enable is peak capability. Three billion parameters of compute per token is the ceiling, and the larger models in this generation — 122B-A10B and 397B-A17B — exist for work that reaches it.

The Open Weights Are Not the Hosted Model

Documented plainly, and worth settling before you plan around a capability.

A hosted version corresponds to this model, with production features these weights do not carry:

So a million-token figure quoted for this model may describe the managed service rather than the checkpoint you are calling.

The number that matters is the one your path supports. 262,144 is what the weights document, and it is what the reference deployments configure.

Specifications

Official builds include a base checkpoint for further training, with community GPTQ, GGUF, and other quantisations alongside.

Capabilities

Three Claims From the Card

Each testable in a different way.

Million-agent reinforcement learning. RL scaled across million-agent environments with progressively complex task distributions, targeting real-world adaptability rather than benchmark scores. Testable on your own agent workload.

201 languages and dialects, with stated attention to cultural and regional nuance — not just languages, but varieties within them.

Unified vision-language foundation through early fusion, with the parity claim above.

Using Qwen3.5-35B-A3B on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: Qwen/Qwen3.5-35B-A3B

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-35B-A3B",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "Qwen/Qwen3.5-35B-A3B",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3.5-35B-A3B",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Images and Video

Native inputs, interleaved with text in one message.

import base64

with open("invoice.png", "rb") as handle:
    encoded = base64.b64encode(handle.read()).decode("utf-8")

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-35B-A3B",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}},
                {
                    "type": "text",
                    "text": (
                        "Transcribe every line item with its quantity and amount, preserving the "
                        "table structure. Report the currency exactly as printed. Mark anything you "
                        "cannot read cleanly as unreadable rather than reconstructing it."
                    ),
                },
            ],
        }
    ],
    max_tokens=8192,
)

Send images at full resolution. The encoder was built for native size; downscaling first discards detail it would otherwise use.

Video consumes input tokens rapidly — faster than the arithmetic on a text prompt suggests. Measure one short clip before processing anything longer; that measurement is the difference between a pipeline that completes and one that fails partway through a file.

Long-Context Work at Low Cost

Where the two sparsity mechanisms pay off together.

from pathlib import Path

corpus = "\n\n---\n\n".join(
    f"### {path.name}\n{path.read_text(encoding='utf-8')}"
    for path in sorted(Path("filings").glob("*.txt"))
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-35B-A3B",
    messages=[
        {
            "role": "system",
            "content": (
                "Identify every figure that appears in more than one document with a different "
                "value. Quote both occurrences and name both files. Report nothing you cannot quote."
            ),
        },
        {"role": "user", "content": corpus},
    ],
    max_tokens=16384,
)

print(f"input: {response.usage.prompt_tokens:,} of 262,144")

Cross-document work is what the window is for. A discrepancy between the third document and the eleventh is invisible to a pipeline that reads them one at a time.

And the ratio matters here. Exact retrieval across a very long input rests on ten full-attention layers out of forty. Measure recall at your actual working length before depending on it — a task that works at 50,000 tokens is not evidence about 250,000.

An Agent Loop

The workload the million-agent RL training targeted.

import json

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read a file relative to the repository root.",
            "parameters": {
                "type": "object",
                "properties": {"path": {"type": "string"}},
                "required": ["path"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "run_tests",
            "description": "Run the test suite and return pass/fail counts with failure output.",
            "parameters": {"type": "object", "properties": {}},
        },
    },
]


def read_file(path: str) -> dict:
    """Replace with your real, sandboxed file access."""
    raise NotImplementedError


def run_tests() -> dict:
    """Replace with your real, sandboxed test runner."""
    raise NotImplementedError


HANDLERS = {"read_file": read_file, "run_tests": run_tests}

session = [
    {"role": "user", "content": "The DZD invoice test fails on totals ending in .005. Find the cause and fix it."}
]

CEILING = 50

for step in range(CEILING):
    response = client.chat.completions.create(
        model="Qwen/Qwen3.5-35B-A3B",
        messages=session,
        tools=TOOLS,
        max_tokens=16384,
    )

    message = response.choices[0].message
    session.append(message)

    if not message.tool_calls:
        print(message.content)
        break

    for call in message.tool_calls:
        handler = HANDLERS.get(call.function.name)
        if handler is None:
            outcome = {"error": "unknown tool", "name": call.function.name}
        else:
            try:
                outcome = handler(**json.loads(call.function.arguments or "{}"))
            except Exception as exc:
                # Failures return as data — RL across agent environments trained for recovery.
                outcome = {"error": type(exc).__name__, "detail": str(exc)}

        session.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(outcome)})
else:
    print(f"Stopped at the {CEILING}-step ceiling.")

Three billion active parameters per step is what makes a fifty-step loop affordable in a way it would not be on a dense model of similar total size.

Returning tool errors as data rather than raising uses what the training produced. A model trained across agent environments with rising task complexity was trained to work around failures; an exception discards that.

Reading the Reasoning

message = response.choices[0].message

trace = getattr(message, "reasoning_content", None)
if trace:
    logger.debug("reasoning: %d characters", len(trace))

print(message.content)

Keep the fields apart in both directions. Merging reasoning into the answer breaks structured-output parsing and puts a working draft in front of readers expecting a conclusion.

Check finish_reason. Reasoning shares the output budget, and a response truncated at the ceiling can contain a complete trace and no answer at all — which reads as an empty result rather than as an error.

Self-Hosting

Four details, and two of them will stop you.

Transformers 5.2.0 or later. The Qwen3.5 architecture is recent enough that older versions do not recognise the model class.

The model class is Qwen3_5MoeForConditionalGeneration — multimodal, distinct from the text-only classes in earlier generations. Loading with the wrong class fails.

MoE expert weights are stored as fused 3D tensors, not as individual linear modules. Quantisation toolchains convert them — and that conversion must also run at load time for quantised kernels to apply correctly. A quantised build that loads without the conversion runs at full precision on the expert path, silently, at the speed you were trying to avoid.

The vision encoder and MTP module stay at BF16 in the published quantisations. Only the text model's expert weights are quantised — which is the right choice, and worth matching if you produce your own build.

YaRN for the extended window. Static scaling holds the factor constant regardless of input length, so leaving it on degrades short prompts. Enable it only when you actually serve long ones.

Enable the MTP module where your stack supports speculative decoding. It is native and trained in; omitting it leaves throughput on the table with no separate draft model to configure.

Where It Fits

High-volume multimodal work — document archives, receipt pipelines, image-heavy traffic — where three billion active parameters decides whether the corpus gets processed.

Long-context analysis at 262,144 tokens on a small model's compute.

Agentic workflows, where the RL training targeted adaptability and the low activation cost makes long loops affordable.

Multilingual deployment across 201 languages and dialects.

Unified text and vision in one integration, if the early-fusion parity claim holds on your workload.

Self-hosted and local deployment — 35 billion parameters quantised is genuinely accessible, and the base checkpoint is published for further training.

Not for peak capability. The 122B and 397B models in this generation are where that lives.

Not for output beyond text. Three input modalities, one output modality.

Practical Notes

Measure long-context recall at your real working length, not a convenient one.

Measure video token consumption before scaling.

Send images at full resolution.

Check finish_reason — reasoning shares the output budget.

Confirm which context your path supports; the hosted version and the open weights differ.

If self-hosting: transformers 5.2.0+, the correct model class, the expert-tensor conversion at load time, MTP enabled, and YaRN only when you need it.

Escalate to the larger models in this generation when the limit is reasoning rather than throughput.

Limitations

Three billion active parameters is the compute ceiling per token. The sparsity buys speed and footprint, not unlimited capability.

Thirty-five billion parameters must be loaded even though three run per token — the MoE saving is in compute, not memory.

262,144 is native; a million is extension, and the hosted version of this model is where the million-token default lives.

Exact retrieval rests on ten layers of forty. Verify recall at long input lengths rather than assuming it holds flat.

Text output only. It reads text, images, and video; it writes words.

Video fills the window quickly. The constraint moves rather than disappearing.

Quantised builds need a load-time tensor conversion. Without it the expert path runs unquantised, silently.

Transformers 5.2.0 minimum, and a model class distinct from earlier generations.

Reasoning traces are working notes. Unpolished, sometimes exploring abandoned branches, and occasionally contradicting the answer that follows.