Qwen3-Embedding-4B

Qwen3-Embedding-4B

Text in, vectors out. Thirty-two thousand tokens of input, up to 2,560 dimensions, and two kinds of customisation.

Paper: arXiv:2506.05176

Two Customisation Axes, Rarely Found Together

Most embedding models give you one lever. This one gives two, and they operate independently.

The first changes the vector. The second changes what goes into it.

Matryoshka representation learning is a storage decision — how much of the vector you keep. Task instructions are a quality decision — telling the model what "relevant" means for your case.

They compose. A 1,024-dimension vector produced under a task-specific instruction is a different thing from a 1,024-dimension vector produced under the generic default, at identical storage cost.

The Instruction Is a Real Input

The lever most often left unused, and the one that costs nothing.

Qwen's measurement: using a tailored instruction typically yields a 1% to 5% improvement across most downstream tasks. Their recommendation is to create instructions specific to your task and scenario rather than accepting the default.

Write yours. The default describes generic retrieval. Your corpus is not generic.

Given a customer support question, retrieve the help-centre article that
resolves it.

Given a natural-language description of a bug, retrieve the source file most
likely to contain it.

And the non-obvious part, stated directly by Qwen: in multilingual contexts, write your instructions in English — because most instructions used during training were originally written in English.

So: documents in Arabic, query in Arabic, instruction in English. Counterintuitive, and it is what the model was trained on.

One instruction per task, not per query. The instruction describes the retrieval job. Write it once, keep it in configuration, and reuse it across every query in that pipeline.

32,768 Tokens Changes What You Embed

The specification that separates this family from most embedding models.

Most embedding models cap at 512 tokens. Some reach a few thousand. That limit is the reason retrieval pipelines chunk everything — the model cannot read a document, so it reads a paragraph.

This one reads 32,768.

What that enables. A contract, a research paper, a support article, a source file — embedded whole, with all its context intact. Not a fragment that happened to contain the right words.

And what it does not change. A vector is still one point in space. The longer and more varied the text behind it, the more that point becomes an average of unrelated meanings — retrieved for nothing in particular because it is close to everything.

So the window is permission, not instruction. Embed a whole document when the document is about one thing. Split it when it is about five, regardless of whether the whole fits.

Specifications

Transformers below 4.51.0 raises an error. The Qwen3 architecture is recent enough that older library versions do not recognise it.

Capabilities

Sizing the Vector

2,560 dimensions is the full output, and MRL means it does not have to be what you store.

The storage arithmetic is linear. An index of ten million vectors at 2,560 dimensions is two and a half times the size of the same index at 1,024 — and two and a half times the comparison cost on every query.

Where to start. Build an evaluation set first: a few hundred query-and-expected-result pairs from your own corpus. Embed at full dimension, measure retrieval accuracy, then repeat at each candidate size. Take the smallest that holds your number.

Truncation is not free everywhere. Matryoshka training concentrates the most significant information in the earliest dimensions, so shortening degrades gracefully — but how gracefully depends on the corpus. Dense technical content and source code typically tolerate it worse than prose.

One rule with no exceptions: every vector in an index must share the same dimension count. Mixing lengths produces results that are silently meaningless rather than an error.

And re-normalise after truncating. Cutting a unit vector leaves something shorter than unit length, and a dot product computed as cosine similarity will be quietly wrong without it.

Using Qwen3-Embedding-4B on DEVUP AI

Endpoint: POST https://api.devupai.com/v1/embeddings

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.embeddings.create(
    model="Qwen/Qwen3-Embedding-4B",
    input=[
        "Adding an index on the filtered column usually eliminates a sequential scan.",
        "Les paiements par carte CIB sont traites en dinar algerien.",
        "Connection pooling reduces the cost of opening a new database session.",
    ],
)

vectors = [item.embedding for item in response.data]

print(f"{len(vectors)} vectors of {len(vectors[0])} dimensions")
print(f"Tokens consumed: {response.usage.total_tokens}")

Results return in the order sent. Batching is the largest single performance factor on an embedding endpoint — one request with a hundred texts beats a hundred requests with one each by a wide margin.

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

const response = await client.embeddings.create({
  model: "Qwen/Qwen3-Embedding-4B",
  input: [
    "Adding an index usually eliminates a sequential scan.",
    "Connection pooling reduces session setup cost.",
  ],
});

const vectors = response.data.map((item) => item.embedding);
console.log(`${vectors.length} vectors of ${vectors[0].length} dimensions`);

cURL

curl https://api.devupai.com/v1/embeddings \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-Embedding-4B",
    "input": "How do I speed up a slow database query?"
  }'

Truncating and Re-Normalising

If the endpoint exposes a dimensions parameter, use it. If not, MRL truncation is something you can do yourself — and the model was trained for exactly that.

import numpy as np


def truncate(vector: list[float], dimensions: int) -> np.ndarray:
    """Truncate a Matryoshka embedding and re-normalise it to unit length."""
    short = np.array(vector[:dimensions], dtype=np.float32)
    return short / np.linalg.norm(short)

The re-normalisation is not optional. Truncating a unit vector leaves a shorter one, and cosine similarity computed as a dot product silently returns wrong magnitudes without the correction.

Measuring Your Own Dimension

The experiment worth running before you commit an index — and it is cheap, because MRL truncation happens client-side.

DIMENSIONS_TO_TEST = (2560, 1536, 1024, 512)


def embed(texts: list[str]) -> list[list[float]]:
    """Embed once, at full dimension."""
    response = client.embeddings.create(model="Qwen/Qwen3-Embedding-4B", input=texts)
    return [item.embedding for item in response.data]


def recall_at_k(raw_corpus, raw_queries, expected, dimensions: int, k: int = 5) -> float:
    """Fraction of queries whose expected passage appears in the top k, at a given dimension."""
    corpus_vectors = np.array([truncate(v, dimensions) for v in raw_corpus])
    query_vectors = np.array([truncate(v, dimensions) for v in raw_queries])

    hits = 0
    for query_vector, target in zip(query_vectors, expected):
        top_k = np.argsort(corpus_vectors @ query_vector)[::-1][:k]
        hits += target in top_k

    return hits / len(query_vectors)


raw_corpus = embed(CORPUS)
raw_queries = embed(QUERIES)

for dimensions in DIMENSIONS_TO_TEST:
    print(f"{dimensions:>5}d  recall@5 {recall_at_k(raw_corpus, raw_queries, EXPECTED, dimensions):.3f}")

Every dimension is derived from one embedding call. Sweeping four sizes costs the same as embedding once — which makes this the cheapest meaningful experiment available on an embedding model, and there is no reason to skip it.

Semantic Search

DIMENSIONS = 1024   # chosen by measurement, fixed for the life of the index


def embed_matrix(texts: list[str]) -> np.ndarray:
    """Embed texts, truncate, re-normalise, and return one row per input."""
    response = client.embeddings.create(model="Qwen/Qwen3-Embedding-4B", input=texts)
    return np.array([truncate(item.embedding, DIMENSIONS) for item in response.data])


CORPUS = [
    "Refunds are issued to the original payment method within 14 days.",
    "Orders above 5000 DZD qualify for free delivery nationwide.",
    "Support is available Sunday through Thursday, 9am to 5pm.",
]

corpus_vectors = embed_matrix(CORPUS)
query_vector = embed_matrix(["How long until I get my money back?"])[0]

scores = corpus_vectors @ query_vector

for rank, index in enumerate(np.argsort(scores)[::-1], start=1):
    print(f"{rank}. [{scores[index]:.3f}] {CORPUS[index]}")

The query and the matching passage share almost no vocabulary. Closing that gap is what embeddings do and keyword search does not.

Cross-Language Retrieval

A query in one language finding a passage written in another, with no translation step.

CORPUS = [
    "Les remboursements sont effectues sous 14 jours ouvrables.",
    "La livraison est gratuite pour les commandes superieures a 5000 DZD.",
    "Le service client est disponible du dimanche au jeudi.",
]

corpus_vectors = embed_matrix(CORPUS)
query_vector = embed_matrix(["When will I be refunded?"])[0]

print(CORPUS[int(np.argmax(corpus_vectors @ query_vector))])

Over a hundred languages, including programming languages — which makes cross-language and cross-modal code retrieval a single-model problem rather than two.

Caching What Does Not Change

Output is deterministic, so re-embedding unchanged text produces an identical vector and buys nothing.

import hashlib
import json
from pathlib import Path

CACHE = Path("embedding_cache.json")


def cached_embed(texts: list[str], dimensions: int = DIMENSIONS) -> list[list[float]]:
    """Embed only texts not already stored, keyed by content and dimension."""
    store = json.loads(CACHE.read_text()) if CACHE.exists() else {}

    keys = [hashlib.sha256(f"{dimensions}:{t}".encode()).hexdigest() for t in texts]
    missing = [(k, t) for k, t in zip(keys, texts) if k not in store]

    if missing:
        response = client.embeddings.create(
            model="Qwen/Qwen3-Embedding-4B",
            input=[t for _, t in missing],
        )
        for (key, _), item in zip(missing, response.data):
            store[key] = truncate(item.embedding, dimensions).tolist()
        CACHE.write_text(json.dumps(store))

    return [store[k] for k in keys]

The dimension belongs in the cache key. Two vectors for the same text at different lengths are different vectors, and treating them as interchangeable corrupts the index quietly.

Pair It With the Matching Reranker

The family ships embedding and reranking models at the same three sizes, and they are built to work together.

Two-stage retrieval is the intended pattern. Embeddings fetch roughly a hundred candidates cheaply; the reranker reads the query and each document together and narrows those to the handful you actually put in the prompt.

Same window, same instruction awareness, same language coverage across both halves — which means the two stages agree about what your corpus contains rather than disagreeing subtly.

And the sizes do not have to match. A 4B embedding model with a 0.6B reranker is a reasonable configuration: the embedding runs once per document, the reranker runs once per query, and those are different cost profiles.

The Family

The 8B entered the MTEB multilingual leaderboard at number one at release.

This one is the middle tier, and the position is a genuine trade rather than a compromise: two and a half times the dimensions of the small model, at a fraction of the largest model's cost per document embedded.

Embedding cost is paid once per document; retrieval quality is paid on every query. That asymmetry usually argues for moving up rather than down — which makes the 4B the default rather than the fallback.

Practical Notes

Write your own instruction. It is worth one to five percent and costs one line of configuration.

Write it in English even when your documents are not.

Measure your dimension before building the index — one embedding call covers every size.

Always re-normalise after truncating.

Record the model and dimension alongside your vectors. Six months on, that metadata is the difference between a migration and a rebuild.

Query and corpus must share a model and a dimension. A mismatch fails silently.

Batch aggressively, and cache anything that does not change.

Add lexical search alongside for identifiers, codes, and rare proper nouns — that is where dense retrieval is weakest and keyword matching strongest.

Debounce anything that embeds on keystroke.

Limitations

Not generative. It produces vectors, not text. It finds the passages another model answers from.

32,768 tokens maximum. Generous for an embedding model, and longer inputs still require splitting.

Dimension is permanent per index. Changing it means re-embedding the corpus.

Instructions were trained in English. Multilingual retrieval works; the instruction should not be in the document's language.

2,560 dimensions is heavier than the small model. Two and a half times the storage and comparison cost at full length, which is why measuring your truncation point matters.

Weak on exact matching. Identifiers, codes, version strings, and rare proper nouns favour lexical search.

Similarity is not relevance. Two passages can sit close together and still be the wrong answer. Chunking, corpus quality, and ranking affect retrieval as much as the model does.

Transformers 4.51.0 or later is required for self-hosting.

Embedded text leaves your system. Apply the same handling policy you would to any other outbound data.