Qwen3-Embedding-8B

Qwen3-Embedding-8B

First on the MTEB multilingual leaderboard at release. Vectors from 32 to 4,096 dimensions, from inputs up to 32,768 tokens.

The Result, With Its Date

No. 1 on the MTEB multilingual leaderboard — score 70.58, as of 5 June 2025.

Qwen state the date alongside the ranking, which is the right way to publish a leaderboard position: leaderboards move, and a claim without a date is a claim that ages badly without saying so.

Read it as "it was the best open multilingual embedding model when it shipped." Whether it still holds that position is a question about the leaderboard today, not about this page.

What the benchmark covers is broader than retrieval alone: text retrieval, code retrieval, text classification, clustering, and bitext mining. A model topping the aggregate is strong across the category rather than tuned for one task in it.

Thirty-Two to Four Thousand and Ninety-Six

The dimension range, and the phrasing matters.

Not a set of presets. A range. Any dimension from 32 to 4,096.

That is unusual. Most Matryoshka models publish a handful of tested sizes — 768, 512, 256. Here the model supports flexible vector definitions across all dimensions, which means the decision is continuous rather than discrete.

What that changes. You are not choosing the nearest preset to your storage budget; you are choosing your storage budget and taking the vector that fits it.

And the range is wide. A 32-dimension vector is 1/128th the storage of a 4,096-dimension one. Even if the low end degrades sharply — and it will — the space between 512 and 4,096 is where most real decisions live, and that is a lot of room to measure through.

The habits do not change. Truncate, re-normalise, and keep one dimension per index. Mixing lengths produces results that are silently meaningless rather than an error.

Merged, Not Trained

The training detail worth knowing, because it explains the model's behaviour profile.

Three stages, following the paradigm of an earlier Qwen series:

Stage one: contrastive pre-training on a large volume of weakly supervised data.

Stage two: supervised training on high-quality labelled data.

Stage three: integrating multiple candidate models through a merging strategy.

That third stage is the unusual one. The released model is not one training run's output — it is several candidates combined.

Qwen's stated reason: the staged mechanism balances generalisation ability against task adaptability.

Why those pull apart. A model trained hard on one distribution adapts well to it and transfers poorly. A model trained broadly transfers well and excels nowhere. Merging candidates that landed in different places is an attempt to get both — and it is a plausible explanation for a model that tops an aggregate benchmark spanning five distinct task types rather than winning one of them.

The Instruction Is Measured

Qwen's evaluation: using a task-specific instruction typically yields a 1% to 5% improvement over not using one, across most downstream tasks.

On a model that already ranks first, one to five percent is not noise. Leaderboard positions at the top are frequently separated by less.

Write yours. The default describes generic retrieval; your corpus is not generic.

Given a customer support question, retrieve the help-centre article that
resolves it.

Given a natural-language description of a bug, retrieve the source file
most likely to contain it.

And the counterintuitive part, stated by Qwen: in multilingual contexts, write your instructions in English — because most instructions used during training were originally written in English.

Documents in Arabic, query in Arabic, instruction in English.

One instruction per task, not per query. It describes the retrieval job rather than the individual question. Write it once and keep it in configuration.

Specifications

An official GGUF build is published by Qwen alongside the standard weights — uncommon for an embedding model, and useful for local deployment.

Capabilities

Where the Family Sits

Three sizes, and the differences concentrate in one place.

The window is identical across all three. Thirty-two thousand tokens on the smallest model as well as this one.

The 4B and 8B share a layer count. Thirty-six each — the difference between them is width rather than depth.

What you are buying at 8B is dimensionality, and everything that follows from it: more representational capacity per vector, more room to truncate before quality degrades, and four times the storage of the smallest model at full length.

Using Qwen3-Embedding-8B on DEVUP AI

Endpoint: POST https://api.devupai.com/v1/embeddings

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.embeddings.create(
    model="Qwen/Qwen3-Embedding-8B",
    input=[
        "Adding an index on the filtered column usually eliminates a sequential scan.",
        "Les paiements par carte CIB sont traites en dinar algerien.",
        "Connection pooling reduces the cost of opening a new database session.",
    ],
)

vectors = [item.embedding for item in response.data]

print(f"{len(vectors)} vectors of {len(vectors[0])} dimensions")
print(f"Tokens consumed: {response.usage.total_tokens}")

Batching is the largest performance factor on an embedding endpoint — one request with a hundred texts beats a hundred requests with one each by a wide margin.

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

const response = await client.embeddings.create({
  model: "Qwen/Qwen3-Embedding-8B",
  input: [
    "Adding an index usually eliminates a sequential scan.",
    "Connection pooling reduces session setup cost.",
  ],
});

const vectors = response.data.map((item) => item.embedding);
console.log(`${vectors.length} vectors of ${vectors[0].length} dimensions`);

cURL

curl https://api.devupai.com/v1/embeddings \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-Embedding-8B",
    "input": "How do I speed up a slow database query?"
  }'

Sweeping the Dimension Range

With a continuous range and one embedding call, this experiment is close to free — and it is the one that decides your storage bill for the life of the index.

import numpy as np


def truncate(vector: list[float], dimensions: int) -> np.ndarray:
    """Truncate a Matryoshka embedding and re-normalise it to unit length."""
    short = np.array(vector[:dimensions], dtype=np.float32)
    return short / np.linalg.norm(short)


def recall_at_k(raw_corpus, raw_queries, expected, dimensions: int, k: int = 5) -> float:
    """Fraction of queries whose expected passage appears in the top k, at a given dimension."""
    corpus_vectors = np.array([truncate(v, dimensions) for v in raw_corpus])
    query_vectors = np.array([truncate(v, dimensions) for v in raw_queries])

    hits = 0
    for query_vector, target in zip(query_vectors, expected):
        top_k = np.argsort(corpus_vectors @ query_vector)[::-1][:k]
        hits += target in top_k

    return hits / len(query_vectors)


def embed(texts: list[str]) -> list[list[float]]:
    response = client.embeddings.create(model="Qwen/Qwen3-Embedding-8B", input=texts)
    return [item.embedding for item in response.data]


raw_corpus = embed(CORPUS)
raw_queries = embed(QUERIES)

for dimensions in (4096, 2048, 1024, 512, 256, 128):
    recall = recall_at_k(raw_corpus, raw_queries, EXPECTED, dimensions)
    storage = dimensions / 4096
    print(f"{dimensions:>5}d  recall@5 {recall:.3f}  storage {storage:>5.1%}")

Printing storage alongside recall is what makes the decision. Recall alone tells you which is best; the pair tells you which is worth it.

Six sizes from one embedding call. Truncation happens client-side, so sweeping the range costs exactly what embedding once costs — and there is no defensible reason to skip it before committing an index.

Build the evaluation set first. A few hundred query-and-expected-result pairs from your real corpus. That set outlives every model decision you make with it.

Semantic and Cross-Lingual Search

DIMENSIONS = 1024   # chosen by measurement, fixed for the life of the index


def embed_matrix(texts: list[str]) -> np.ndarray:
    """Embed, truncate, re-normalise, and return one row per input."""
    response = client.embeddings.create(model="Qwen/Qwen3-Embedding-8B", input=texts)
    return np.array([truncate(item.embedding, DIMENSIONS) for item in response.data])


CORPUS = [
    "Les remboursements sont effectues sous 14 jours ouvrables.",
    "La livraison est gratuite pour les commandes superieures a 5000 DZD.",
    "Le service client est disponible du dimanche au jeudi.",
]

corpus_vectors = embed_matrix(CORPUS)
query_vector = embed_matrix(["When will I be refunded?"])[0]

print(CORPUS[int(np.argmax(corpus_vectors @ query_vector))])

An English query retrieving a French passage with no translation step. Over a hundred languages plus programming languages, which makes cross-lingual and code retrieval one model's problem rather than three.

Cosine is the stated similarity metric, and the vectors come back normalised — so a dot product is already the cosine similarity.

Pairing It With a Reranker

Two-stage retrieval, and one detail from Qwen's own evaluation is worth copying.

Their reranker results were all measured on the top-100 candidates retrieved by the 0.6B embedding model — not by this one.

Read that as a configuration recommendation. The first stage is a filter; it needs to be cheap and to have high recall at a hundred candidates. Precision at position one is the reranker's job.

So the strongest embedding model is not automatically the right first stage. If a smaller model puts the correct document somewhere in the top hundred just as reliably, the reranker does not care which of them found it — and the smaller model embedded the corpus for a fraction of the cost.

Where this model earns its place instead:

Where there is no second stage. Embedding-only retrieval, classification, clustering, or similarity scoring — all of which depend entirely on vector quality.

Where recall at a hundred is genuinely hard. A large, semantically dense corpus where a weaker first stage misses the answer entirely.

Where the vectors serve several purposes. An index used for retrieval, deduplication, clustering, and recommendation at once justifies more capacity than retrieval alone would.

Caching and Indexing at Scale

import hashlib
import json
from pathlib import Path

CACHE = Path("embedding_cache.json")
BATCH = 64
DIMENSIONS = 1024


def embed_corpus(documents: list[str]) -> list[list[float]]:
    """Embed in batches, skipping anything already stored, checkpointing as it goes."""
    store = json.loads(CACHE.read_text()) if CACHE.exists() else {}

    keys = [hashlib.sha256(f"{DIMENSIONS}:{d}".encode()).hexdigest() for d in documents]
    missing = [(k, d) for k, d in zip(keys, documents) if k not in store]

    for start in range(0, len(missing), BATCH):
        chunk = missing[start : start + BATCH]

        response = client.embeddings.create(
            model="Qwen/Qwen3-Embedding-8B",
            input=[d for _, d in chunk],
        )

        for (key, _), item in zip(chunk, response.data):
            store[key] = truncate(item.embedding, DIMENSIONS).tolist()

        CACHE.write_text(json.dumps(store))
        print(f"{start + len(chunk):>7,} / {len(missing):,}")

    return [store[k] for k in keys]

The dimension belongs in the cache key. Two vectors for the same text at different lengths are different vectors, and treating them as interchangeable corrupts the index quietly.

Checkpoint every batch. On a corpus large enough to justify this model, a run that dies at document 800,000 should resume rather than restart.

Practical Notes

Write your own instruction, in English, whatever language your corpus is in.

Sweep the dimension range before committing an index — it costs one embedding call.

Print storage alongside recall when you do.

Always re-normalise after truncating.

Record the model and dimension with your vectors.

Query and corpus must share a model and a dimension.

Consider a smaller model for the first stage if a reranker follows. Qwen's own evaluations did.

Cache by content hash and checkpoint every batch when indexing at scale.

Add lexical search alongside for identifiers, codes, and rare proper nouns.

Treat the leaderboard position as dated. It was first in June 2025; check the current board if the ranking is your reason for choosing it.

Limitations

Not generative. It produces vectors, not text.

4,096 dimensions at full length — four times the smallest model in this family in storage and comparison cost. That is the price of the capacity.

32,768 tokens maximum. Generous for an embedding model, and longer inputs still require splitting.

Dimension is permanent per index. Changing it means re-embedding the corpus.

Instructions were trained in English. Multilingual retrieval works; the instruction should not be in the document's language.

The leaderboard position is dated 5 June 2025. Qwen say so, and leaderboards move.

Weak on exact matching. Identifiers, codes, version strings, and rare proper nouns favour lexical search.

Similarity is not relevance. Two passages can sit close together and still be the wrong answer. Chunking, corpus quality, and ranking affect retrieval as much as the model does.

It may be more model than your first stage needs. Qwen's own two-stage evaluations used the smallest model to retrieve and a reranker to select.

Transformers 4.51.0 or later is required for self-hosting.

Embedded text leaves your system. Apply the same handling policy you would to any other outbound data.