Llama-4-Scout-17B-16E-Instruct

Llama 4 Scout 17B 16E Instruct

Same active parameters as its larger sibling. A fraction of the stored capacity. More room left for context.

The Comparison That Defines It

Two models, released together, with the same activated parameter count.

Identical compute per token. Seventeen billion parameters of work in both cases — the router picks from a smaller pool here, not a smaller expert.

What differs is stored capacity, and the consequence reaches further than the parameter count suggests.

Lighter weights mean more room for context

A 400-billion-parameter model's weights consume most of the available memory before any context is allocated. What remains is what the KV cache gets — and the KV cache is the context window.

At 109 billion parameters, far more memory is left over. vLLM's own documented configurations reflect that directly: Scout is served at a full million tokens on eight H100s where the larger model is configured at roughly 430,000 on identical hardware.

So the smaller model is the long-context model. Not a compromise — a direct consequence of where the memory goes.

iRoPE: Removing Position to Extend Reach

The architectural idea, shared across Llama 4, and it inverts the usual approach.

Llama 4 interleaves global attention without RoPE with chunked local attention with RoPE, at a 1:3 ratio.

The global layers carry no positional encoding at all.

Why removing it helps

Rotary position embeddings encode distance through rotation, and that rotation decays with separation. A token three hundred thousand positions away has been rotated so far from the query that attending to it becomes unreliable.

RoPE is the mechanism that limits long-range attention, not the one that enables it.

So Llama 4 removes it where range matters. Global layers see the entire sequence with no distance penalty, because no distance function is applied.

Position is handled by the local layers instead — three for every global one, each applying RoPE across non-overlapping chunks. Precise where precision is needed, and bounded in cost, because a chunk does not grow as the input does.

vLLM's assessment: this significantly reduces the quadratic complexity of attention as context length scales.

On this model, that design has more room to operate than on its larger sibling — because the memory the weights did not take is memory the cache can use.

Early Fusion, Not an Attached Encoder

Native multimodality through early fusion — text and vision tokens enter the same backbone together, rather than a vision encoder feeding a finished language model.

The difference that makes. A bolted-on encoder produces a language model taught to read a summary of an image. Early fusion produces one where visual and textual information share a representational space from the first layer.

Documented limit: 8 to 10 images per request. Beyond that range, behaviour is outside what Meta validated.

Specifications

Languages: Arabic, English, French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Tagalog, Thai, Vietnamese.

Arabic leads Meta's own list. Twelve is narrow by current standards, which makes what is in it more significant than the count.

Released as BF16, and Meta publish on-the-fly int4 quantisation code that brings it within a single H100 — described as minimising performance degradation. That is the deployment path worth knowing about: no separate quantised checkpoint to download, quantisation applied at load.

Capabilities

⚠️ Read the Licence

A custom commercial licence — not Apache, not MIT.

Meta's Llama licences carry conditions that permissive ones do not, and the terms are specific to this generation rather than inherited from earlier releases. Read it against your deployment before you build on it.

Meta ship safety tooling alongside the model and name it: Llama Guard 3 for filtering input prompts and output responses, plus Prompt Guard and Code Shield where relevant.

Their own recommendation is to evaluate applications in context and build a dedicated evaluation dataset for your use case — right advice on any model, and unusually direct from a vendor.

Using Llama 4 Scout on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: meta-llama/Llama-4-Scout-17B-16E-Instruct

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-4-Scout-17B-16E-Instruct",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

A Third of a Million Tokens

The workload this model suits, and what fits inside it.

from pathlib import Path

REPO = Path("src")

sources = "\n\n".join(
    f"=== {path.relative_to(REPO.parent)} ===\n{path.read_text(encoding='utf-8')}"
    for path in sorted(REPO.rglob("*.py"))
)

response = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages=[
        {
            "role": "system",
            "content": (
                "You are auditing a codebase. Identify every path where a database write can occur "
                "outside a transaction. For each finding, name the file, the function, and the call "
                "chain that reaches it. Report nothing you cannot trace."
            ),
        },
        {"role": "user", "content": sources},
    ],
    max_tokens=16384,
)

print(f"input: {response.usage.prompt_tokens:,} of 327,680")

Asking for the call chain rather than the line is what uses a large window rather than a search. A write outside a transaction is easy to grep for; the path reaching it from three modules away needed the whole subsystem in one context.

327,680 tokens holds a substantial codebase — a subsystem, a service, a document set. A large monorepo needs retrieval or compaction in front of it.

Measure recall at your working length. One global layer in four, carrying no positional encoding, is a genuinely different retrieval mechanism from conventional attention — and whether it holds at three hundred thousand tokens on your data is an empirical question rather than an architectural guarantee.

Print the input against the ceiling. One line, and it tells you how much headroom remains before compaction becomes necessary.

Images, Up to Eight

import base64
from pathlib import Path


def encode(path: str) -> str:
    return base64.b64encode(Path(path).read_bytes()).decode("utf-8")


pages = ["page_1.jpg", "page_2.jpg", "page_3.jpg"]

if len(pages) > 8:
    raise ValueError(f"{len(pages)} images exceeds the validated range of 8–10 per request")

content = []
for index, path in enumerate(pages, start=1):
    content.append({"type": "text", "text": f"Page {index}:"})
    content.append({"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encode(path)}"}})

content.append({
    "type": "text",
    "text": (
        "Transcribe each page, preserving its structure, and label each transcription with its page "
        "number. Mark anything you cannot read cleanly as unreadable rather than reconstructing it."
    ),
})

response = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages=[{"role": "user", "content": content}],
    max_tokens=16384,
)

Keep the guard clause. Eight to ten is where the model was validated; twenty images may work and is untested, and discovering that from degraded output is worse than discovering it from an exception.

Labelling images before they appear gives the model something to cite and you something to verify against.

Working Inside 16,384 Output Tokens

327,680 tokens in. 16,384 out. A twenty-to-one ratio, and it shapes what a single request can do.

It suits analysis perfectly — a large corpus producing a focused answer.

It rules out long-form generation in one call.

SECTIONS = ["Overview", "Findings", "Risks", "Recommendations"]


def write_section(name: str, context: str) -> str:
    """Generate one section against the full context, inside the output ceiling."""
    response = client.chat.completions.create(
        model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
        messages=[
            {
                "role": "system",
                "content": (
                    "Write only the section requested. Do not summarise other sections, add a "
                    "conclusion, or restate the brief."
                ),
            },
            {"role": "user", "content": f"Source material:\n\n{context}\n\nWrite the '{name}' section."},
        ],
        max_tokens=8192,
        temperature=0.4,
    )

    choice = response.choices[0]

    if choice.finish_reason == "length":
        raise ValueError(f"section '{name}' hit the output ceiling — narrow the brief")

    return f"## {name}\n\n{choice.message.content}"

Send the full context every time. With 327,680 tokens available, repeating the source across four requests is affordable — and each section is then written with complete context rather than from a summary of it.

The finish_reason check is mandatory. A section that runs long stops mid-sentence, and it looks like a short section until someone reads it.

Grounding Against an August 2024 Cutoff

from datetime import date

GROUNDED = f"""Answer only from the material provided below. Quote the passage supporting each
statement. Where the material does not contain the answer, say so plainly and stop — do not fill the
gap from general knowledge.

Your training data ends in August 2024. The current date is {date.today().isoformat()}. Treat
anything you recall about events, versions, prices, regulations, or people in roles as potentially
out of date."""

response = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages=[
        {"role": "system", "content": GROUNDED},
        {"role": "user", "content": f"{documents}\n\nQuestion: {question}"},
    ],
    max_tokens=8192,
    temperature=0.2,
)

Injecting the current date alongside the cutoff removes a category of quiet error. A model with a fixed cutoff and no sense of today will reason about "recently" against the wrong anchor, with full confidence.

And a large window is what makes grounding practical. Moving the model from recalling to reading is the fix for a two-year-old cutoff, and 327,680 tokens is enough room to supply what it needs to read.

Self-Hosting

The deployment story is the reason to choose this model over its sibling.

Released as BF16, with on-the-fly int4 quantisation code from Meta that brings it within a single H100 — no separate quantised checkpoint, quantisation applied at load, and described as minimising performance degradation.

vLLM's documented long-context configuration for this model:

VLLM_DISABLE_COMPILE_CACHE=1 vllm serve meta-llama/Llama-4-Scout-17B-16E-Instruct \
  --tensor-parallel-size 8 \
  --max-model-len 1000000 \
  --override-generation-config='{"attn_temperature_tuning": true}'

Two flags worth understanding.

attn_temperature_tuning is attention temperature scaling for long-context stability — behaviour that holds as the sequence grows rather than degrading with it.

VLLM_DISABLE_COMPILE_CACHE=1 appears in vLLM's own launch commands for this family.

A documented configuration reaches 3,600,000 tokens on eight H100s. That is an indication of how much headroom the lighter weights leave rather than a recommendation — and it is worth knowing when sizing your own deployment, since the constraint is memory rather than architecture.

Inference accuracy was validated against Meta's reported figures using lm-eval-harness — a reproducibility claim rather than a benchmark claim, and a useful one.

Choosing Between Scout and Maverick

Both activate seventeen billion parameters. The decision is not about speed.

Choose Scout when a single H100 is what you have, when deployment simplicity matters, or when the task draws on breadth of input rather than breadth of stored knowledge.

Choose Maverick when stored capacity is the constraint. Four hundred billion parameters hold more than a hundred and nine, and on tasks drawing on knowledge rather than on what you supply, that shows.

The test is cheap. Same interface, same request shape, one changed identifier — run twenty of your real prompts through both and compare. That result is worth more than any reasoning about expert counts.

Where It Fits

Subsystem and corpus-scale work, where 327,680 tokens holds the whole thing in one context.

Single-GPU deployment — int4 on one H100, which the larger sibling does not offer.

Multimodal assistants through native early fusion.

Multilingual work across twelve languages, Arabic first in Meta's own list.

Document and page understanding, inside the eight-to-ten-image range.

Code analysis and generation with multilingual support.

Not for long-form generation. 16,384 output tokens.

Not as a current knowledge source. August 2024.

Not for reasoning-heavy work. No thinking mode.

Not where stored breadth matters most — the larger sibling holds nearly four times the parameters.

Practical Notes

Design around 327,680, not the architecture's headline figure.

Stay inside eight to ten images per request.

Check finish_reason on generation work — the output ceiling is 16,384.

Chunk long-form output by section, sending the full context each time.

Inject the current date alongside the August 2024 cutoff.

Measure long-context recall at your real working length.

If self-hosting: int4 on a single H100, attn_temperature_tuning for long context, and VLLM_DISABLE_COMPILE_CACHE=1.

Compare against the larger sibling on your own prompts — same interface, one changed identifier.

Read the Llama 4 Community License, and build a moderation layer.

Limitations

Knowledge ends August 2024. The model will answer about later events with full confidence.

327,680-token window. Generous, and not the million the architecture describes — a large monorepo or archive still needs retrieval in front of it.

16,384-token output ceiling against a 327,680-token input window.

Eight to ten images per request is the validated range.

Sixteen experts against a hundred and twenty-eight on the larger sibling. Same compute per token, roughly a quarter of the stored capacity.

Twelve languages. Narrow by current standards, though Arabic is among them.

Text output only. It reads images; it does not generate them.

No reasoning mode. Multi-step logic belongs on a model built for it.

Long context needs a tuning flag when self-hosted — attn_temperature_tuning rather than default settings.

A custom commercial licence, not Apache or MIT.

An April 2025 model. Mature and widely supported, and the field has moved — choose it for the long-context design and the deployment footprint, not for frontier capability.

Confident answers with no visible reasoning. Ground factual work and require an explicit way for the model to say it does not know.