Llama-Guard-4-12B

Llama Guard 4 12B

A safety classifier that happens to be a language model. Two lines of output, and that is the whole interface.

What It Returns

First line: safe or unsafe.

Second line, only when unsafe: a comma-separated list of violated categories.

That is the entire output format. No prose, no explanation, no hedging.

safe

unsafe
S9,S2

Which makes parsing trivial and makes the model easy to misuse. It is not a chat model that gives safety opinions — it is a classifier whose output goes into an if statement.

The corollary matters too. Asking it to explain its reasoning, to be more nuanced, or to "consider context" is asking it to do something it was not trained to do. The format is the product.

How It Was Built

An unusual construction, and Meta describe it precisely.

Take Llama 4 Scout — a mixture-of-experts model. Remove the routed experts and the router layers. Keep only the shared expert.

The result is dense, and Meta state the next part plainly: no additional pre-training was performed.

Read that as an efficiency argument. Llama 4 Scout's pre-training cost a great deal. Pruning it to a dense twelve-billion-parameter model inherits that pre-training rather than repeating it — and what follows is fine-tuning for safety classification alone.

And one consequence is practical. Llama Guard 4 shares the same tokenizer and vision encoder as Llama 4 Scout and Maverick. If you are already serving those models, the moderation layer speaks the same language — literally, in the tokeniser sense.

Two Models, Replaced by One

The previous generation needed two deployments.

Combined into a single classifier with twelve billion parameters, supporting English and multilingual text as well as mixed text-and-image prompts.

What that removes. Two model servers, two sets of weights, two integrations, and the routing logic deciding which one a request belongs to.

Natively multimodal through early fusion, trained jointly on text and multiple images rather than having vision attached.

Input Filtering Against Output Filtering

Meta discuss both, and they are not the same decision.

Input filtering classifies the user's prompt before it reaches your model.

Advantage: unsafe content is caught very early, before the LLM even responds. No generation cost, no latency spent producing something you will discard.

Output filtering classifies your model's response before it reaches the user.

Advantage, in Meta's own framing: the LLM is given a chance to potentially respond to an unsafe prompt in a safe way. A poorly-phrased question can receive a good answer, and input filtering alone would have blocked it.

Most production systems do both, and the reason is that they catch different failures. Input filtering catches intent. Output filtering catches what the model actually produced — which is not always what the prompt asked for.

Published metrics are recall and false positive rate, reported for output filtering. Both numbers matter, and they pull against each other: a classifier tuned for high recall flags more false positives, and one tuned for few false positives misses more.

Which one you prioritise is a product decision, not a technical one. A children's education platform and an internal engineering assistant should not sit at the same operating point.

The Taxonomy

Aligned to the standardised MLCommons hazards taxonomy, covering fourteen hazard categories plus code interpreter abuse.

Standardisation is the useful part. The categories are not Meta's invention — they come from a cross-industry taxonomy, which means a classification here is comparable to one from another system using the same taxonomy, and your policy can be written against a definition someone else maintains.

And the category set is customisable. The classification instruction is a prompt, and the categories it lists can be edited — removing ones irrelevant to your application, or narrowing the set to what you actually enforce.

That matters more than it sounds. A classifier checking fourteen categories on every request spends effort on thirteen you may not care about, and each one is an opportunity for a false positive. Narrowing the list reduces both cost and noise.

Verify the exact category codes and definitions from the model card before writing policy against them — the taxonomy is versioned, and the codes are what your logs will contain.

Specifications

Context window is not stated in the sources consulted. Confirm it from your own model listing before designing around a figure — a classifier's usable input length is what determines whether a long response can be checked in one pass.

Capabilities

Using Llama Guard 4 on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: meta-llama/Llama-Guard-4-12B

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="meta-llama/Llama-Guard-4-12B",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "meta-llama/Llama-Guard-4-12B",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-Guard-4-12B",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Set max_tokens low. The output is two lines; a large budget serves no purpose and a small one guards against a malformed response running long.

Parsing the Result

The function that turns two lines into a decision.

from dataclasses import dataclass, field


@dataclass
class Verdict:
    safe: bool
    categories: list[str] = field(default_factory=list)
    raw: str = ""


def classify(content: str) -> Verdict:
    """Classify content and return a structured verdict."""
    response = client.chat.completions.create(
        model="meta-llama/Llama-Guard-4-12B",
        messages=[{"role": "user", "content": content}],
        max_tokens=128,
        temperature=0,
    )

    raw = (response.choices[0].message.content or "").strip()
    lines = [line.strip() for line in raw.splitlines() if line.strip()]

    if not lines:
        raise ValueError("classifier returned nothing")

    first = lines[0].lower()

    if first == "safe":
        return Verdict(safe=True, raw=raw)

    if first == "unsafe":
        categories = []
        if len(lines) > 1:
            categories = [c.strip() for c in lines[1].split(",") if c.strip()]
        return Verdict(safe=False, categories=categories, raw=raw)

    # Neither word — do not guess.
    raise ValueError(f"unexpected classifier output: {raw[:200]}")

Three decisions in that function, and each one matters.

temperature=0. Classification has one correct answer. Sampling variance on a safety decision is variance in whether content is blocked, which is not a thing to leave to chance.

Raise on unexpected output rather than defaulting. A classifier that returns something other than safe or unsafe has failed, and the failure must be visible. Defaulting to safe silently disables your moderation; defaulting to unsafe silently blocks legitimate traffic. Both are worse than an exception.

Keep the raw output. When a decision is disputed, the exact string the classifier produced is the evidence.

Both Directions

The full pattern, and it is two calls rather than one.

def moderated_completion(user_message: str, *, model: str) -> str:
    """Filter the input, generate, then filter the output."""

    # Stage one — is the request acceptable?
    incoming = classify(user_message)
    if not incoming.safe:
        log_block("input", incoming.categories, user_message)
        return "I'm not able to help with that request."

    # Stage two — generate.
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": user_message}],
        max_tokens=4096,
    )
    answer = response.choices[0].message.content

    # Stage three — is what we produced acceptable?
    outgoing = classify(answer)
    if not outgoing.safe:
        log_block("output", outgoing.categories, answer)
        return "I'm not able to provide a response to that."

    return answer

Note that the two blocks are caught for different reasons — and logging which stage fired is what lets you tune later. A system blocking mostly at input has a user-intent problem; one blocking mostly at output has a model-behaviour problem, and they need different fixes.

Three model calls per user request is the cost of doing this properly. On a twelve-billion-parameter classifier that is a fraction of the generation cost, and it is the price of knowing what your system produced.

Log every block with its categories. Not to punish users — to find out whether your classifier is calibrated. A category firing constantly is either a real pattern in your traffic or a false-positive source, and you cannot tell which without the data.

Images

Natively multimodal, which is what replaced the separate vision model.

import base64
from pathlib import Path


def classify_image(path: str, caption: str = "") -> Verdict:
    """Classify an image, optionally with accompanying text."""
    encoded = base64.b64encode(Path(path).read_bytes()).decode("utf-8")

    content = [{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encoded}"}}]
    if caption:
        content.append({"type": "text", "text": caption})

    response = client.chat.completions.create(
        model="meta-llama/Llama-Guard-4-12B",
        messages=[{"role": "user", "content": content}],
        max_tokens=128,
        temperature=0,
    )

    return parse_verdict(response.choices[0].message.content or "")

Classify the image and its caption together where both exist. Text and image are each innocuous in many cases where the combination is not — and a system checking them separately misses exactly those cases.

This is the check that matters most on an image-generation path. A prompt that passed input filtering can still produce an image that should not be delivered, and the only way to know is to look at what was produced.

Narrowing the Categories

The customisation worth doing before you deploy at volume.

The classification instruction lists the categories to check, and it is a prompt. Removing categories your application does not enforce reduces both the classifier's work and your false positive rate.

Where that applies. An internal engineering assistant probably does not need election-integrity checking. A children's platform may want every category and a lower threshold. A code tool cares intensely about interpreter abuse and little about defamation.

Write the category list into your policy, version it, and log which version produced each decision. Six months from now, explaining why a piece of content was blocked requires knowing what the classifier was asked to check at the time.

Calibration Is Your Job

The most important operational point, and no model ships it for you.

Published metrics are recall and false positive rate. Those two numbers describe a trade-off, and where you sit on it is a decision about your product rather than about the model.

Build the evaluation set. A few hundred examples from your real traffic, labelled by a person: content that should be blocked, content that should pass, and the ambiguous cases in between.

Then measure. How much genuinely harmful content gets through, and how much legitimate content gets blocked. Both numbers, from your traffic, not from a benchmark.

And measure again after every change — a new category list, a new upstream model, a shift in who is using your product. Calibration is not a one-time task, and a classifier that was well-tuned a year ago may not be now.

Where It Fits

Input filtering on any public-facing path — before a prompt reaches a generative model, at a fraction of the generation cost.

Output filtering, catching what the model produced rather than what was asked.

Image moderation, including generated images, which the separate vision model previously handled.

Mixed text-and-image checking, where the combination is the risk rather than either part.

Multilingual moderation, across the languages the previous text model supported.

Alongside Llama 4 models specifically, sharing their tokenizer and vision encoder — though it works as a classifier for any model's input and output.

Not a conversational model. It answers in two lines and nothing else.

Not a replacement for policy. It applies a taxonomy; deciding what your product allows is a decision it cannot make for you.

Practical Notes

Set temperature=0 and max_tokens low.

Raise on unexpected output. Never default to safe or unsafe silently.

Filter both directions — they catch different failures.

Log which stage fired, with categories.

Classify images with their captions rather than separately.

Narrow the category list to what you actually enforce.

Build an evaluation set from your own traffic and measure recall and false positives.

Re-calibrate after any change to the categories, the upstream model, or your audience.

Confirm the context window from your model listing before relying on single-pass checking of long responses.

Limitations

It is a classifier, not a judge. It applies a taxonomy. Whether your product allows something is a policy decision the model does not make.

Recall and false positives trade against each other, and no setting optimises both. Where you sit is your decision.

A classification is not an explanation. Two lines of output contain no reasoning, and asking for it is asking the model to do something it was not trained for.

Context window is unstated in the available sources. Verify it before assuming a long response can be checked in one pass.

The taxonomy is versioned. Category codes and definitions can change between releases; log the version alongside the decision.

No additional pre-training was performed after pruning. That is an efficiency claim, and it also means capability is bounded by what survived the pruning of a model built for a different purpose.

Twelve billion parameters. Capable and not infallible — adversarial content designed to evade classification is a category of input every safety model struggles with.

A custom commercial licence, not Apache or MIT.

An April 2025 model. Threat patterns move faster than model releases, and a classifier trained on last year's taxonomy is checking last year's definitions.

It does not catch everything, and it blocks things it should not. Every moderation system has both failure modes. The question is whether you have measured yours.