gemma-3-27b-it

Gemma 3 27B Instruct

The largest model in its family, sized so that it fits on one card.

Fifty-Four Gigabytes to Fourteen

The number that defines this model's position, and it is published by Google rather than measured by the community.

That drop puts the largest Gemma 3 on a single desktop RTX 3090.

And the method matters. This is quantisation-aware training, not a conversion applied to finished weights — the model learned at reduced precision, so quality stays close to bfloat16 rather than degrading toward it.

One caveat Google state plainly: that figure covers the weights only. Running the model also needs VRAM for the KV cache, which grows with context length. A 24 GB card holding 14.1 GB of weights has roughly ten left, and how far that stretches depends entirely on how much context you configure.

Which makes the attention design below the other half of the story.

5:1, and the Number It Produces

Gemma 3 interleaves five local attention layers for every global one, and Google published what that buys rather than merely describing it.

KV-cache memory overhead falls from 60% to under 15%.

That is an unusually specific claim — most architectural descriptions say a design "reduces memory cost" and leave you to infer by how much.

How it works. Local layers attend to a 1,024-token span and cost the same regardless of how much input arrived. Global layers handle the extended context and cost what that implies. Five in six layers never build a cache that grows with the input.

Put it beside the quantisation figure. Fourteen gigabytes of weights plus a cache that stays under fifteen percent overhead is what makes 128K context on a consumer card arithmetic rather than aspiration.

The Window Was Extended, Not Trained

Worth knowing for the same reason it matters on any long-context model.

Pre-training ran at 32K sequences. The 4B, 12B, and 27B were then extended to 128K; the 1B stayed at 32K.

So 128K is a scaled window rather than a natively trained one — and Gemma's technical reporting puts the cost at under five percent perplexity penalty, which is a figure most long-context extensions leave undocumented.

Under five percent for four times the length is a good trade. It is also not zero, and behaviour at 120,000 tokens is worth measuring on your own data rather than assuming it matches behaviour at 20,000.

⚠️ The Output Ceiling Is 8,192

A hard limit, and one out of proportion with the input window.

128K in. 8,192 out. One sixteenth.

What that rules out. Any single request producing long-form output — a full document translation, a complete transcription of a large scan set, an extended report, a long code file.

What it suits perfectly. Analysis, extraction, summarisation, classification, and answering — all the tasks where a large input produces a modest output.

And the practical consequence for generation work: chunk by output rather than by input. Send the full context every time, ask for one section per request, and assemble the result yourself. The input window is generous enough that repeating the context across requests costs less than you might expect.

Images: 256 Tokens, and Pan & Scan

Every image is normalised to 896 × 896 and encoded to exactly 256 tokens.

That fixed budget is the efficiency and the problem. An image costing 256 tokens is why multimodal input is affordable at all. Squeezing a 3,000-pixel-wide document scan into an 896-pixel square destroys exactly what you needed from it.

Pan & Scan is the answer, and it runs at inference rather than being baked into the weights: an adaptive windowing algorithm that segments non-square images into 896 × 896 crops, processing each separately.

Each crop is another 256 tokens. Four crops means 1,024 tokens for one image. On a batch of documents, the crop count — not the image count — is what determines your context consumption.

The vision encoder is frozen and shared across the 4B, 12B, and 27B. Moving up the family improves reasoning over images, not perception of them. The eyes are the same at every size.

What the Extra Fifteen Billion Buys

Against the 12B in the same family, on coding:

Read the spread, not the scores. Two and a half points on HumanEval, one and a half on MBPP — those benchmarks are near saturation at this scale and have stopped discriminating.

Seven points on LiveCodeBench is the real difference. That benchmark uses recent problems specifically to resist contamination, and it is the one still separating the tiers.

So the decision between 12B and 27B rests on difficulty. If your workload looks like HumanEval, the smaller model saves you fifteen billion parameters for almost nothing. If it looks like LiveCodeBench, the gap is worth paying for.

Specifications

The licence is Google's own, not Apache or MIT. Commercial use is permitted subject to its terms and a prohibited-use policy. Read both against your deployment.

Official quantisations: int4, int4 per-block, and switched fp8, all through quantisation-aware training.

Capabilities

It Answers Directly

No thinking mode, no effort parameter, no reasoning_content field.

max_tokens covers the answer alone — nothing shares it. Which matters more here than usual, because the ceiling is 8,192 and a reasoning trace would consume a meaningful share of it.

Latency tracks input and output length, not how hard the model judged the problem. On an interactive endpoint that predictability is frequently worth more than depth.

Post-training used knowledge distillation and reinforcement learning, so reasoning behaviour is distilled into the answers rather than exposed as a trace.

Using Gemma 3 27B on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: google/gemma-3-27b-it

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="google/gemma-3-27b-it",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "google/gemma-3-27b-it",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-3-27b-it",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Working Inside 8,192 Output Tokens

The pattern that makes long-form generation possible on a model with a small output ceiling.

from pathlib import Path

SECTIONS = [
    "Executive summary",
    "Findings",
    "Risk assessment",
    "Recommendations",
]


def write_section(name: str, context: str) -> str:
    """Generate one section against the full context, staying inside the ceiling."""
    response = client.chat.completions.create(
        model="google/gemma-3-27b-it",
        messages=[
            {
                "role": "system",
                "content": (
                    "You are writing one section of a report. Write only the section requested. "
                    "Do not summarise other sections, do not add a conclusion, do not repeat the "
                    "brief."
                ),
            },
            {"role": "user", "content": f"Source material:\n\n{context}\n\nWrite the '{name}' section."},
        ],
        max_tokens=4096,
        temperature=0.4,
    )

    choice = response.choices[0]

    if choice.finish_reason == "length":
        raise ValueError(f"section '{name}' hit the output ceiling — narrow the brief")

    return f"## {name}\n\n{choice.message.content}"


source = Path("findings.txt").read_text(encoding="utf-8")
document = "\n\n".join(write_section(name, source) for name in SECTIONS)
Path("report.md").write_text(document, encoding="utf-8")

The full source goes into every request. With 128K of input available that is comfortable, and it means each section is written with complete context rather than from a summary of it.

The finish_reason check is mandatory here. At 8,192 tokens a section that runs long stops mid-sentence, and it looks like a short section until someone reads it.

Reading a Document

Where Pan & Scan decides the outcome.

import base64

with open("invoice.png", "rb") as handle:
    encoded = base64.b64encode(handle.read()).decode("utf-8")

response = client.chat.completions.create(
    model="google/gemma-3-27b-it",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}},
                {
                    "type": "text",
                    "text": (
                        "Transcribe every line item with its quantity and amount, and report the "
                        "currency exactly as printed. Mark anything you cannot read cleanly as "
                        "unreadable rather than reconstructing it."
                    ),
                },
            ],
        }
    ],
    max_tokens=4096,
)

Send the page at full resolution. Downscaling before upload is the most common way to make this model read badly — the encoder resizes anyway, and Pan & Scan works from what you sent. A page shrunk to 1,024 pixels wide has already lost the characters.

If transcription disappoints, check the image before the prompt. Pan & Scan activates on aspect ratio; a near-square page may be processed whole and squeezed into 256 tokens.

Count crops, not images, when budgeting a batch. Twenty tall pages at four crops each is eighty images' worth of tokens.

Self-Hosting

The design point: the most capable model you can run on a single GPU or TPU host.

Three official quantisations through quantisation-aware training — int4, int4 per-block, and switched fp8 — with the int4 build published as GGUF for llama.cpp.

Text-only and image-input paths are both supported in llama.cpp, with separate invocations.

14.1 GB of weights on a 24 GB card leaves roughly ten for the KV cache. How much context that buys is the calculation worth doing before you configure --ctx-size — and the 5:1 attention ratio is what makes the answer larger than it would otherwise be.

Transformers requires a Gemma 3-specific version if you are on the library path rather than a serving framework. Check the constraint before assuming an existing installation works.

Where It Fits

Single-GPU deployment at maximum capability — which is the model's stated purpose rather than a side effect.

Document and screenshot reading, where Pan & Scan is worth a measured 8 to 17 percent on tasks requiring text extraction from images.

Long-input analysis — 128K in, a modest answer out, which is exactly the shape the output ceiling suits.

Multilingual work across 140+ languages with a vocabulary built for diverse scripts.

Interactive assistants, where no reasoning pass means predictable latency.

Harder coding work, where the seven-point LiveCodeBench margin over the 12B is the reason to be here rather than there.

Not for long-form generation in a single request. 8,192 tokens is the ceiling.

Not for audio or video.

A March 2025 model. Mature, widely supported, and stable — and later generations in this family changed the architecture substantially, including sparse variants, configurable reasoning, and a permissive licence. Worth comparing if neither an existing integration nor the licence ties you here.

Practical Notes

Budget VRAM for weights and cache separately. 14.1 GB is the first number, not the total.

Send images at full resolution.

Count Pan & Scan crops rather than images when estimating context.

Check finish_reason on any generation task — the ceiling is 8,192.

Chunk long-form work by output section, sending the full context each time.

Weigh LiveCodeBench rather than HumanEval when choosing between this and the 12B.

Measure long-context behaviour past 100K on your own data.

Read the Gemma licence and prohibited-use policy against a commercial deployment.

Limitations

8,192-token output ceiling — one sixteenth of the input window, and the binding constraint on generation work.

128K context is extended, not natively trained. Pre-training ran at 32K, with a documented perplexity cost under five percent.

Every image costs 256 tokens minimum, and more once Pan & Scan activates.

The vision encoder is frozen and shared across sizes. Moving up the family improves reasoning over images, not perception of them.

No reasoning trace. Improvements from post-training arrive inside the answer, so there is nothing to inspect when one is wrong.

No audio or video input.

Gemma Terms of Use, not Apache or MIT, with a prohibited-use policy attached.

Text output only. It reads images; it does not generate them.

Quantisation reduces weights, not the KV cache. A 24 GB card running the int4 build still needs memory for context, and that is what limits your usable window.

Confident answers with no visible reasoning. Ground factual work and require citations where correctness matters.