gemma-3-4b-it

Gemma 3 4B Instruct

Two point six gigabytes, a 128K window, and the same eyes as a model seven times its size.

The Vision Encoder Does Not Shrink

The most consequential fact about this model, and it is easy to miss.

The SigLIP vision encoder is frozen and shared across the 4B, 12B, and 27B. Identical weights. Not trained alongside the language model, not scaled with it.

Which means visual perception is the family's, not the size's.

A four-billion-parameter model reads an image with exactly the encoder the 27B uses.

What scales is the reasoning over what it saw, not the seeing.

Which sorts the workload cleanly. Transcribe this receipt is perception — this model does it as well as any model in the family. Reconcile three receipts against a contract and explain the discrepancy is language, and that is where four billion parameters is the limit.

Two Point Six Gigabytes

And the method matters. Quantisation-aware training rather than a conversion applied afterwards — the model learned at reduced precision, so quality stays close to bfloat16 rather than degrading toward it.

What 2.6 GB puts this model on:

A phone. Not a flagship laptop — a phone.

A single-board computer with adequate memory.

An integrated GPU, where dedicated VRAM does not exist.

Alongside everything else, on a machine already running your application, your database, and your browser.

One caveat Google state directly: that figure covers weights only. The KV cache needs VRAM too, and it grows with context length. On a 4 GB budget, how much context you configure is the whole question — which makes the attention design below the other half of the arithmetic.

5:1, With the Number Attached

Gemma 3 interleaves five local attention layers for every global one, and Google published what it saves rather than merely describing it.

KV-cache memory overhead falls from 60% to under 15%.

Local layers attend to a 1,024-token span at constant cost regardless of input length. Global layers handle the extended context.

On a model this small that ratio is what makes the window usable. Five in six layers never build a cache that grows — which is the difference between 128K context being a specification and being something a 4 GB device can actually hold.

Google's Own Claim

From the Gemma 3 release material: Gemma 3 4B is competitive with the previous generation's 27B instruction-tuned flagship.

Roughly a seventh of the parameters, matching the model that headlined the generation before it.

That is a vendor claim about their own models, which is the cleanest kind — and it is testable: if you have a workload currently running on a Gemma 2-class model, pointing it here costs one changed identifier.

⚠️ Where Four Billion Parameters Costs You

Measured, across the family, on coding:

Read the gaps between adjacent sizes, not the absolute numbers.

1B to 4B is enormous — thirty points on HumanEval, eighteen on LiveCodeBench. The 1B is a different class of model.

4B to 12B is the large step — fourteen points on HumanEval, nine on LiveCodeBench.

12B to 27B is small on the easy benchmarks and seven points on the hard one.

Which tells you where the decision actually is. The meaningful jump in this family is 4B to 12B, not 12B to 27B. If this model is not enough, the next step up is where most of the remaining capability lives — and it costs three times the parameters rather than seven.

And LiveCodeBench at 23.0% is the honest ceiling. That benchmark resists contamination by using recent problems, and it is the one that separates a model you can rely on for hard code from one you cannot.

Specifications

The licence is Google's own, not Apache or MIT. Commercial use is permitted subject to its terms and a prohibited-use policy. Read both against your deployment.

Official quantisations: int4, int4 per-block, and switched fp8, all through quantisation-aware training, with GGUF published for llama.cpp.

Capabilities

The Output Ceiling Is 8,192

128K in. 8,192 out. One sixteenth, and the same ceiling the 27B carries.

Which suits this model's workloads well. Extraction, classification, summarisation, and answering all produce modest output from large input — exactly the shape the ratio favours.

And rules out long-form generation in a single request. That is not what a four-billion-parameter model is for anyway.

No reasoning trace shares the budget, which makes 8,192 a real 8,192 rather than a figure divided between thinking and answering.

Using Gemma 3 4B on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: google/gemma-3-4b-it

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="google/gemma-3-4b-it",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "google/gemma-3-4b-it",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-3-4b-it",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

A Document Pipeline at Volume

Where the shared vision encoder and the small footprint combine into something the larger models cannot offer.

import base64
import json
from pathlib import Path

SCANS = Path("receipts")
OUTPUT = Path("extracted")
OUTPUT.mkdir(exist_ok=True)

PROMPT = (
    "Transcribe this receipt. Return a single JSON object with keys: merchant, date, currency, "
    "total, line_items (an array of {description, quantity, amount}). Use null for any field the "
    "receipt does not show. Where a value is present but you cannot read it cleanly, use the string "
    "\"UNREADABLE\" rather than guessing. Reply with JSON only."
)


def encode(path: Path) -> str:
    return base64.b64encode(path.read_bytes()).decode("utf-8")


for path in sorted(SCANS.glob("*.jpg")):
    target = OUTPUT / f"{path.stem}.json"
    if target.exists():
        continue  # already done

    try:
        response = client.chat.completions.create(
            model="google/gemma-3-4b-it",
            messages=[
                {
                    "role": "user",
                    "content": [
                        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encode(path)}"}},
                        {"type": "text", "text": PROMPT},
                    ],
                }
            ],
            max_tokens=2048,
            temperature=0.1,
        )
        data = json.loads(response.choices[0].message.content)
    except Exception as exc:
        print(f"{path.name}: {type(exc).__name__} — {exc}")
        continue  # one bad file must not end the run

    target.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")

    unreadable = json.dumps(data).count("UNREADABLE")
    print(f"{path.name}: {unreadable} unreadable field(s)")

The perception here is the flagship's. That is the whole argument for running a document pipeline on a four-billion-parameter model — you are not trading reading accuracy for cost, only reasoning capacity you were not using.

Three lines carry the operational weight. Skip-if-exists so a failed run resumes; catch-and-continue so one corrupt file does not end the job; and counting UNREADABLE, which gives you the escalation list for free.

Escalate on the count, not on a guess. Files with unreadable fields go to a larger model; the rest are done.

Pan & Scan, and the Token Arithmetic

Every image becomes 256 tokens at 896 × 896. Pan & Scan segments a non-square image into multiple 896 × 896 crops, each processed separately.

Each crop is another 256 tokens.

Count crops, not images, when budgeting. Fifty pages is not fifty images — it is potentially two hundred, and on a device with limited memory that is the number that matters.

Send pages at full resolution. Downscaling before upload is the most common way to make this model read badly. The encoder resizes anyway, and Pan & Scan works from what you sent — a page shrunk to 1,024 pixels wide has already lost the characters.

If a document reads poorly, check the image before the prompt. Pan & Scan activates on aspect ratio; a near-square page may be processed whole and squeezed into 256 tokens.

Running It on the Edge

This is where the model is unusual rather than merely small.

2.6 GB of weights at int4 through quantisation-aware training, published as GGUF for llama.cpp.

Both text-only and image-input paths are supported in llama.cpp, with separate invocations.

Budget for the cache separately. 2.6 GB is the weights; context needs more, and the 5:1 attention ratio is what keeps that number smaller than it would otherwise be. On a 4 GB device, the context length you configure is the variable that decides whether it runs.

Transformers requires a Gemma 3-specific version on the library path. Check the constraint before assuming an existing installation works — the pipeline task is image-text-to-text.

What this makes possible. Offline document reading on a phone. A kiosk that processes forms without a network. An air-gapped deployment where the model runs beside the data it is not permitted to send anywhere. Those are not use cases a 27-billion-parameter model serves at any price.

Where It Fits

On-device and edge multimodal work — 2.6 GB with the family's full vision capability.

High-volume document and image pipelines, where per-item cost decides whether a corpus gets processed at all.

Offline and air-gapped deployment, where the data cannot leave the device.

The first tier of a two-model pipeline, escalating flagged items to a larger sibling — same prompt, same request shape, one changed identifier.

Interactive assistants on modest hardware, with no reasoning pass and predictable latency.

Multilingual work across 140+ languages.

Not for hard reasoning or difficult code. LiveCodeBench at 23.0% is the honest ceiling, and the 12B is where the meaningful step up lives.

Not for long-form generation. 8,192 output tokens.

Not for audio or video.

Practical Notes

Send images at full resolution — the encoder is the flagship's and it needs the pixels.

Count Pan & Scan crops rather than images when budgeting context.

Use an explicit UNREADABLE marker rather than allowing reconstruction, and count it as your escalation signal.

Build batch runs with skip-if-exists and catch-and-continue from the start.

Budget VRAM for weights and cache separately on edge deployment.

Keep reasoning tasks off this tier — the perception does not degrade here, the language does.

Step up to the 12B rather than the 27B when this is not enough. That is where the gap closes.

Read the Gemma licence and prohibited-use policy against a commercial deployment.

Limitations

Four billion parameters of language capability. The vision is the family's; the reasoning over it is not. LiveCodeBench at 23.0% is the measured ceiling on hard code.

8,192-token output ceiling — one sixteenth of the input window.

128K context is extended, not natively trained. Pre-training ran at 32K, with a documented perplexity cost under five percent.

Every image costs 256 tokens minimum, and more once Pan & Scan activates.

Pan & Scan is aspect-ratio triggered. A near-square document may be processed whole and read poorly.

No reasoning trace. There is nothing to inspect when an answer is wrong.

No audio or video input.

Gemma Terms of Use, not Apache or MIT, with a prohibited-use policy attached.

Quantisation reduces weights, not the KV cache. On a small device, configured context length is what decides whether the model runs.

A March 2025 model. Later generations in this family changed the architecture substantially and moved to a permissive licence — worth comparing if neither an existing integration nor the licence ties you here.

Confident answers with no visible reasoning. On a small model that matters more, not less — ground factual work and require an explicit way for it to say it does not know.