phi-4

Phi-4

Fourteen billion parameters, and a thesis about what that is enough for.

The Student Exceeded the Teachers

The result that justifies the whole approach.

Phi-4's training data was substantially synthetic, generated by larger models — and on STEM question answering, the fourteen-billion-parameter student exceeds the models that taught it.

That should not obviously work. A model trained on another model's output would be expected to approximate it, not surpass it.

Microsoft's explanation is the Phi thesis, held across four generations: carefully curated and synthetically generated training data delivers outsized capability per parameter.

What the synthetic data does that scraped text does not. A generated dataset can be designed — textbook-like explanations of mathematics, worked reasoning, code with its rationale, coverage of a concept at graded difficulty. Scraped text contains those things incidentally, buried in a great deal that teaches nothing.

So the teacher's output is not what the student learns from. The student learns from a curriculum the teacher was used to write — and a good curriculum can produce a student who outperforms its author on the subject it covers.

The scope matters. This is a claim about STEM reasoning, mathematics, and code. It is not a claim about world knowledge, breadth, or multilingual capability, and Microsoft do not make one.

16K Was Trained, Not Scaled

Most long-context claims come from positional scaling applied at inference. This one does not, and the technical report documents the work.

The base architecture has a default context length of 4,096.

A dedicated midtraining stage extended it to 16K, and the configuration is published:

Read the 70% recall portion. Training on long data alone causes a model to forget what it learned at short lengths. Mixing in the original pretraining distribution is what preserves it — and publishing the ratio is more useful than most papers manage.

The RoPE frequency increase is the mechanical half. Rotary embeddings encode position by rotating representations at a frequency derived from a base value; raising that base slows the rotation and extends the range before positions become indistinguishable.

And the learning rate drop by a factor of ten is what keeps midtraining from disturbing what pretraining established.

A Finding Worth Taking Elsewhere

The technical report reports an ablation whose conclusion generalises beyond this model.

We try data sources that are inherently long context, and compare them with artificially created long context data where samples are padded together to fill the sequence. We observe the former to perform better.

Naturally long documents beat concatenated short ones.

Why that is not obvious. Padding short samples together produces sequences of the right length, costs nothing to construct, and is what a great many long-context training pipelines do.

Why it fails. A concatenated sequence has no long-range structure — the tokens at position 12,000 have nothing to do with the tokens at position 200. A model trained on that learns to handle long sequences without learning to use them.

A genuinely long document does have that structure, and learning to track it is the capability you wanted.

And a second finding that saved them money

We observe a high rank correlation between the performance of the 7B and 14B models on different data mixtures, given a large enough distance between the data mixtures.

Which allowed the ablations to run at 7B and the findings to transfer.

That is a practical research result with a general lesson: if a smaller model ranks data mixtures the same way a larger one does, you can search the mixture space at a fraction of the cost. The caveat in their own sentence matters — given a large enough distance between the mixtures — meaning the correlation holds for coarse comparisons and not for fine ones.

Specifications

The architecture closely follows the previous generation's medium model, with one change the report names: the tokenizer moved to tiktoken.

1,920 H100s for 21 days is a concrete figure most releases do not publish. It puts the model's cost in perspective against its size — this is not a cheap fourteen-billion-parameter model; it is an expensively-trained one.

MIT licence. Fully permissive — commercial use, modification, and redistribution with no conditions, no attribution requirement, and no user threshold. Uncommon at this level of capability.

Post-training used DPO rather than reinforcement learning from human feedback — a newer alignment method than the generation of models released alongside it.

Capabilities

Microsoft note the prompt format specifically: the model is best suited for prompts in the chat format. Raw completion-style prompting is not what it was tuned for.

⚠️ 16,384 Tokens

The constraint that decides which workloads this model fits, and it is short by current standards.

Sixteen thousand tokens is roughly forty to sixty pages of prose, or a moderate source file with room for an answer.

What it does not hold. A repository. A document archive. A long agent session with accumulated tool output. A conversation that runs for hours.

And it is honest. A model advertising 128K through inference-time scaling may retrieve poorly across most of it; this one trained at its stated length with a documented data mixture. Sixteen thousand tokens that work are worth more than a hundred and twenty-eight thousand that degrade.

Plan retrieval rather than volume. On a corpus larger than the window, the design is a retrieval layer selecting what matters and this model reasoning over it — which suits the model's strengths anyway, since reasoning is what it was built for.

⚠️ Primarily English

Multilingual data constitutes about 8% of the overall training data.

Microsoft state the model is intended for applications primarily in English, and the eight percent figure is the honest quantification of what that means.

What it will do. Produce text in other languages, with quality that was neither measured nor optimised.

What it will not do reliably. Match a model trained with multilingual coverage as a priority.

If your users write in Arabic, French, or anything outside English, this is a reason to choose a different model rather than a caveat to manage — and there are models in this catalogue at similar size with a hundred languages or more.

Where it does apply. An English-language product, or an internal system where the interface language is fixed regardless of the user's own.

Where It Was Built to Be Used

Microsoft name three primary use cases, and the list is unusually specific.

Memory and compute constrained environments.

Latency bound scenarios.

Reasoning and logic.

Read the first two together. Fourteen billion parameters with no reasoning pass produces predictable latency and a modest footprint — which is the argument for choosing it over something larger when the task is within reach.

And the third is where the Phi thesis pays. Mathematics, code, and structured reasoning are what the synthetic curriculum was designed to teach, and they are where this model punches above its parameter count.

What is conspicuously absent from that list. Broad world knowledge. Multilingual work. Long-context analysis. Creative writing. Microsoft do not claim them, and the model's design does not support them.

Using Phi-4 on DEVUP AI

Base URL: https://api.devupai.com/v1 · Model ID: microsoft/phi-4

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="microsoft/phi-4",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Node.js

import DevupAI from "devupai";

const client = new DevupAI({
  apiKey: process.env.DEVUP_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    model: "microsoft/phi-4",
    messages: [{ role: "user", content: "Hello world!" }],
    max_tokens: 1024,
  });

  console.log(response.choices[0].message.content);
}

main();

cURL

curl -X POST "https://api.devupai.com/v1/chat/completions" \
  -H "Authorization: Bearer $DEVUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "microsoft/phi-4",
    "messages": [
      { "role": "user", "content": "Hello world!" }
    ],
    "max_tokens": 1024
  }'

Reasoning and Mathematics

The workload this model was curated for.

SYSTEM = """Solve the problem step by step.

Rules:
- Show each step of your working, including the arithmetic.
- State any assumption you make explicitly before using it.
- End with your final answer on its own line, prefixed with 'ANSWER: '.
- If the problem is underspecified, say what is missing rather than assuming a value."""

response = client.chat.completions.create(
    model="microsoft/phi-4",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": problem},
    ],
    max_tokens=4096,
    temperature=0.2,
)

Step-by-step working has to be asked for. There is no thinking mode here — reasoning happens inside the visible answer, and a model told to answer will answer without showing the path.

"State any assumption explicitly" is the clause that makes the output checkable. Most wrong answers to well-posed problems come from an unstated assumption, and surfacing it turns a wrong result into a diagnosable one.

Low temperature. Mathematics has one correct answer, and variance is not creativity here.

Code

SYSTEM = """You write production code.

Rules:
- Handle errors explicitly. Never swallow an exception.
- Validate inputs at the boundary, and raise rather than defaulting on invalid input.
- Add a brief docstring stating what the function does and what it raises.
- If the requirements are ambiguous, say which decision you made and why, in a comment."""

response = client.chat.completions.create(
    model="microsoft/phi-4",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": requirement},
    ],
    max_tokens=4096,
    temperature=0.2,
)

Code is one of the three things the synthetic curriculum targeted, alongside mathematics and reasoning — and it is where fourteen parameters go further than the count suggests.

The ambiguity clause matters on a model this size. A larger model may infer your intent correctly; a smaller one is more likely to pick one reading and proceed. Asking it to name the decision surfaces a choice you might otherwise discover in review.

Within 16,384 tokens, a request and its answer both fit comfortably for a function or a module. A whole file plus a substantial rewrite is tighter — check finish_reason.

Grounded Question Answering

Where a 16K window and a June 2024 cutoff meet.

from datetime import date

GROUNDED = f"""Answer only from the material provided below. Quote the passage supporting each
statement. Where the material does not contain the answer, say so plainly and stop — do not fill the
gap from general knowledge.

Your training data ends in June 2024. The current date is {date.today().isoformat()}. Treat anything
you recall about software versions, APIs, prices, or regulations as potentially out of date."""

response = client.chat.completions.create(
    model="microsoft/phi-4",
    messages=[
        {"role": "system", "content": GROUNDED},
        {"role": "user", "content": f"{documents}\n\nQuestion: {question}"},
    ],
    max_tokens=2048,
    temperature=0.2,
)

June 2024 is newer than several larger models in this catalogue, which is worth noting — the Phi line shipped later in its generation than the Llama 3.x family, and its knowledge reflects that.

It is still two years old. Ground anything factual, and with only 16,384 tokens available, that means selecting what to supply rather than supplying everything.

Retrieval quality matters more here than on a long-context model. When the window is small, the retrieval layer decides what the model gets to read — and a model reasoning well over the wrong passages produces a confident wrong answer.

Budgeting Inside 16,384

The calculation worth building into any path with variable input.

MAX_CONTEXT = 16_384
SAFETY_MARGIN = 256


def output_budget(estimated_input_tokens: int, want: int = 2048) -> int:
    """How many output tokens are actually available for this request."""
    remaining = MAX_CONTEXT - estimated_input_tokens - SAFETY_MARGIN

    if remaining <= 0:
        raise ValueError(
            f"input of ~{estimated_input_tokens:,} tokens leaves no room within {MAX_CONTEXT:,}"
        )

    return min(want, remaining)

On a 16K window this check is not optional. A model with 128,000 tokens absorbs an oversized prompt; this one does not, and a document that fits comfortably in most of this catalogue will not fit here.

Check finish_reason on every generation request. A truncated answer reads as a short answer until someone looks.

MIT, and What That Removes

Worth a section, because it is rare at this level.

No conditions. No attribution requirement. No monthly-active-user threshold. No separate commercial agreement. No prohibited-use policy layered on top.

Against the models it sits beside in this catalogue — custom community licences with numbered thresholds, agreements specific to a version, terms that change between sizes in the same family — MIT is a different category of answer to the question your legal team will ask.

And it applies to fine-tunes. A model you train on this base is yours under the same terms, which is not true of every open-weight release.

Fourteen billion parameters is also the size where fine-tuning is genuinely accessible — a full fine-tune on a single large card, LoRA on much less. The licence and the footprint together make this a practical base rather than a theoretical one.

Where It Fits

Mathematics, logic, and structured reasoning — what the synthetic curriculum was built to teach.

Code generation and review, within a 16K window.

Memory and compute constrained deployment, named by Microsoft as a primary use case.

Latency-bound paths, with no reasoning pass and fourteen billion parameters.

Grounded question answering over a retrieved, focused context rather than a large one.

Fine-tuning, with MIT licensing and an accessible footprint.

English-language applications.

Not for long-context work. 16,384 tokens, and honestly stated.

Not for multilingual products. Eight percent of the training data.

Not for broad world knowledge. The curriculum optimised reasoning, not coverage.

Not for images. Text only — the Phi family has multimodal members, and this is not one.

Practical Notes

Use the chat format. Microsoft say so explicitly.

Ask for step-by-step working — there is no thinking mode to produce it.

Require explicit assumptions on reasoning tasks; that is where wrong answers come from.

Compute the output budget from what the input leaves. 16,384 is not much.

Check finish_reason on every generation request.

Ground factual work, and invest in retrieval quality — the window is small enough that selection matters.

Keep it on English unless you have tested your language yourself.

Consider fine-tuning: MIT licence, accessible size, and a base built for reasoning.

Route long-context and multilingual work to a different model.

Limitations

16,384-token context. Short by current standards, and honestly trained rather than scaled.

Approximately 8% multilingual training data. Primarily English, and Microsoft say so.

Text only. No image, audio, or video input; other members of the Phi family handle those.

No reasoning mode. Reasoning happens in the visible answer and has to be prompted for.

Knowledge ends June 2024 — newer than several larger models here, and still two years back.

Fourteen billion parameters. Strong on its targeted domains, bounded elsewhere.

Breadth was not the objective. The synthetic curriculum optimised reasoning, mathematics, and code; world knowledge and general coverage were not the goal and do not match a model trained for them.

Microsoft explicitly place evaluation on the deployer for high-risk scenarios — accuracy, safety, and fairness are named as things to assess in your own context before use.

A December 2024 model. Mature and well understood, and the field has moved — choose it for reasoning per parameter, an unconditional licence, and a small footprint, not for frontier capability.