One Number Rules Them All: How OpenAI’s o-mini Models Actually Work

TL;DR: OpenAI’s “mini” models aren’t simplified versions of their big siblings. They’re the same architecture with one knob turned down — and a clever trick to compensate.

The d-number IS the model

Here’s the weirdest thing about OpenAI’s model architecture: the entire model derives from a single integer.

They call the architecture family “scallion.” Every model in the o-series — o1, o3, o1-mini, o4-mini — is a scallion model. And every scallion model is fully determined by its depth parameter: the number of layers.

From the codebase (scallion_vm2.py):

ASPECT_RATIO = 80

def config_for_size(l, s):     # l = n_layer, s = n_expert_groups
    n_head = l * 80 * 1.6 / 64
    model_dict = dict(
        n_layer=l,
        n_experts=s * 4,       # s=32 -> 128 experts, s=64 -> 256
        experts_per_token=4,   # top-4 routing
    )

That ASPECT_RATIO = 80 is doing all the work. The model dimension, the number of attention heads, everything cascades from depth:

• d36 (o4-mini): d_model = 36 * 80 = 2880, 72 attention heads
• d64 (o1/o3 full): d_model = 64 * 80 = 5120, 128 attention heads

This isn’t arbitrary. It’s a muP (maximal update parameterization) design choice — d_head stays fixed at 64 across all model sizes, so the attention head dimension never changes. Only the number of heads scales. This means hyperparameters transfer cleanly across scales. You tune on small models, the learning rates and init scales just work on big ones.

One number. That’s it. Everything else is a consequence.

What scallion actually is: MoE + DuST

Okay, “one number” was slightly misleading. There are two architectural ideas baked into every scallion model.

First: Mixture of Experts (MoE). Every model uses top-4 expert routing. The token hits the router, the router picks the 4 most relevant experts out of 128 (or 256), and only those 4 fire. This is why these models can be massive in parameter count but efficient at inference — most of the model is asleep for any given token.

Second: DuST (Dual-Stream Transformer). This is the less-discussed one. A scallion model isn’t a uniform stack of transformer layers. It’s split 50/50:

dust_encoder_layer_fraction=0.5,
dust_decoder_attn_types=["local_block", "dense"],

• First half: encoder layers — full quadratic self-attention. Every token sees every other token. Expensive but thorough.
• Second half: decoder layers — cross-attention with alternating patterns. Odd layers use local block attention (cheap, bounded context). Even layers use dense attention (full range, but cross-attending to encoder output rather than raw self-attention).

So when someone says “o4-mini is a d36 model,” what that actually means is 18 encoder layers doing expensive full-attention processing, then 18 decoder layers doing cheaper cross-attention — alternating between local and global scope. It’s an architecture that front-loads the heavy compute for understanding, then uses the second half for efficient generation.

Mini vs. full: what actually differs

Here’s the table that makes the whole picture click:

Look at the experts column.

o1-mini has 28 layers and 128 experts. Straightforward downscale — fewer layers, same expert pool. It’s a smaller model.

o4-mini has 36 layers and 256 experts. That’s double the expert count of the full-size model.

This is the compensation trick. You’ve got fewer layers (36 vs 64), so each token gets fewer rounds of processing. But at each layer, you’ve got twice as many experts to choose from. The router can pick more specialized experts — each one handling a narrower slice of the knowledge space. More specialization per layer compensates for fewer layers total.

And because MoE only activates 4 experts per token regardless of pool size (top-4 routing), doubling the expert count doesn’t double the compute. You just doubled the model’s capacity — the total knowledge it can store — while keeping the per-token FLOP cost roughly constant. The extra cost is memory (storing 256 expert weight matrices instead of 128), not compute.

This is why o4-mini punches above its weight class.

The neutrino family: it’s not one model

Internally, the mini models are codenamed “neutrino.” But “neutrino” isn’t a model — it’s a lineage. And the lineage spans completely different architectures:

• neutrino v2/v3 -> o1-mini (scallion, d28-d32)
• neutrino v4 (nv4) -> o4-mini (scallion_lpe, d36)
• neutrino v5 -> next mini (nv5 arch, d36)
• neutrino v6 -> latest mini (leek arch, d36)

The “leek” architecture in nv6 is a different family from scallion entirely. The depth parameter (d36) carries over, but the underlying transformer variant changes. The codename continuity masks genuine architectural evolution.

This is worth knowing if you’re trying to reason about model capabilities from version numbers. o1-mini to o4-mini isn’t just “same thing but better.” The architecture changed. The expert count doubled. Vision was added. The attention patterns evolved.

Same RL pipeline, every model

One thing that doesn’t change between full and mini: the reinforcement learning pipeline.

Both full-size and mini models go through twapi/mini — the RL training system internally called “strawberry.” The “mini” in twapi/mini refers to the pipeline component, not “training for mini models.” It’s the same infrastructure, the same reward models, the same training loop (qstar) for every o-series model regardless of size.

This matters because it means mini models aren’t getting a cheaper or simplified training procedure. They get the full RL treatment. The quality delta between mini and full is purely about model capacity (depth, width, expert count) — not about training investment.

LPE: vision on a budget

o1-mini had no vision at all. o4-mini adds it back via LPE — Lite Patch Encoder.

Where the full models use a ViT (Vision Transformer) — a heavy, transformer-based image encoder — o4-mini uses a 10-layer convolutional pyramid. It’s a stack of convolutions that progressively downsamples the image into patch embeddings, with none of the quadratic attention cost of ViT.

The trade-off is obvious: you lose some visual understanding fidelity, but you gain a vision encoder that doesn’t wreck your inference budget. For a model whose entire selling point is “capable but cheap,” that’s the right trade.

What’s next: GPT-5 mini is already there

Snapshot transfer configs in the codebase already reference zen-os4-gpt5mini-s160-4shard. The d36 depth, 4-shard GPU layout — the same pattern that defines o4-mini carries forward into the GPT-5 generation.

The “mini” isn’t going away. It’s a permanent tier in the lineup, running the same architectural playbook: moderate depth, compensatory expert count, lightweight vision, full RL training. One number to set the scale. Everything else follows.

This analysis is based on code tracing through the OpenAI monorepo codebase — architecture configs, training pipeline definitions, and snapshot transfer manifests. Model names and internal codenames reflect the state of the codebase at time of analysis.