Kimi-Linear ran a real 1M context on my Mac Studio. I checked all million tokens.

The mirage, and the one gap it left open

The prequel to this post was a teardown. On a 512GB M3 Ultra, "1M context" on the big local models turned out to be a training window of about 200K with a flag bolted on top: ask llama.cpp for a million tokens and it allocates the cache, warns you, and caps the slot back to the trained length. The sparse-attention paths that make a real million-token window work in the frontier labs simply are not implemented in llama.cpp Metal or MLX. I even wrote a custom Metal kernel to port one and could not get coherent output from it.

But I left one door open, in the watchlist at the end: what if a small-active-parameter model shipped a window that was native by construction, so it needed no sparse-attention rescue the runtime could not provide? That is exactly what happened. This post is the evidence file, same machine, same rule: every number below comes from my own run, and the raw results JSON is in the repo if you want to check them.

Why this one is built differently

Kimi-Linear-48B-A3B is a 48B mixture-of-experts with about 3B active parameters per token. What matters is how it handles length. Of its 27 layers, 20 are Kimi Delta Attention, a linear-attention variant whose state does not grow with the sequence, and 7 are full multi-head latent attention. The full-attention layers use no rotary position encoding at all, and the config sets no RoPE scaling. The million-token window is not a stretched 200K; it is the number the architecture was built and trained for.

// Kimi-Linear-48B-A3B config.json, the parts that decide it:
"model_type":          "kimi_linear",
"num_hidden_layers":   27,          // 20 KDA linear-attn + 7 MLA full-attn
"mla_use_nope":        true,        // the attention layers carry no RoPE
"rope_scaling":        null,        // NOT YaRN, NOT DCA: nothing is stretched
"model_max_length":    1048576      // a real, trained million

That single design choice is why it runs when the giants do not. Linear attention is implemented in mlx-lm as a first-class model class, so the long-context path executes natively instead of falling back to dense attention or crashing. The models that reach 1M through learned sparse attention still have nowhere to run that attention on this hardware. The gap my last post ended on was never really about the window size; it was about whether the runtime could execute the mechanism behind it. Here it can.

Don't trust the flag. Make it prove it.

A "1M context" claim is worthless if the loader quietly hands the model a fraction of your prompt. This model is a live example of the trap: LM Studio caps it at 4,096 tokens, because it ships a model_max_length field where the loader looks for max_position_embeddings. A benchmark that trusted the flag would report a triumphant 1M pass that was secretly a 4K pass.

So the harness does not trust anything. It builds a token-accurate haystack with the real tokenizer, hides 8 unique codes at 8 depths from 3% to 90%, asks for all of them back, and scores exact code-to-city association. Before it believes a single result, it asserts that the number of tokens the model reports ingesting equals the number it was handed. That assertion is the whole test. Everything else is commentary.

# The one assertion the whole test rests on.
ids   = tok.apply_chat_template(msgs, tokenize=True, add_generation_prompt=True)
built = len(ids)                     # tokens I handed the model
resp  = last(stream_generate(model, tok, prompt=ids, ...))
assert resp.prompt_tokens == built   # tokens it actually ingested

# 1M run:  built = 1,048,692   saw = 1,048,692   ->  no silent cap.

At the top of the ladder, the guard confirmed the model saw all 1,048,692 tokens of the prompt. No cap, no window, no quiet truncation. The million was real.

The ladder

I ran the same test at seven context sizes, from a 2K control up to a million tokens. Recall was perfect at every rung: 8 out of 8 needles, every time, with full ingestion verified each time. Quality simply did not decay with length, which is what "native" is supposed to mean and so rarely does.

† The 256K row's prefill and decode are slower than the trend because I was probing that run with profilers while it worked; the clean prefill curve is set by 128K, 512K, and 1M. Full per-depth data is in the benchmark repo; the interactive, hoverable version of the charts below lives there too.

The curves

The same seven runs, four ways. Recall stayed flat while everything about the clock bent: prefill throughput and decode both fall away as the context grows large, memory climbs but never threatens the machine, and wall-clock time (log scale), which at the long end is almost all prefill, is the line that decides whether a length is practical.

Measured on a Mac Studio M3 Ultra (512GB), Kimi-Linear-48B-A3B MLX 8-bit, mlx-lm 0.31.3. The interactive version, with hover values and per-length detail, is in the benchmark repo.

It works. It just takes five hours.

Read the last two columns of that table and the honest verdict falls out. Memory was never the problem: 220GB at a million tokens, comfortable inside this machine's 490GB usable window, because the 20 linear-attention layers keep the cache flat and only the 7 full-attention layers grow it. The problem is the clock. Those 7 full-attention layers cost quadratically with length, so wall time went from 3 seconds at 2K to 73 minutes at 512K to about five hours at a million, almost all of it prefill at the long end. The model read all 1,048,692 tokens at roughly 58 tokens per second before it wrote a word, and then decoded at 5.1.

This is the same wall from last time, wearing different clothes. Decode is memory-bandwidth-bound, so a giant context that must be re-read every token crawls no matter how clever the attention. Native 1M fixed the thing that was fake, that the window was a lie, and changed nothing about the thing that was slow. Below about 128K, though, it is genuinely pleasant: a couple of minutes of prefill and 59 tokens/sec, which is a real working range for a big document.

What a five-hour prefill is actually for

A five-hour ingest sounds like a dealbreaker, and for a chat window it is. But it is a one-time cost, and that changes what the model is good at. Point it at a corpus you load once and then query many times, a legal bundle, a full codebase, a book, a quarter of logs, and the prefill amortizes across every question you ask afterward. This is the natural companion to the last post's conclusion: the giant local model is not a copilot, it is a background worker, and this one is a background worker with a genuinely photographic memory of a million tokens.

To actually reach that window you have to sidestep two traps. Run it in-process through mlx-lm, or a server that lets the cache grow, and not LM Studio, or the 4K metadata cap silently wins. And do not bound the KV cache to save memory, because a fixed-size (rotating) cache drops your oldest tokens: the exact truncation the whole test was built to catch.

# Run in-process. The model builds its own cache and it grows to fit:
#   ArraysCache(size=4) for the 20 linear layers, a growing KVCache for the 7 full-attn.
for r in stream_generate(model, tok, prompt=ids, max_tokens=N):  # cache grows, no cap
    ...
# Don't bound the KV cache to save memory (e.g. mlx_lm.generate --max-kv-size, which
# swaps in a *rotating* cache): it drops the oldest tokens and your "1M" run becomes a
# silent sliding window. And skip LM Studio for this model: it caps at 4K, reading
# model_max_length where its loader wants the absent max_position_embeddings.

Good on both numbers, at last

My rule from last time was two numbers: 100 tokens/sec and a native 1M window. The giants failed both, badly. Kimi-Linear is one of the few models I have run that does well on both instead of sacrificing one for the other. On speed, it decodes in the 80s of tokens/sec up to 32K (82 to 87), easing to 59 by 128K, just short of the bar but a different league from the roughly 7 to 37 tokens/sec the giant MoEs managed in the last post. On context, the million-token window is real, the recall is perfect, the ingestion is verified, and the mechanism behind it is something the runtime can actually execute. The one honest asterisk is that the two do not stay together at the very top of the range: push all the way to a million tokens and the speed is spent on a five-hour prefill. Bandwidth still sets that ceiling, and no flag moves it.

So the watchlist updates rather than closes. Native long context that the runtime supports: found. A small-active MoE that stays coherent to a million tokens and quick in the range you actually use: found. The missing piece is only speed at the very top, and that is a hardware story: the day M5-class tensor units reach a Mac Studio, the same architecture that just proved its memory is real gets a real shot at proving it fast all the way up. Until then, I have a model on my desk that remembers a million tokens perfectly, keeps pace at working lengths, and takes an afternoon to read a full million. That is a stranger and better place to be than the last post left off.