MLX vs llama.cpp on an M3 Ultra: mlx-lm decodes 1.49x faster, and one run is not an answer

The one comparison that exists

Across a long benchmarking campaign on this machine, exactly one model ran on both mlx-lm and llama.cpp, at exactly one depth. Every MLX-versus-llama.cpp number I have traces to this table. The model is Qwen3.6-27B, a dense hybrid-attention 27.8B. The prompt is a raw file with no chat template, which all four engines independently tokenized to the same 234,158 tokens.

Peak RAM is system-wide for oMLX, mlx-lm and llama.cpp, because an HTTP client cannot see its server's allocator. That is a structural gap in the method, not an oversight. surge is reported on a resident basis and is not directly comparable to those three.

The headline result is that no single engine wins. oMLX prefills fastest and decodes slowest. mlx-lm decodes fastest. llama.cpp uses the least memory of the three server-based engines. My own engine is 34x and 14x behind on the two speed axes; on the one basis where a clean comparison exists, allocator against allocator, it uses 47.0 GiB to mlx-lm's 70.64 GB, a 1.40x difference rather than the larger gap the table's resident figure suggests.

What the 1.49x is actually made of

7.58 divided by 5.10 is 1.486. Four separate things make that number softer than it looks, and I found all four after publishing it.

One: it is n=1 against n=1. The llama.cpp arm was measured once and never repeated. I have since run that exact mlx-lm configuration three times and got prefill of 101.9, 114.0 and 109.3, and decode of 7.58, 7.89 and 8.33. So MLX's own number moves 1.12x on prefill and 1.10x on decode without anything changing. Taking the mlx-lm range against llama.cpp's single point, the honest decode ratio is a band of 1.49x to 1.63x, not a value.

Two: it sits right on my own threshold. Earlier in the same project I measured one configuration three times and got a 1.47x spread, and wrote down the consequence: ranking two things at this depth on single runs is not sound unless the gap exceeds about 1.5x. The decode gap is 1.486x. It is on the boundary, above the exact 1.47x spread and below the rounded rule, which is precisely the position where a second llama.cpp run would settle everything and none exists.

Three: the two decode numbers are not the same measurement.

# The two decode numbers are not the same measurement.

llamacpp_niah_client.py:103   pred_n / pred_ms        <- the server's MEAN
mlx_raw_niah_client.py:107    least-squares SLOPE     <- fitted per token

# Checked on the SAME MLX generation, the two agree within 1.2%.
# That check has never been run on a llama.cpp generation.

A mean over a whole generation and a least-squares slope over per-token timestamps are close: measured from the same generation on three MLX models they agree to within 1.2%. But that check has never been run on a llama.cpp generation, and the llama.cpp arm is the one row in this comparison reported as a server mean. It is a protocol gap rather than a quantified error bar, which is exactly the kind of thing that is easy to leave unexamined when the answer already looks right.

Four: the arms ran at different clocks. mlx-lm's median was 700 MHz, llama.cpp's 655. This machine's full clock is 1379 MHz, so neither arm was anywhere near it. That matters more than it sounds, because I measured two runs of identical work at 802 and 1375 MHz that differed 1.26x on decode while both reported zero percent clamped. No clock correction has ever been applied to this engine pair, and I have no clock-sensitivity coefficient for this model to apply one with.

There is also an observation I cannot explain. llama.cpp's system-wide peak was 34.7 GiB lower than mlx-lm's, and its median clock was 45 MHz lower too. I have no causal account of that, and I am not going to invent one.

The prefill "win" is a tie

101.9 against 95.6 is 1.07x. On this machine prefill reproduces to somewhere between 0.08% and 4.7% depending on which pair you check, and the same mlx-lm configuration spans 101.9 to 114.0 across three runs. A 7% difference sits inside that. It is a tie, and I have written it as a tie every time since.

The one thing that does hold: the earliest version of this comparison said "llama.cpp is the fastest engine here on both axes", beating MLX 2.1x on prefill and 1.4x on decode. I retracted that the same day. The mlx-lm arm had been measured with the machine's cooling effectively idle. Re-running it with nothing changed but verified fans moved prefill 2.1x and decode 2.0x. Other severe fan failures on this machine reached roughly 4x to 5x on prefill, though that does not transfer to every mildly clamped decode. The retraction was correct. The replacement claim is just weaker than I first wrote it.

Where llama.cpp actually wins

"MLX is faster" is the claim people go looking for. On my numbers it is true on exactly one axis, against exactly one MLX configuration.

llama.cpp decodes faster than the MLX serving stacks. Against oMLX 0.5.5 on the same model and prompt it is 5.10 against 3.66, a 1.39x lead, and still 1.23x against that same configuration's later re-run at 4.15. Against oMLX 0.6.2's baseline it is 5.10 against 3.91, a 1.30x lead. oMLX bundles the same mlx-lm 0.31.3, so those runs share compute kernels with the MLX arm above and differ only in the serving layer. On my 256K decode board llama.cpp sits at rank 11, above oMLX 0.6.2 baseline, oMLX 0.5.5 and a dense Qwen2.5-14B on mlx-lm.

llama.cpp wins memory outright, 85.2 GiB against 101.9 for oMLX and 119.9 for mlx-lm, on the same model at the same prompt length. mlx-lm's system-wide peak was 1.41x llama.cpp's.

And it is the more robust long-generation path, by a distance. On a separate long-run job, mlx-lm's server died like this, three times:

# mlx_lm.server, 4B model, long generation:
RuntimeError: [metal::malloc] Resource limit (499000) exceeded
    at mlx_lm/generate.py:1565

# The generate thread dies. The SSE stream never closes.
# The client hangs forever. Nothing is logged as an error.
# The arm failed three times: two attempts at a 32k budget hit
# this same wall, a third at 16k made no usable progress.

That is on a 4B model, on a machine with 512GB, at around 20,840 generated tokens. It is a single allocation scaled by the max-token setting inside the batch path, not cumulative residency, and the silence is the worst part: it looks exactly like a slow run. A separate long llama-server job, LiveCodeBench on a 30B, ran 2 hours 39 minutes and completed cleanly. I have not put that identical 4B arm through llama-server, so this is a difference in what I have seen fail rather than a controlled comparison. For long unattended work it still shaped which path I reach for.

llama.cpp is also the better-instrumented path for verifying you measured what you think you measured. Its server reports the prompt token count it actually served, unconditionally, so the client can compare against its own tokenizer count and prove nothing was truncated. Five older oMLX rows could not do that, because OpenAI-style streaming suppresses the usage block unless the client explicitly asks for it, and mine did not. A later direct-guard re-run confirmed they had been fine, which is luck rather than method.

One llama.cpp footgun worth knowing

On b10200, running llama-server with its default four slots aborted mid-generation:

# llama.cpp b10200 with its default 4 slots:
ggml-metal-device.m:656: GGML_ASSERT([rsets->data count] == 0) failed

# It had already generated 2,210 correct tokens at 61 t/s.
# A Metal-backend bug, not a model or file problem.
# -np 1 serialises the slots and the run completes clean.

It is worth flagging because the run was visibly healthy right up to the assert. The interactive llama-cli path is separately unusable at this depth: it hung for 2 hours 47 minutes with the GPU idle. So llama.cpp is not the failure-free option either. The reliable path in my notes is specifically llama-server -np 1, and everything above went through it.

What I cannot tell you

Nothing here says anything about short context. There is no mlx-lm versus llama.cpp comparison in my logs at any depth below 234,158 tokens. If you are running 4k prompts on an M-series laptop, which is what most people asking this question are doing, this post does not answer your question and neither does any number in it.

One model, one quantization pairing, one depth. Qwen3.6-27B is dense with hybrid attention. For this comparison I have no llama.cpp arm on a mixture-of-experts model, on a 4-bit variant, or at any other context length. The quantizations are matched at class level only: GGUF Q8_0 against MLX 8-bit, not bit-identical weights.

The runs were sequential, which my later protocol forbids. llama.cpp at 11:27, the templated mlx-lm re-measure at 12:13, and the raw mlx-lm arm at 13:04 on 17 August 2026. This machine's throughput decays measurably across consecutive runs and recovers after idling, which is why the protocol now says never to use A-then-B ordering. These runs had verified cooling and a fresh compute gate, so I stand behind each number individually. They would not satisfy the interleaving standard I hold later work to, and the pairing is the part I would redo.

So which should you use

On the evidence I actually have, for one dense model at a quarter-million tokens on an M3 Ultra: mlx-lm if you are generating tokens, llama.cpp if memory is tight or the job runs unattended for hours, oMLX if you are ingest-heavy and barely generating. The prefill difference between MLX and llama.cpp is not a reason to pick either.

The more useful takeaway is about the shape of the question. "MLX vs llama.cpp" gets answered constantly with one benchmark run per side, and I now have a concrete sense of what that is worth: on this machine a single configuration re-run gives up to a 1.47x spread, the metric you choose moves things about 1%, and the clock alone can move decode 1.26x between two runs that both look clean. Most published comparisons, including mine, are inside that noise.

Provenance

Mac Studio M3 Ultra, 512 GB unified memory, 80 GPU cores, macOS 26.3. Runs on 17 August 2026 with later mlx-lm repeats through 20 August. mlx-lm 0.31.3, llama.cpp Metal b10200 via llama-server, oMLX 0.5.5 which bundles the same mlx-lm 0.31.3. Every run had its fan speed verified by direct reading and cleared a fresh GEMM gate above 20.5 TFLOPS before starting, because a firmware limiter on this machine understates an unguarded run by 2x to 5x. Both arms of the head-to-head carry a direct truncation guard: 234,158 tokens built, 234,158 served, not truncated. Recall is scored against the built prompt's own needle record.