I bought an M5 Ultra with 256GB of unified memory, mostly so I could run serious models locally without paying per token. Then I went looking for a guide on which inference engine to actually run on it. There isn't one. There are a dozen engines, each with a README claiming to be 2x faster than llama.cpp, each measured on their own hardware with their own model in their own way. So I did what I always end up doing: I built all of them from source and measured them myself.

Why Bother?

A year ago the answer was "use llama.cpp, or MLX if you're feeling fancy." That's no longer true. Now there are engines that write their own Metal kernels for a single model family, engines that split work onto the Neural Engine, engines that tune themselves on your GPU before they'll load anything, and a whole zoo of speculative decoding schemes (MTP heads, DFlash2, DSpark, n-gram) that can triple your decode speed or do nothing at all, depending on who implemented them. The M5 Ultra is new enough that almost nobody has published numbers on it. If I'm going to run an agentic coding model on this machine all day, I want to know which engine to pick, and why.

Methodology, and What I Left Out

I really wanted to use the exact same weights in every engine. That turned out to be impossible. Every engine has opinions about file formats. Some don't load GGUF. Some only do. Some have their own special format that requires converting from BF16 source. Where possible, I used the same weights. Where impossible, I used their tools or pulled down their released weights. This is mildly annoying, but it's one thing Claude is good at. Yes, I am well-aware that testing performance with only one connection is not the same as measuring 4x or 128x at a time. If you think some other test is better, you're welcome to run them on your own hardware and report those results how you see fit. My blog, my rules.

A few engines didn't get tested: DwarfStar and Strata are both excellent at what they do, but they don't run Qwen 3.8 27B. Simply don't. They focus on bigger modes (which is why I bought this machine), so they'll get their own post later.

The Results

Three numbers matter: how fast it writes, how fast it reads, and how long you wait before anything happens. Each row is one engine with one set of weights, and each bar is a drafting mode.

The Process

• One model: Qwen3.8-27B, because it's everywhere, it's smart, and I plan to use it as a helper.
• One machine: M5 Ultra, 256GB.
• Every engine built from source on the Mac (where possible), with the best optimization settings each one offers.
• I chose MXFP4 or the best Q4 weights available. It's one of the paths most optimized for local inference, and MXFP4 has hardware support since the M4, so any improvements might be seen there.
• Each engine tested w/o drafting first, then with every speculative draft options I could find.
• One benchmark client for everything. Greedy decoding, varied prefill length and content, 512 output tokens (thinking counts), batch size one.

BLUF: Drafting matters more than anything else.

• MTP uses the model's own built-in multi-token prediction head
• DFlash2 is a separate small block-diffusion model trained for Qwen3.8-27B
• n-gram just copies text that already appeared in the context

N.B.: Building from source ftw! oMLX silently ignored the DFlash drafter on MXFP4 models because of a 1-line bug. Magnitude choked on a couple of new format keywords in GGUFs made by the current llama.cpp. Both were small patches. This is why I love open source.

Pros and Cons

TensorFold

• Pro: Hand-tuned kernels and the best draft model integration here. Speculative decoding actually pays off.
• Con: Fast on one weight layout only. Step off it and performance falls off a cliff.

uzu

• Pro: Chews through long prompts faster than anything else, so agents feel snappy.
• Con: Locked to Mirai's own quants and drafter. You can't bring your own weights.

Splash

• Pro: Steady at every prompt size, and it takes GGUF or MLX.
• Con: Only a handful of supported models, and the draft model is always on.

oMLX

• Pro: Lots of knobs and solid MTP support. Feels like a real server product.
• Con: Per-model JSON config, and it fails silently when a feature doesn't load.

baseRT

• Pro: One file holds the weights, vision and MTP head, and the superfluid server is excellent.
• Con: Closed-source engine, so some documented features are out of reach.

llama.cpp

• Pro: Runs anything, and supports every draft scheme out there.
• Con: Slow to read long prompts, and gets nothing from the M5's FP4 hardware.

Magnitude

• Pro: Moving fast, and ships tested model and drafter pairs.
• Con: Picky about file formats, and trails the pack.

vLLM

• Pro: The same API and batching you'd run in production on NVIDIA.
• Con: On hybrid models like Qwen3.8 it can't use a real draft model, so it's the slowest here.

Takeaways

The draft model matters more than the engine. Every engine lands in roughly the same 42 to 58 tok/s range with drafting off. That's the memory bandwidth talking. The spread only opens up once speculative decoding is on, and that's where the engineering effort shows. TensorFold gets 3.6x out of DFlash2. Magnitude gets 1.8x out of the same drafter. Same model, same drafter, same hardware.

Prefill is the number nobody puts on the box. For an agent that keeps re-reading a 32K context, time to first token is what you feel. uzu gets there in about 14 seconds and Splash in about 21. llama.cpp takes about 40.

MXFP4 only helps if the engine has kernels for it. The M5 has hardware for FP4, but llama.cpp ran MXFP4 at the same speed as an integer 4-bit quant. Splash and oMLX, which have native paths, are where it pays off.

If you're setting up a Mac like this for agentic coding today, I'd run TensorFold if your model is one it supports, and Splash if you want the shortest wait on long prompts. Then I'd check back in a month, because half of these projects ship a release every day or two and these numbers will be stale by Christmas.