Metal shim boosts TinyLlama 16.36× in macOS VM

Two reported capability values stood between llama.cpp and a 16.36× gain in TinyLlama token generation. Researchers changed those values inside a macOS virtual machine on an M1 Ultra. The physical GPU, guest kernel and model remained unchanged.

That isolates the bottleneck. For some Metal workloads in Apple Silicon VMs, conservative software reporting can matter more than unavailable hardware. The GPU may be able to run the faster code while the application is told to choose a slower path.

Francesco Bonacci and Johnny Franks ran the tests through Lume, Cua’s macOS virtualization stack. Their research release provides source code, build scripts, capability probes and raw benchmark logs. Reproducibility is critical here because Apple does not present the behavior behind the technique as a stable, public compatibility contract.

Apple’s Virtualization.framework exposes a virtual graphics device to a macOS guest. The guest submits Metal work through a virtualization-aware driver, then the host runs that work on the physical Apple GPU. This differs from the direct PCI device assignment commonly described as GPU passthrough on Linux.

The stock Tahoe VM reported an Apple 5-era GPU family, a maximum of 32 KB of threadgroup memory and no SIMD-group matrix support. llama.cpp accepted those reported limits and chose older GPU kernels.

Think of it as a fast road behind the wrong speed-limit sign. The road is open, but the software slows down because the sign says it must.

The researchers created a process-scoped Metal compatibility shim that intercepts selected capability queries. It changes the Apple-family and threadgroup-memory values returned to one guest process. The tested llama.cpp builds could then use newer SIMD-group reduction, matrix and bfloat16 paths.

The scope is important. Other guest processes kept their stock settings, and the workload still ran through Apple’s existing paravirtualized GPU path. The shim changed what one application believed the virtual GPU could do, not the VM architecture beneath it.

TinyLlama 1.1B produced the largest relative improvement on an M1 Ultra with a 48-core GPU. Prompt processing climbed from 432 to 4,787 tokens per second, an 11.08× increase. Token generation improved 16.36×. The modified VM delivered 98% of bare-metal prompt performance, while generation reached 72.06%.

Larger models showed the same broad pattern:

• Google’s Gemma 4 12B QAT Q4_0 rose from 71.66 to 515.76 prompt tokens per second, or 7.20×. Generation increased from 3.41 to 49.67 tokens per second, or 14.54×. That represented 99.59% of bare-metal prompt speed and 94.82% of generation speed.
• Meta’s Muse Glimmer 30B Q4_K-M moved from 25.83 to 194.97 prompt tokens per second, or 7.55×. Generation increased from 2.38 to 21.08 tokens per second, or 8.87×.
• MLX-LM performance remained flat. Changing the reported capabilities does not automatically speed up every Metal application.

The evidence is deliberately narrow. Most tests ran on one M1 Ultra with macOS 26.6.1 on the host and macOS 26.5.2 in the guest. TinyLlama and Gemma results used ten-sample medians. Muse Glimmer used three samples and ran on a host shared with another intermittently active VM.

The Muse test also covered text-only llama.cpp. It did not test Ollama, multimodal inference or speculative decoding. The results show what worked in this configuration, not a universal performance guarantee for macOS virtualization.

The shim also depends on private, version-sensitive behavior in the guest Metal implementation, the authors warn. Apple could change that behavior between releases. When the researchers advertised broader Metal 3 support, MLX requested a residency feature that the virtual device could not provide. The released shim therefore changes only the tested values.

That trade-off is the core tension: a broader capability profile can expose faster kernels, but it can also send applications toward unsupported features. Every chip, macOS combination and Metal API needs separate validation.

For builders operating local inference servers or automated workers in macOS VMs, the immediate move is to benchmark the GPU path the application actually selects before buying more hardware. A process-specific shim may recover substantial performance without changing the VM architecture, but it needs reproducible tests, checksums and a rollback path. The release recommends restricting activation to each workload through environment variables or a LaunchAgent.

If Apple’s virtual GPU can safely execute these newer kernels, what prevents Virtualization.framework from reporting the capabilities llama.cpp needs to select them by default?