Local open-weight inference, Apple Silicon and accelerator research

ANEMLL tests hybrid Apple M6 inference that splits a 27B model across the GPU and Neural Engine

A research-only Qwen3.8-27B runtime improves long-prompt throughput on a 32GB M6 Mac, but requires disabling a major macOS security protection.

Open Model Weights published19 Nov 2026
Primary sourceANEMLL Forge — original M6 Neural Engine and GPU hybrid research report
Source published2026-11-18
FOLLOW OUR COVERAGE

Follow Open Model Weights on Google See more of our open-weight AI reporting in Google Search.

Add as a Preferred Source on Google
Original minimalist editorial illustration representing compute in AI research; conceptual, not actual model output.
AI-generated editorial illustration · Not official model imagery.

ANEMLL Forge has published a research implementation that uses both major accelerators of an Apple M6 Mac on the same open-weight language model. The experiment places Qwen3.8-27B's feed-forward networks on the Apple Neural Engine and its token-mixing operations on the GPU, then pipelines the two kinds of work across prompt blocks. The objective is to shorten the wait before a long-context local coding agent begins responding, without maintaining two complete copies of the model's weights in memory. Primary source ↗

The hardware constraint is central: the team uses a 32GB M6 system and a three-bit quantized representation of the 27B model. Qwen3.8-27B has 48 DeltaNet and 16 full-attention layers in the configuration tested. Rather than assigning entire contiguous layer ranges to one processor, the research runtime sends mixing to the GPU and the large feed-forward matrices to the Neural Engine. The approach attempts to keep both processing units occupied during prompt ingestion, which is often the expensive phase of an agent conversation.

In ANEMLL's published comparison, the hybrid processes a 64K-token prompt at 420 tokens per second, against 329 for splash 1.3.2 and about 216 on ANEMLL's Neural Engine-only route. At 95K tokens, the corresponding hybrid and splash prefill rates are 347 and 283 tokens per second. In one end-to-end example involving a 95K cold prompt plus 256 generated tokens, the authors report 278 seconds versus 348 seconds. These are measurements from their own machine and harness, not an independent multi-device benchmark.

The improvement is not simply a faster GPU. The hybrid maps about 10.8 GiB of already compiled, palettized Neural Engine weights so GPU kernels can read the same underlying data rather than retaining a duplicate model copy. The team also makes larger Neural Engine calls, fuses several INT8 operations, and uses speculative decoding. Its own report notes contention between both accelerators, sensitivity to thermal conditions and differences in quantized model files across the comparison engines.

One limitation rules out recommending the current method to ordinary users: the prototype requires disabling macOS System Integrity Protection and granting additional read access to protected compiled-model caches. Disabling that defense reduces the security of the entire machine. The authors present it as work for a dedicated research device, with macOS 27.3 and a particular M6 compilation path, not a supported production feature or a routine performance tweak for a personal laptop.

The news therefore concerns an openly inspectable inference technique, not new Qwen foundation weights. It complements other local-MoE memory experiments by exploring accelerator overlap and zero-copy sharing instead of disk-streaming experts. A broadly deployable version would need a supported, protected weight-sharing interface, portable reproducibility and independent tests across hardware and workloads. For now, ANEMLL's open research is evidence of what joint accelerator scheduling might make possible, with a clear security boundary.

OMW REGISTRY WATCH

What this changes for the evidence layer

This is a new experimental inference stack for existing Qwen3.8-27B weights, NOT a fresh base-model release. Source measurements are author-run, single-system comparisons; source research branch requires turning off macOS System Integrity Protection and was not validated on other machines. Do not recommend weakening endpoint security as normal deployment advice.

PRIMARY SOURCE

ANEMLL Forge — original M6 Neural Engine and GPU hybrid research report

This brief is based on the cited primary source. Performance, benchmark and comparative claims remain attributed unless Open Model Weights publishes an independent measurement.

Open source article ↗