What happened
On September 22, 2026, Hugging Face announced that transformers can now run GGUF models efficiently, letting people use checkpoints sized to fit laptop memory. The launch centered first on on-device inference on Apple Silicon hardware, beginning with the Qwen3.5 architecture, and the packed inference path worked only with MPS. It relied on the newest main branch of transformers along with a compatible kernels release.
Why it matters
Apple Silicon Mac owners can load these models by giving the Hub model_id and the file name through the gguf_file argument to from_pretrained, or expose the same checkpoint through transformers serve using an OpenAI-compatible API. The packed loader supported Qwen3.5 dense and mixture-of-experts designs, plus compatible Qwen3.8 checkpoints.
Read the original at Hugging Face