The iPhone 17 Pro has achieved what was thought impossible: running a 400-billion-parameter AI model locally with just 12GB of unified memory. This feat, accomplished by developer Anemll, challenges conventional wisdom about on-device AI capabilities and opens new possibilities for mobile computing.
Traditionally, large language models (LLMs) of this size require massive server farms with hundreds of gigabytes of RAM. The idea that a smartphone could handle such a model seemed absurd. Yet, recent developments in inference engines and clever optimization are proving the skeptics wrong.
The Breakthrough: Flash-MoE Inference Engine
At the heart of this achievement is a new inference engine called Flash-MoE, created by developer Daniel Woods (aka @dandeveloper). Flash-MoE is an open-source project available on GitHub, designed to efficiently run massive models on consumer hardware.

Woods demonstrated Flash-MoE by running the Qwen 3.5 397B model—the full version, without distillation or quantization—on a MacBook Pro with 48GB of RAM. The model itself takes up 209GB on disk, but Flash-MoE's architecture allows it to be loaded and executed in a memory-constrained environment.
What makes Flash-MoE special is its use of Mixture of Experts (MoE) architecture. In MoE models, only a fraction of the parameters are activated for each token generated. This sparse activation drastically reduces the memory footprint during inference, making it feasible to run gigantic models on devices with limited RAM.
From MacBook to iPhone: Scaling Down Further
Inspired by Woods' work, Anemll took on the challenge of running a similar 400B model on an iPhone 17 Pro with only 12GB of RAM. Despite the significant memory reduction—from 48GB to 12GB—Anemll succeeded.
The key was further optimizing the inference process, likely leveraging the iPhone's Neural Engine and careful memory management. While the speed is reportedly slow, the fact that it works at all is a testament to the potential of on-device AI.
Other developers have pushed the boundaries even further, running models like DeepSeek-V3 (671B parameters) and Kimi K2.5 (1.026 trillion parameters) on MacBooks. These experiments show that the barrier to running large models locally is lower than previously thought.
Why This Matters for the Future of AI
Running large AI models locally on devices like smartphones offers several compelling advantages:
- Privacy and Security: Data stays on the device, reducing the risk of breaches during transmission or server storage.
- Offline Functionality: AI assistants can work without an internet connection, ideal for remote areas or situations with poor connectivity.
- Low Latency: On-device processing eliminates network round-trip times, enabling real-time responses.
- Cost Efficiency: Avoiding cloud inference costs can be significant for both users and service providers.
- Personalization: Models can be fine-tuned on user data without exposing it to third parties.
These benefits are driving research into efficient inference techniques. Flash-MoE and similar projects are paving the way for a future where powerful AI is accessible anytime, anywhere.
Challenges and Limitations
Despite the excitement, there are hurdles to overcome. The most obvious is performance: running a 400B model on a phone is slow, likely taking seconds per token. This limits real-time applications like conversational agents.
Battery life is another concern. Sustained heavy computation will drain the battery quickly, making it impractical for all-day use. Thermal management also becomes critical, as prolonged inference can cause overheating.
Moreover, the models themselves are huge—hundreds of gigabytes—so storage is a constraint. Users would need to download and store these models, which is not feasible for most.
Finally, the complexity of setting up such systems is high, requiring technical expertise. Widespread adoption will depend on simplifying the process.
Comparison of On-Device AI Capabilities
To put things in perspective, here's a comparison of different devices and their ability to run large models:
| Device | RAM | Model Size | Feasibility |
|---|---|---|---|
| iPhone 17 Pro | 12GB | 400B | Yes (slow) |
| MacBook Pro (M-series) | 48GB | 671B | Yes (moderate) |
| High-end Server | 512GB+ | 1T+ | Yes (fast) |
| Typical Smartphone | 8GB | 7B | Yes (real-time) |
As the table shows, the iPhone 17 Pro's achievement is remarkable but comes with trade-offs in speed.
What to Watch Next
The field of on-device AI is moving fast. We can expect further optimizations that improve speed and reduce memory usage. Techniques like quantization, pruning, and better MoE routing will likely make large models run more efficiently on mobile devices. The background to this story is set out in iPhone 17 Pro Hits Lowest Price Ever in 48-Hour Flash Sale. A related thread runs through iPhone 18 Pro Max vs iPhone 17 Pro Max: Key Upgrades Compared. Earlier coverage in iPhone 18 Pro vs iPhone 17 Pro: Key Differences to Decide If It's Worth shows how the argument developed.
Apple and other chip manufacturers are also investing in AI accelerators. Future iPhones may have dedicated hardware for LLM inference, making local AI more practical. A closer look at the same subject appears in iPhone 17 Pro Hits Record Low Price in August Sale. Anyone following the wider picture can start with MediaTek Dimensity 9600 Pro: 2nm Chip Brings On-Device AI to Phones. The same theme surfaces in AI in Unified Endpoint Management: Smarter Corporate Device Management, which takes a different angle.
In the meantime, developers will continue to experiment and share their findings, pushing the boundaries of what's possible. Context on how this played out before sits in iPhone 17 Pro Max Captures Iconic Artemis II Space Video. A companion piece, MediaMarkt Daily Deals: iPad and iPhone 17 Pro Discounted for Black Friday, approaches the same subject from another angle.
FAQ
How can a 400B model run on a phone with only 12GB RAM?
It's possible because the model uses a Mixture of Experts (MoE) architecture, where only a small subset of parameters is active at any given time. The Flash-MoE engine efficiently manages memory, loading only the necessary parts of the model.
Is the performance practical for everyday use?
Currently, the speed is slow, so it's not suitable for real-time applications. However, for tasks that can tolerate delays, it demonstrates the potential for future improvements.
What are the main benefits of running AI locally?
Local AI enhances privacy by keeping data on-device, enables offline functionality, reduces latency, and can lower costs by avoiding cloud inference.
Will this work on other smartphones?
Potentially, yes. The techniques are not iPhone-specific, but they require sufficient RAM and optimization. Phones with less than 12GB may struggle with such large models.