██╗ ██╗██╗██████╗ ███████╗███╗ ███╗██╗ ██╗ ██╗ ██║ ██║██║██╔══██╗██╔════╝████╗ ████║██║ ╚██╗██╔╝ ██║ ██║██║██████╔╝█████╗ ██╔████╔██║██║ ╚███╔╝ ╚██╗ ██╔╝██║██╔══██╗██╔══╝ ██║╚██╔╝██║██║ ██╔██╗ ╚████╔╝ ██║██████╔╝███████╗██║ ╚═╝ ██║███████╗██╔╝ ██╗ ╚═══╝ ╚═╝╚═════╝ ╚══════╝╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝
The Numbers Don't Lie
Real benchmarks. Real hardware. No asterisks.
Models
34
Avg Decode
158.7
tok/s
Avg Prefill
87.7
tok/s
Avg TTFT
44
ms
Where It Really Matters
When you send a message and wait for the AI to respond, TTFT determines how long you stare at a blank screen. This is the metric that makes or breaks conversational UX.
| Model | VibeMLX | mlx-engine | Speedup |
|---|---|---|---|
| TinyMistral-248M | 3.6ms | 117ms | 32× |
| QwQ-0.5B | 4.8ms | 228ms | 47× |
| DeepSeek-R1-1.5B | 5.9ms | 389ms | 66× |
| Llama-3.2-1B | 14.4ms | 271ms | 19× |
| Mistral-7B-v0.3 | 20.3ms | 1,283ms | 63× |
| Phi-3-mini | 11.1ms | 737ms | 66× |
| Qwen2.5-Coder-3B | 11.2ms | 638ms | 57× |
Why the massive difference? VibeMLX compiles directly into the application. No Python interpreter. No cross-language FFI overhead. No framework abstraction layers. The moment your prompt is ready, tokens start flowing.
Test System: Apple M3 Max | 16 cores | 128GB Unified Memory
Bundle & Resource Comparison
How Vibe Mode compares to popular AI coding assistants.
| Metric | Vibe Mode | Other Tools* |
|---|---|---|
| Download Size | 100 MB | 1,000+ MB |
| Memory at Idle | < 100 MB | 3-5 GB |
| Time to Launch | < 1 sec | 3-8 seconds |
| Framework | SwiftUI | Electron |
| Inference | Native | Subprocess |
| Terminal | Ghostty | xterm.js |
| Dependencies | Zero | node_modules |
* Measured against popular AI coding assistants. We're not naming names, but you know who. If you want specifics, ask in Discord.
How We Test
- •All tests on Apple M3 Max (128GB unified memory)
- •Each model tested 3+ times, results averaged
- •Prefill: 40-token prompts
- •Decode: 100-token generations
- •Models loaded fresh for fair comparison
- •Background processes killed before testing
- •Memory cleared between tests
Benchmarks are reproducible. Code available on request.