Nine days ago I posted Off Grid here and you showed up - 124 points, 66 comments, bug reports I fixed same-day, and the kind of feedback that makes open source worth it.
You told me what you wanted. Here's what I shipped:
Your AI can now use tools — entirely offline.
Web search, calculator, date/time, device info — with automatic tool loops.
Your 3B parameter model doesn't just generate text anymore. It reasons, calls tools, and synthesizes results.
On your phone. No API key. No server. No cloud function.
So what? It means the gap between "local toy" and "useful assistant" just got dramatically smaller.
You don't need GPT-4 to look something up and give you an answer. A quantized Qwen 3 / SMOLLM3 running on your Snapdragon can do it in no time.
3x faster with configurable KV cache.
You can now choose between f16, q8_0, and q4_0 KV cache types. On q4_0, models that were doing 10 tok/s are hitting 30. The app even nudges you after your first generation: "Hey, you could be running faster." One tap.
So what? The #1 complaint about on-device AI is "it's too slow to be useful." That argument just lost a lot of weight. 30tokens/second on a phone is faster than most people read.
Live on both stores. No sideloading. No Xcode.
Off Grid is now on the App Store and Google Play. Install it like any other app. Your parents could use this.
So what? On-device AI just went from "cool weekend project for developers" to "thing normal people can actually try." That matters because privacy shouldn't require a CS degree.
What hasn't changed:
- MIT licensed. Fully open source. Every line
- Zero data leaves your device. No analytics. No telemetry. No "anonymous usage data."
- Text gen (15-30 tok/s), image gen (5-10s on NPU), vision AI, voice transcription, document analysis
— all offline
- Bring any GGUF model. Run Qwen 3, Llama 3.2, Gemma 3, Phi-4, whatever you want.
I'm building this because I believe the phone in your pocket should be the most private computer you own — not the most surveilled. Every week the models get smaller and faster. The hardware is already there. The software just needs to catch up.
https://github.com/alichherawalla/off-grid-mobile
I'm in the comments. Tell me what to build next.
reply