Model selection

Choosing small language models for Android phones

The best local model is the largest model your phone can run reliably for the task you actually have. Bigger is not automatically better when memory pressure, heat and slow generation turn a simple question into a waiting simulator.

Start with the task, not the leaderboard

For short chat, rewriting and basic explanations, a small general-purpose instruct model may be enough. Coding, multilingual reasoning, long documents and vision can need different models and more memory. Evaluate with prompts you care about rather than relying only on benchmark scores.

Use these practical selection rules

Low memory budget: begin with a smaller parameter count and a Q4 quantization, then test stability.
More available RAM: try a higher quantization or larger model only if the app remains responsive.
Long conversations: reduce context length if memory use becomes a problem; context cache can be a significant part of the total.
Vision: choose a vision-capable model and confirm the runtime supports its architecture and input format.
Battery and heat: test several minutes of continuous generation, not just one short prompt.

Model compatibility is a matrix

There is no universal model-to-phone table that remains accurate across devices. Compatibility depends on the model architecture, quantization, runtime version, available memory, GPU drivers and whether the app supports the required tokenizer or vision components. Treat model names as candidates to test, not guaranteed performance tiers.

A simple testing checklist

Download the model from a trustworthy source and review its license.
Check file size and ensure enough free storage remains.
Run a short prompt, then a longer prompt with the context you expect to use.
Watch for crashes, memory warnings, excessive heat and generation stalls.
Compare answer quality on the same small set of representative tasks.

Run local models with Snor

Snor supports GGUF through llama.cpp with Vulkan acceleration and LiteRT-LM models on supported CPU, GPU or NPU paths. Actual performance varies by phone and model. Download the app and model ahead of time if you plan to use local inference offline.

Get Snor from the latest releases