seeme-speed is a from-scratch inference engine — no llama.cpp, no PyTorch, no cloud. Hand-tuned kernels that beat llama.cpp on Apple Silicon and modern CPUs, run a 35B model on a $500 desktop, and hit 77 tok/s on an iPhone. One Rust core → Metal, CUDA, CPU. Your data never leaves the device.
Every number is measured on the same box, against llama.cpp built from source with matching quantization and warmup.
How to read this: measured same-machine vs llama.cpp built from source. Decode is memory-bandwidth-bound — our edge comes from hitting the roofline, plus lossless speculative decode and delta-net kernels. On-device, we measure ~100% of the memory roofline. Methodology available on request.
Written from scratch in Rust, tuned kernel by kernel, portable by design.
One Rust core compiles to Metal (Apple GPU), CUDA (incl. Blackwell FP4), and CPU (AVX-512 + ARM NEON). iOS, Android, desktop, server — the same engine, no rewrites.
MTP/EAGLE draft-and-verify that is bit-exact — up to 1.8× faster decode with zero quality loss. The one technique that beats the bandwidth roofline.
Linear-attention hybrids with constant-size KV — token 5,000 decodes as fast as token 50 — plus MoE decode. A strong 35B-A3B on commodity CPU.
Hand-written Q4_K / Q6_K / FP4 kernels — fit bigger models in less memory, and read fewer bytes per token, right at the bandwidth roofline.
Private, offline, zero marginal cost. Inference runs on the device — your data never leaves it, not even to our servers. Sovereignty by construction.
No llama.cpp, no PyTorch, no vendor runtime. One binary you control — auditable, embeddable, and yours to ship anywhere.
On-device LLMs without the cloud bill — real-time, offline, on the phone in your users' pockets.
Healthcare, legal, finance, public sector — inference that never leaves the device. The strongest sovereignty story there is.
Extract the maximum tok/s from your chip — a cross-platform, hand-tuned software stack that makes your silicon shine.
A genuinely strong 35B-A3B on a $500 CPU desktop — no datacenter, no GPU, no per-token bill.
Partnerships, benchmarks, licensing, press — if you build chips, devices, or edge products, we should talk.