SpellType

I wanted autocomplete to work inline across ordinary Mac apps, feel nearly instant, keep writing private, and run on an 8 GB M1 Mac. It also had to avoid Screen Recording permission and always-on resource use. I had to discover an architecture that could meet those constraints together.

I built the native app around focused-field access through macOS Accessibility APIs and local MLX inference. Suggestions remain under the user's control, writing context stays on the device, and continued typing takes priority over the model's work.

I built probes and automated experiments to investigate how applications exposed text, selections, and caret state. I used AI to research unfamiliar system behavior, automate investigations, synthesize failures, and build an internal encyclopedia of what I learned. Those results became the evidence for the architecture.

I separated application adapters, product engines, and model runtimes so inconsistent app behavior stayed isolated from shared product logic and inference. That structure let me support and test ten writing apps while keeping each application's quirks from spreading through the system.

An instruction-tuned model offered straightforward prompt control, but the product needed to continue existing text with minimal latency and prompting overhead. I chose the smaller base causal OLMo 2 1B and adapted its continuation behavior directly.

I built a Python/MLX/MLX-LM pipeline with completion-only supervision and LoRA across all 16 transformer layers, using 1,680 cursor-position examples derived from 560 human-authored situations. The dataset used no private user writing or teacher-model output.

I kept training, validation, and test data separate, selected the checkpoint using validation loss, and evaluated 200 held-out cases against an unadapted 8-bit control. Comparing both models at 8-bit isolated adaptation from quantization; matched 8-bit and 4-bit evaluation then informed the deployment choice. I shipped 8-bit.

I designed the runtime around selective activation, cache reuse, quantization, and careful memory management. Persistent KV-cache reuse reduces repeated prefill; token healing repairs the changing text boundary; idle prewarming prepares likely work.

I used Swift Concurrency for cancellation and preemption as input changes, with freshness checks before a completion reaches the interface. A suggestion must still match the current text, so stale work cannot overtake the user's typing.

• Persistent KV-cache reuse to reduce repeated prefill work
• Token healing at the changing text boundary
• Idle prewarming for the next likely request
• Cancellation, preemption, and freshness checks to prevent stale suggestions

I took SpellType from prototype to public release in about three months, owning the product decisions, model adaptation, native integration, verified model distribution, Developer ID signing, notarization, and signed auto-updates. The result is a complete desktop product that supports autocomplete across ten writing apps.

Business-context autocomplete is in development. I am exploring retrieval-augmented generation with embeddings and vector databases to ground suggestions in approved company content and workflow context.