on device processing is a given for a feature that is months ahead of Apple
Apple has just concluded its presentation at WWDC, the Developers Conference keynote, where they showcased updates to their operating systems and introduced new Apple Intelligence products. One notable aspect is the integration with ChatGPT 4.0, which will be available for free, along with on-device processing.
They showed the ability to proofread and summarize text. This is already possible in Linux and Windows environments using ollama with on-device processing. The process involves:
- Starting with rough text, typically dictated using whisper.cpp with blurt.
- Copying that text, pasting it into ollama, and requesting a proofreading or summary.
- Copying the resulting text back for review.
The integration of ollama, all on the device, into the proofreading workflow is analogous to having another pair of eyes; it’s not the final word, but it may catch things that are easily missed. And this is where it gets interesting: with the speed of dictation to whisper.cpp, you can speak quickly and freely, without worrying about syntax, grammar, or any other aspects. You can simply throw down your thoughts and words. Once you have that draft, you can send it off to ollama for proofreading. After this first draft one then has more time for manual editing. Ideally, quality should increase while consuming less time.
Felix qui nihil debet 