[FEATURE] AI writing assistant for selected-text correction and memory-aware generation

[jacksonkasi1]

Summary

It would be valuable for Handy to expand beyond basic speech-to-text and offer an optional all-in-one AI writing assistant.

The feature could combine:

1. AI-polished voice dictation, similar to Wispr Flow
2. Selected-text correction and rewriting, similar to Grammarly
3. Context-aware text generation, similar to Goldfish

These features should remain optional so Handy can continue working as a simple, private, offline transcription tool.

1. AI-polished voice dictation like Wispr Flow

Handy could automatically clean and improve transcribed speech before inserting it.

It could:

• Correct grammar, spelling, and punctuation
• Remove filler words and repeated phrases
• Understand spoken corrections
• Convert unstructured speech into clear sentences
• Format messages, lists, and paragraphs
• Preserve the original meaning and tone

Example

Spoken input:

Hi, I completed the changes today, actually not everything, only the payment changes, and it is ready to test.

Generated output:

Hi, I completed the payment-related changes today, and they are ready for testing.

2. Correct and rewrite selected text like Grammarly

Similar to Grammarly, Handy could allow users to select existing text and improve it directly.

Suggested workflow

1. The user selects text in any application.

2. The user activates a Handy shortcut.

3. Handy captures the selected text.

4. The user speaks or chooses an instruction, such as:

   • Correct the grammar
   • Make it professional
   • Make it friendlier
   • Shorten it
   • Rewrite it clearly
   • Translate it

5. Handy replaces the selected text with the improved version.

Example

Selected text:

Hi, the deployment completed but some issue is there and I checking now.

Instruction:

Correct the grammar and make it professional.

Replacement:

Hi, the deployment has been completed, but I found an issue that I am currently investigating.

This would remove the need to copy text, open another AI tool, paste it, enter a prompt, and copy the result back.

3. Generate text using contextual memory like Goldfish

Similar to Goldfish, Handy could optionally remember useful work context and use it when generating text.

The memory could include:

• The user’s name and role
• Writing preferences
• Frequently contacted people
• Common projects or topics
• Preferred terminology
• Recent work context
• Frequently used message formats

Example

The user could say:

Write a professional update to my manager saying the payment changes are complete and ready for testing.

Handy could generate:

Hi, the payment-related changes have been completed and are now ready for testing. I’ll share another update if any issues are found during verification.

The user would not need to repeatedly explain the same background, writing style, or relationship context.

Suggested shortcuts

Handy could provide separate configurable shortcuts for:

• Standard transcription
• AI-polished transcription
• Correct selected text
• Rewrite selected text
• Generate text using memory
• Custom AI prompts

Privacy considerations

Since privacy is a major benefit of Handy, these features should be optional and disabled by default.

Suggested safeguards:

• Support local AI models where possible
• Allow users to configure their own AI provider
• Store memory locally
• Allow users to view, edit, disable, export, and delete memory
• Allow specific applications to be excluded
• Clearly show what context is sent to external AI providers
• Send only the minimum required context
• Provide a temporary mode that does not save memory

Why this would be useful

This would make Handy an all-in-one voice-powered writing assistant.

Users could:

• Speak naturally and receive polished text
• Correct existing text without switching applications
• Rewrite selected text using voice instructions
• Generate messages using remembered context
• Reduce repetitive copy-pasting and explanations

[MikaKrul]

The app already has post-processing, see docs how to enable it.
https://handy.computer/docs/post-processing

[kqsk]

I built a working implementation of the selected-text part of this idea (Windows for now).

A separate hotkey ("Process Current Text", in the Post Processing settings) takes the selected text, or the whole field if nothing is selected, from the focused text field, runs it through the existing post-processing provider/model/prompt, and replaces it in place. No microphone, no transcription.

• Reads and writes the field through Windows UI Automation: no clipboard, no simulated Ctrl+C / Ctrl+V
• Password fields and fields that don't expose their text via UI Automation are refused with an overlay message
• Reuses the existing post-processing code; the only new setting is the hotkey, active only when post-processing is enabled
• Results are saved to history (without audio)

Tested in Notepad and Chrome (textarea and contenteditable). On macOS/Linux it currently reports "unsupported".

Branch: https://github.com/kqsk/Handy/tree/feature/process-focused-text
Diff: main...kqsk:Handy:feature/process-focused-text

Built with AI assistance (Claude Code) and tested manually. Would this be welcome as a PR? Happy to adjust the scope.