Make possible to copy from text in screenshot mode

[wesleymatosdev]

Apple screenshot tool already provides that. I can see myself using Cap way more if this was a thing

ps: I'd love to hear maintainers' input on it and work this feature myself

[linear]

CAP-681

[tembo]

Thanks for the feature request! This would be a nice addition to complement the existing screenshot workflow.

Looking at the codebase, there's currently no OCR/text recognition functionality implemented. The screenshot infrastructure is well-established in crates/recording/src/screenshot.rs for capture and apps/desktop/src-tauri/src/screenshot_editor.rs for the editor backend. The frontend lives in apps/desktop/src/routes/screenshot-editor/.

For implementation, you'd likely want to:

1. Add an OCR dependency - On macOS, you could use Apple's Vision framework via objc2-vision or similar bindings since it already powers Live Text. For Windows, consider Tesseract (tesseract-rs or leptonica-sys) or Windows.Media.Ocr runtime APIs.

2. Integrate with the screenshot pipeline - The screenshot capture returns image data as RGBA bytes (screenshot.rs:capture_screenshot()). You'd process this through the OCR engine before or during the editor phase.

3. Add a UI layer for text selection - The screenshot editor already has annotation layers (AnnotationLayer.tsx) and tool selection. A text recognition layer could follow similar patterns, overlaying selectable text regions on the image.

4. Hook into existing clipboard - There's already write_clipboard_string and related clipboard functions in apps/desktop/src-tauri/src/lib.rs (around line 4715) that handle text copying. The extracted text would use these existing utilities.

Since Cap already has Whisper (whisper-rs) for audio transcription in videos, you might look at apps/desktop/src-tauri/src/captions.rs for patterns on how ML-based processing is integrated into the Tauri app.

The main architectural decision would be whether to run OCR:

• At capture time - Detect text immediately when screenshot is taken
• On-demand - Add a "Copy Text" tool in the screenshot editor that triggers OCR

Happy to see you're interested in contributing this!