fix(windows): support custom dialog button callbacks via TaskDialog API

[leaanthony]

Summary

• Implement Windows TaskDialog API (TaskDialogIndirect from comctl32.dll) for dialogs with custom buttons, enabling AddButton and SetDefaultButton callbacks to actually fire on Windows
• Add case-insensitive button label matching in the fallback MessageBox path
• Fall back to MessageBox API on systems where TaskDialog is unavailable (pre-Vista)
• Add comprehensive tests for button callbacks, case-insensitive matching, custom labels, and Unicode support

The root cause was that Windows MessageBox API only supports a fixed set of predefined buttons (OK, Cancel, Yes, No). Custom button labels like "Quit" were never matched against the predefined responses, so callbacks never fired. The TaskDialog API (available since Vista) supports arbitrary button labels.

Fixes #4402

Test plan

• TestWindowsDialogButtonCallbacks — OK button, case-insensitive matching, multiple buttons, whitespace handling
• TestDialogButtonTypes — Info, Error, Warning, Question dialog types
• TestCustomButtonLabels — custom labels, emoji/Unicode, default/cancel flags
• Example code in tests/windows_dialog_callback_example.go

Note: Runtime tests require Windows. The unit tests verify callback wiring without showing dialogs.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 76d29b88-1d7f-4cae-9665-329919fe67b9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/4402-windows-dialog-buttons

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}