MailChat MVP: email-as-chat spec

[Hohnik]

Destination

A locked technical spec for the MailChat MVP — an app that ingests mail from multiple IMAP/SMTP accounts and presents it as chat-style conversations grouped by person and subject. Done when someone could start implementing without hitting open architectural questions. Single-user tool: every user runs their own instance (no shared multi-tenant deployment).

Notes

• Stack: FastAPI + async SQLAlchemy + SQLite — SQLite is the production DB too, replicated to S3 via Litestream (no server DB needed: single-user, single-process, no concurrent-writer or multi-machine access pattern)
• Frontend: htmx (v4, beta/RC) + htpy for HTML generation (plain Python function calls, no template files) — not a JSON API. See Decide frontend technology. FastAPI routes render HTML for this one UI; dedicated JSON endpoints can be added alongside later if a second consumer (e.g. native app) shows up.
• Credentials encrypted at rest (Fernet-style, master key from environment); one Account + linked Credentials row per connected mailbox
• Sync: polling-based background worker per account; IMAP IDLE deferred as a later optimization
• Testing: prefer disposable test accounts (e.g. Ethereal Mail) for repeated/automated runs; the real T-Online account (credentials in src/mailchat/.env) is for the initial proof-of-connection only
• Consult /grilling and /domain-modeling for decision tickets; /research for fact-finding tickets

Decisions so far

• Research: does T-Online require an app-specific password for IMAP/SMTP when TOTP is enabled? — Yes, unconditionally: T-Online always requires a separate "Passwort für E-Mail-Programme" for IMAP/SMTP, independent of TOTP/MFA status; the regular account password never authenticates against mail protocols. Generated via E-Mail Center or Kundencenter. src/mailchat/.env needs this generated password in place of the current one. Findings: docs/research/t-online-imap-smtp-totp.md

• Task: connect T-Online account via IMAP (read-only proof of concept) — Confirmed working with stdlib imaplib only, using the app-specific password from Research: does T-Online require an app-specific password for IMAP/SMTP when TOTP is enabled? #2; implemented as src/mailchat/sync/imap_client.py (connect/list_folders/fetch_recent_messages), with mocked unit tests in tests/test_sync/test_imap_client.py. Live-verified against the real account (10 folders, 3 messages fetched read-only). Real subject lines confirm German double Re: Re: prefixing that Decide the person+subject conversation-grouping model #5 will need to handle. Also added missing project plumbing ([build-system], pytest dev group, sync/__init__.py) since none existed yet.

• Task: connect T-Online account via SMTP (send a test email) — Confirmed working with stdlib smtplib only: securesmtp.t-online.de:465 via implicit SSL (SMTP_SSL, no STARTTLS needed), same app-specific password as IMAP. Implemented as src/mailchat/sync/smtp_client.py (SmtpCredentials, connect, send_message), with mocked unit tests in tests/test_sync/test_smtp_client.py. Live-verified end-to-end: sent a test email to the account itself and confirmed delivery via the existing IMAP client.

• Decide the person+subject conversation-grouping model — Conversation = (person, normalized_subject) tuple. "Person" is the primary correspondent only (CC'd others don't affect grouping); contact identity is exact address, no auto-alias-merging. Subject normalization strips Re/Aw/Fwd/Fw/Wg: tokens wherever they occur in the string (not just anchored at the start — real Louis.de thread data breaks anchor-only stripping), case-insensitively, idempotent. Header threading (Message-ID/In-Reply-To/References) is stored but not used for grouping. Same contact across multiple connected accounts merges into one conversation (core to the app's value prop); "self" exclusion is account-set-aware.

• Decide frontend technology — htmx (v4, beta/RC) + htpy (pure-Python HTML generation, no template files — amended after close, replacing an initial Jinja2 pick) rendered directly by FastAPI, not a JS SPA behind a JSON API — chosen over React/Vue/Svelte for minimal runtime weight and zero build tooling, consistent with the project's existing lean bias. This supersedes the map's earlier "backend exposes a JSON API" framing. A future native app (iOS, etc.) would wrap the htmx web app via Capacitor/Tauri, or get dedicated JSON endpoints added alongside the HTML routes later. Build a minimal read-only htmx view early, alongside sync + grouping work, to visually validate the grouping model against real data rather than waiting until the backend is fully built.

Not yet specified

• Deployment/hosting target for running the FastAPI app + sync worker in production (Litestream+S3 covers persistence, not where/how the process runs)
• Participant-set/group-chat-aware conversation grouping (when the CC set itself should matter, beyond just the primary correspondent) — real need, deliberately deferred past the MVP grouping model in Decide the person+subject conversation-grouping model #5
• Manual contact-merge UX (for cases like the same human using two email addresses, e.g. gmail.com/googlemail.com) — deferred past the MVP in Decide the person+subject conversation-grouping model #5, which keeps contact identity = exact address only
• Native/wrapped mobile app packaging (Capacitor/Tauri around the htmx web app) — real future direction named while deciding Decide frontend technology #6, not sharp enough to ticket yet

Out of scope

• Gmail-specific OAuth API integration (the gmail_client.py stub) — generic IMAP already covers Gmail; provider-specific APIs are a future effort
• Realtime WebSocket push — polling is sufficient for the MVP; realtime/ stays unimplemented for now
• Search, attachments beyond basic display, multi-user auth on the app itself — not needed to prove the core concept

[Hohnik]

Map complete — the route to the destination is clear.

All five child tickets are resolved and closed:

1. Research: does T-Online require an app-specific password for IMAP/SMTP when TOTP is enabled? — yes, unconditionally; app-specific password required regardless of TOTP.
2. Task: connect T-Online account via IMAP (read-only proof of concept) — confirmed working, sync/imap_client.py implemented and live-verified.
3. Task: connect T-Online account via SMTP (send a test email) — confirmed working, sync/smtp_client.py implemented and live-verified end-to-end.
4. Decide the person+subject conversation-grouping model — conversation = (person, normalized_subject), grounded in real messy T-Online thread data.
5. Decide frontend technology — htmx v4 (beta/RC) + htpy, no JSON API, chosen for minimal runtime/toolchain weight.

Remaining fog (deliberately not ticketed): the deployment/hosting target for running the FastAPI app + sync worker in production. This doesn't block starting implementation — Litestream+S3 already answers persistence, and this is a "where does the process run" ops question that only matters once there's something ready to deploy. Pick it up as a fresh ticket when that becomes the live question.

Everything needed to start implementing the MailChat MVP — stack, credential handling, sync strategy, conversation-grouping model, and frontend approach — is now decided and linked above. Closing the map.