docs(charter): scheduled fetch as first-class intake + mailbox connection spec (HT-92)

CHARTER.md

@@ -1,21 +1,21 @@
 # Helpthread — Founding Charter
 
-> Helpthread is an open-source, serverless helpdesk — shared inbox, threaded email conversations, knowledge base — built for teams who live on Vercel and Supabase rather than a LAMP server. It is aimed at the bar Help Scout set for ease of use — the experience this project exists to make ownable — rebuilt in modern TypeScript with no daemons, no polling loops, and an extension system designed for how serverless software actually deploys. FreeScout, the established self-hosted PHP helpdesk, proved the self-hosted market is real; Helpthread is built for that market on modern rails. Core is AGPL-3.0; the project is dogfooded first, built by the team behind Resonant IQ as their own production support system before anyone else touches it.
+> Helpthread is an open-source, serverless helpdesk — shared inbox, threaded email conversations, knowledge base — built for teams who live on Vercel and Supabase rather than a LAMP server. It is aimed at the bar Help Scout set for ease of use — the experience this project exists to make ownable — rebuilt in modern TypeScript with no daemons, no long-running processes, and an extension system designed for how serverless software actually deploys. FreeScout, the established self-hosted PHP helpdesk, proved the self-hosted market is real; Helpthread is built for that market on modern rails. Core is AGPL-3.0; the project is dogfooded first, built by the team behind Resonant IQ as their own production support system before anyone else touches it.
 
 ## 1. Mission & positioning
 
 Helpthread starts from a personal fact: its founder ran customer support on Help Scout for years and loved it. Help Scout remains the gold standard for a helpdesk that feels effortless on both sides of the conversation — and that experience is the bar this project holds itself to. What Help Scout never offered is ownership: your helpdesk, your data, your customization, on your own infrastructure, with no rent forever. Help Scout's ease, fully owned — that combination is the product.
 
-The self-hosted market is real: thousands of teams run FreeScout instead of paying rent to Zendesk or Help Scout. We ran it ourselves — and hit its walls. It's a decade-old Laravel 5.5 app held together by IMAP polling daemons, its extensibility means dropping PHP into a runtime plugin folder, and even basic customization — the knowledge base's design, its URLs — is out of reach without forking. Proof of the demand, and proof of the ceiling. FreeScout serves one further purpose here: because Help Scout is closed SaaS you can't inspect, FreeScout is our open *window* into that experience — a self-hostable reference for modeling the interface toward Help Scout's ease of use. It is a UX reference, never a source of code. The core is our own, built on modern, permissively-licensed foundations.
+The self-hosted market is real: thousands of teams run FreeScout instead of paying rent to Zendesk or Help Scout. We ran it ourselves — and hit its walls. It's a decade-old Laravel 5.5 app that needs a persistent host running scheduler daemons, its extensibility means dropping PHP into a runtime plugin folder, and even basic customization — the knowledge base's design, its URLs — is out of reach without forking. Proof of the demand, and proof of the ceiling. FreeScout serves one further purpose here: because Help Scout is closed SaaS you can't inspect, FreeScout is our open *window* into that experience — a self-hostable reference for modeling the interface toward Help Scout's ease of use. It is a UX reference, never a source of code. The core is our own, built on modern, permissively-licensed foundations.
 
-Helpthread is the answer for people who already live on serverless infrastructure. Same ownership promise — rebuilt for a world of edge functions, managed Postgres, and push-based delivery instead of cron loops watching a mailbox. Two audiences have to come out of this feeling the same way: customers should find getting support dead easy, and operators should find running the thing dead easy. Neither of those is negotiable in favor of the other.
+Helpthread is the answer for people who already live on serverless infrastructure. Same ownership promise — rebuilt for a world of edge functions, managed Postgres, and push-based delivery where providers offer it, bounded scheduled fetches where they don't — and no resident process either way. Two audiences have to come out of this feeling the same way: customers should find getting support dead easy, and operators should find running the thing dead easy. Neither of those is negotiable in favor of the other.
 
 ## 2. Product principles
 
 - **Innovate on the platform. Be boringly faithful on mail semantics.** Everything about *how* Helpthread runs — compute model, storage, deployment — is fair game for rethinking from scratch. How it parses, threads, and sends email is not. Mature systems — FreeScout and Chatwoot among them — have earned a decade of edge-case scars in production email handling; we respect that earned knowledge, reproduce proven behavior, and verify against fixtures rather than re-deriving from first principles. Any change to mail behavior needs proof of equivalence or an explicit, written justification — not a hunch that the old way looked wrong. During this project's own early development we watched well-intentioned "improvements" to mail handling silently destroy message content in testing; that lesson is why this rule exists and why it's not up for debate per-PR.
 - **Threading authority lives on the outbound side.** Inbound threading headers (`References`, `In-Reply-To`) are written by every mail client on earth, inconsistently — they cannot be trusted. The one header the engine fully controls is the Message-ID it emits. So threading is anchored there: signed reply tokens in outbound Message-IDs, authenticating replies on their way back in — a pattern as old as mailing-list software. This is the mechanism the whole system leans on; treat it accordingly.
 - **API-first — and the UI isn't the only client.** Anything the UI can do, a typed public API can do. The UI is a client of that API, not a special case; so are AI agents. An MCP server exposing the same operations ships as a first-class client of the same API, so any operator can point their own AI tooling at their own helpdesk.
-- **Serverless-native, Vercel-first, not Vercel-only.** No daemons, no polling loops. Inbound mail arrives via push webhooks (Gmail push through Pub/Sub, and equivalents as we add providers); anything that needs to happen later is a scheduled action or a cron trigger, not a process sitting in a loop. The engine's core reaches every platform service through interfaces the project owns — see the platform posture note in the architecture section.
+- **Serverless-native, Vercel-first, not Vercel-only.** No daemons, no long-running processes. Inbound mail arrives either by push webhook (Gmail push through Pub/Sub, and equivalents as we add providers) or by a bounded, stateless scheduled fetch — a function that connects, reads from a stored cursor, and exits. **Neither is "the primary" — the operator chooses at setup, trading latency against setup cost:** push is near-instant but needs a GCP project and roughly six provisioning steps (`specs/deploy/gmail-inbound-runbook.md` Part A); scheduled fetch costs up to a cron interval of latency and needs an app password or OAuth. Both are fully supported intake paths; neither is a fallback for the other. Nothing stays resident either way: no held connections, no IMAP IDLE, no worker loops. Anything that needs to happen later is a scheduled action or a cron trigger, not a process sitting in a loop. The engine's core reaches every platform service through interfaces the project owns — see the platform posture note in the architecture section. *(Amended 2026-07-20 — scheduled-fetch intake; see §7 appendix.)*
 - **Own your data.** Self-hosted on the operator's own Vercel and Supabase accounts. Conversation data never touches Resonant IQ-operated infrastructure — we don't sit in the data path — and AI features preserve that: assistants call model providers the operator configures, with the operator's own keys. **The promise is absolute and carries no exception.** *(The 2026-07-19 "managed hosting" carve-out was **rescinded 2026-07-20** on TJ's review — he had selected managed hosting from a picker that labelled it "Recommended", and rejected the model once its consequences were clear. See the §7 appendix.)*
 
 ## 3. Licensing & intellectual property
@@ -63,7 +63,7 @@ The founding public API surface is six conversation operations: list conversatio
 No dates; phases are ordered by dependency, not calendar.
 
 - **Phase 0 — Foundations.** Name, domains, GitHub org, and npm org secured (done). This charter. A public-ready engine repo with clean history.
-- **Phase 1 — Core engine, dogfooded.** Mail engine: event-driven ingestion (bounded reconciliation fetches, never a long-running poller), parsing, threading, sending, signed reply tokens, auto-responder handling, bounce handling, HTML sanitization. The six-operation conversation API. An agent inbox UI. Gmail push for inbound. This runs as Resonant IQ's actual production support desk before it runs as anyone else's.
+- **Phase 1 — Core engine, dogfooded.** Mail engine: event-driven ingestion (bounded reconciliation and intake fetches, never a long-running poller), parsing, threading, sending, signed reply tokens, auto-responder handling, bounce handling, HTML sanitization. The six-operation conversation API. An agent inbox UI. Gmail push for inbound. This runs as Resonant IQ's actual production support desk before it runs as anyone else's.
 - **Phase 2 — Production cutover.** Resonant IQ retires its FreeScout instance and switches to Helpthread at the config level — a clean cutover with no legacy data requiring migration.
 - **Phase 3 — Public launch.** Deploy-to-Vercel button, public docs site, and the start of a community — **with the marketplace live on day one**: license keys, module distribution, and the first paid modules (the knowledge base and AI-powered modules — draft-reply suggestions, auto-triage, KB-grounded auto-answers in the widget — lead the catalog), all having served as Resonant IQ's own install path through the dogfood phase. A helpdesk launched free with its commercial layer already real avoids the worse story: a community formed around ambiguity about how the project sustains itself.
 
@@ -99,4 +99,19 @@ Solo-maintainer, BDFL model for now — there is one project, one person account
 
 ~~§2 — the "own your data" bullet's promise is stated as *conversation data never touches Resonant IQ-operated infrastructure — we don't sit in the data path*, with one explicit, opt-in exception: a module an operator enrolls in Resonant IQ's managed hosting processes that operator's conversation data (and holds the operator-supplied keys it needs) on Resonant IQ infrastructure, under disclosed data-handling terms consented to at enrollment. This also corrects a mis-scoped earlier phrasing that put "the core" out of the data path — the operator-run core *is* the data path; the promise that matters is that **Resonant IQ-operated** infrastructure stays out of it. Rationale: the managed-hosting amendment (same date, above) made an unqualified promise falsifiable for enrolled operators; a data promise that is precisely true beats one that is absolutely worded and false in an opt-in case. The mail engine and system of record remain on the operator's own accounts in every configuration, and the self-host path preserves the original absolute promise. Decided by TJ acting as counsel, 2026-07-19; the managed-hosting data-handling terms themselves remain on the §8 pre-revenue counsel gate (`specs/modules/marketplace-v1.md` §8).~~
 
+
 **Amended 2026-07-19 (HT-75):** §3 and §4 — the knowledge base reclassified from free-forever core to a paid first-party module. FreeScout's own reference instance runs its Knowledge Base as a paid purchase (`specs/ui/admin-ia.md`); keeping ours core was chosen 2026-07-18 as a marketed free-vs-FreeScout differentiator (`specs/modules/catalog.md` §1), and TJ reversed that call on review. Not a retroactive paywall under the §3 invariant — the knowledge base was never shipped (`specs/modules/catalog.md` §2.1's shipped list never included it). Full paid-module catalog: `specs/modules/catalog.md`.
+
+**Amended 2026-07-20 (HT-92, scheduled-fetch intake):** §2 — the "Serverless-native" bullet's prohibition is restated from *"no daemons, no polling loops"* to **"no daemons, no long-running processes."** Inbound mail may arrive either by push webhook or by a **bounded, stateless scheduled fetch** — a function that connects, fetches from a stored cursor, and exits. **Neither transport is designated primary.** The operator picks one at setup and trades latency against setup cost — push is near-instant but requires a GCP project and ~6 provisioning steps plus two known org-policy landmines (`specs/deploy/gmail-inbound-runbook.md` Part A); scheduled fetch costs up to a cron interval and requires an app password or OAuth. Consequential wording updated at §1's opening summary, the FreeScout comparison, the serverless positioning line, and §4's Phase 1 description.
+
+*Wording ratified by TJ, 2026-07-20.* He initiated the change — *"Wait, wait, wait - why would we forbit imap polling?"* and *"Wasn't the no polling clause just to say... we as Helpthread will never poll their code?"* — gated it on evidence (*"let's get this fully tested out before we change the charter. I want to see it in action first"*), and approved the §1 positioning line verbatim: *"push-based delivery where providers offer it, bounded scheduled fetches where they don't — and no resident process either way."* An earlier draft of this amendment elevated scheduled fetch to a *"first-class primary transport"*; that framing was the drafting assistant's, not TJ's, and was removed on review — the operator, not this charter, chooses the transport.
+
+Rationale: the original wording conflated two separable things — a *daemon* (a process that stays resident and holds an open connection, which is what makes FreeScout require a persistent PHP host) and a *scheduled fetch* (an invocation that ends). Only the first is incompatible with serverless. §4's Phase 1 language already permitted "bounded reconciliation fetches, never a long-running poller," and the daily watch-maintenance cron has always been exactly that — it enqueues the same reconcile job the push path enqueues, and that job reads the mailbox's *stored cursor*, never the push notification's `historyId`. The engine already did on a schedule what §2 appeared to forbid; this amendment removes an internal inconsistency rather than introducing a capability.
+
+What it unlocks: making Pub/Sub **optional rather than mandatory** — six setup steps an operator no longer has to complete, including the two that fail silently (the domain-restricted-sharing org-policy block and the missing `serviceAccountTokenCreator` grant). Gmail push itself remains fully supported and preferred where an operator wants sub-minute latency; only its status as a setup prerequisite changes. It also unlocks IMAP/SMTP mailbox connection via app password, which needs no DNS records, no MX change, no cloud console, and no third-party provider account. Measured against the ~26-step, 4-console setup an adopter faces today (HT-82 dogfood), this is the difference between Helpthread being evaluable and not being evaluated. §1's own standard — *operators should find running the thing dead easy* — was not being met.
+
+Verified before amending (2026-07-20, live against the dogfood desk): IMAP + app password authenticated `help@resonantiq.app` and returned raw RFC822 with a real reply's `ht.ht1.…` signed token intact in its `References` chain, connection closed on exit; SMTP + the same credential sent with our own `Message-ID` preserved verbatim through Gmail's submission path.
+
+What does not change: no process may stay resident — no IMAP IDLE, no held connections, no worker loops; each fetch is a fresh bounded invocation. Threading authority stays on the outbound Message-ID (§2). Mail semantics remain sacred (§2) — any new transport ships with fixtures proving equivalence before it becomes a default. The data-path promise (§2, "Own your data") is untouched, and is strengthened: a direct mailbox connection puts fewer third parties in the path than a provider webhook would.
+
+Positioning consequence, accepted deliberately: §1 previously sold against FreeScout partly on this line. The honest differentiator survives and is restated — FreeScout requires a host running resident scheduler daemons; Helpthread runs the same logical work as stateless invocations on infrastructure the operator does not administer. The claim was never truthfully "we never fetch on a schedule," since the engine always did. It is "nothing of ours stays running." Decided by TJ, 2026-07-20.

README.md

@@ -13,7 +13,7 @@ Open-source, serverless helpdesk — shared inbox, threaded email conversations,
 
 ## Architecture in one breath
 
-TypeScript on Vercel (Fluid Compute, Workflows, Queues, Cron — no daemons, no polling) plus Supabase (Postgres, Auth, Storage, Realtime). Inbound mail arrives by push webhook, not IMAP polling. Threading authority lives on the outbound side: signed reply tokens in the Message-IDs the engine emits. See the charter for the reasoning.
+TypeScript on Vercel (Fluid Compute, Workflows, Queues, Cron — no daemons, nothing stays resident) plus Supabase (Postgres, Auth, Storage, Realtime). Inbound mail arrives by push webhook where the provider offers one, or by a bounded scheduled fetch where it doesn't — either way as an invocation that ends, never a process sitting in a loop. Threading authority lives on the outbound side: signed reply tokens in the Message-IDs the engine emits. See the charter for the reasoning.
 
 ## Development