Skip to content

feat: draft → approve → file interaction, thread context, robustness - #46

Merged
haasonsaas merged 1 commit into
mainfrom
feat/interaction-and-robustness
Jul 25, 2026
Merged

feat: draft → approve → file interaction, thread context, robustness#46
haasonsaas merged 1 commit into
mainfrom
feat/interaction-and-robustness

Conversation

@haasonsaas

Copy link
Copy Markdown
Contributor

Summary

Reworks the bot's interaction model and hardens the runtime, per the approved improvement list. Nothing is filed by the LLM anymore; filing requires an explicit human ✅ and (for real issues) DRY_RUN=false.

A. Interaction model: draft → approve → file

  • Deterministic filing — the create_github_issue tool is gone; the agent's only tool is the terminal submit_prd. New fileIssues(prd, github, dryRun) loops prd.userStories and files one issue per story (title = story title, body = description + acceptance-criteria checklist). LLM proposes, code disposes.
  • Approval gate — the bot posts the PRD draft with a "React with ✅ to file N issues" line, seeds its own ✅ for one-click approval, and listens for reaction_added. A human ✅ on the latest draft triggers filing; the bot's own reaction and bot users are ignored.
  • DRY_RUN stays a global override — with DRY_RUN=true (default) the approval flow works end-to-end but only posts would-be payloads.
  • Partial failure visibility — per-story errors are caught, filing continues, and the results reply reports "filed X of N" plus each failure's error message.

B. Thread context

  • Follow-ups in a thread (or DM) fetch history (conversations.replies; conversations.history for flat DMs), bot messages marked as assistant context, and pass it into runPrdAgent(idea, deps, history?) — so "make the non-goals stricter" revises the pending draft.
  • A revised draft supersedes the pending one (posted as a new reply, no editing). A ✅ on a superseded draft gets a "stale draft" note. Pending drafts are in-memory only — a restart loses them (documented in README).

C. Robustness

  • Run timeoutagent.abort() after AGENT_TIMEOUT_MS (default 180000), surfaced as a "timed out" reply. Deviation: pi-agent-core 0.82.0's prompt() has no maxTurns option (checked dist/agent.d.tsprompt(input, images?) only), so no turn cap; timeout only.
  • Truncation — replies capped at 3900 chars + marker (Slack's ~4000 limit).
  • Event dedup — in-memory FIFO set of event_ids (cap 1000); duplicates dropped silently.
  • Serialization — runs chained per conversation (channel+thread); different conversations run in parallel.
  • Placeholder ts is now checked before chat.update (the old ts ?? "" wart is gone).

D. Config + logging

  • PROVIDER = openai (default) | anthropic | google → pi-ai's openaiProvider/anthropicProvider/googleProvider (export names verified in dist). API key env per provider (OPENAI_API_KEY/ANTHROPIC_API_KEY/GEMINI_API_KEY); model defaults gpt-4o-mini / claude-sonnet-4-6 / gemini-2.5-flash — all three verified to resolve non-undefined via models.getModel() at runtime.
  • Structured run logging — one JSON console.log per completed run: ts, conversation, durationMs, storyCount, dryRun, outcome (drafted|filed|failed), error.

E. Docs

  • New root AGENTS.md: pi conventions (Type from pi-ai, errors-don't-throw, fresh Agent per run, lockstep version pinning), dry-run discipline (approval AND DRY_RUN=false), and the no-subsystem-without-producer-and-consumer rule.
  • README rewritten for the new flow, new env vars, in-memory caveat, and the two new Slack scopes (reactions:read, reactions:write) + reaction_added event subscription the approval flow needs.

Test plan

bun test47 pass / 0 fail, no network or keys: filing loop incl. partial failure and dry-run, history prepend into the outgoing prompt, run timeout via abort, truncation, dedup incl. FIFO eviction, queue serialization + post-failure recovery, draft store supersede/stale semantics, provider config + model defaults + timeout parsing. bunx tsc --noEmit, bunx biome ci ., and bun install --frozen-lockfile all clean. src/ is 698 lines.

Not verified

Live Slack/GitHub E2E still hasn't been run (no workspace/tokens here). First real run should keep DRY_RUN=true and exercise draft → ✅ approve → would-be payloads before enabling real filing.

Interaction model (was: agent filed issues while drafting):
- The LLM now only drafts: create_github_issue tool removed, sole
  tool is the terminal submit_prd. Filing is deterministic code
  (fileIssues) - LLM proposes, code disposes.
- Approval gate: the bot posts the PRD draft, seeds its own checkmark
  reaction, and files only when a human adds a checkmark to the
  latest draft. Stale (superseded) drafts are rejected with a note.
- DRY_RUN stays a global override on top of per-PRD approval:
  approved drafts produce would-be payloads only.
- Partial filing failures are caught per story and reported as
  'filed X of N' with per-story error messages.

Thread context:
- Follow-ups in a thread (or DM) fetch conversation history and pass
  it into the run, so 'make the non-goals stricter' revises the
  pending draft instead of starting over. Pending drafts are tracked
  in memory (lost on restart, documented).

Robustness:
- Agent runs abort after AGENT_TIMEOUT_MS (default 180s).
- Slack replies truncated defensively at ~3900 chars.
- Slack event ids deduped (FIFO, cap 1000).
- Runs serialized per conversation via a promise-chain queue.
- Placeholder ts is checked before chat.update.

Config/logging:
- PROVIDER=openai|anthropic|google with per-provider API key env and
  model defaults (gpt-4o-mini / claude-sonnet-4-6 / gemini-2.5-flash,
  verified against the installed catalogs).
- One structured JSON log line per completed run (outcome
  drafted|filed|failed, durationMs, storyCount, dryRun).

Docs: README rewritten for the new flow; AGENTS.md added with pi
conventions, dry-run discipline, and the no-subsystem-without-
producer-and-consumer rule.
@haasonsaas
haasonsaas merged commit db127b7 into main Jul 25, 2026
3 checks passed
@haasonsaas
haasonsaas deleted the feat/interaction-and-robustness branch July 25, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant