refactor(ai): skills-only AI surface — real Actions instead of 10 fictional tools, retire the two agents - #512
Merged
Conversation
…tional tools, retire the two agents (objectstack#3820, ADR-0109) HotCRM was the corpus that motivated objectstack#3820: 6 skills declaring 16 tool references, of which 10 named tools that exist nowhere. The copilot's instructions therefore claimed abilities it did not have — the runtime silently drops an unresolved tool, so the model improvised or failed when asked to use one. Every instance passed `objectstack validate` and `lint` cleanly. The fix is mostly deletion, because the references were a category error: "analyze the pipeline", "generate email copy", "score this lead" are REASONING, not tools. What actually needed a tool was the handful of steps that change state — and those already exist as HotCRM Actions, reachable through the `action_<name>` tools the runtime materialises from them (ADR-0109's default path: no tool records to author). - case_triage: `triage_case` → the priority rubric moves into instructions; `action_escalate_case` / `action_close_case` do the work. Also repoints the hand-off that named `response_drafting`, a skill that never existed (the audit's "hand-off to a nonexistent skill"), at the real `email_drafting`. - email_drafting: 4 fictional copy tools → the model writes the copy; `action_send_email` sends it, so recipient resolution, permissions and audit match the UI path. - lead_qualification: `analyze_lead` / `suggest_next_action` → BANT reasoning in instructions over `get_record`; `action_convert_lead` / `action_schedule_followup` act, and conversion requires confirmation. - revenue_forecasting: 3 fictional analytics tools → `aggregate_data` / `query_records` / `visualize_data` with the actual forecasting method spelled out in instructions. - live_data, customer_360: already correct, untouched but for prose. The two agents (`sales_copilot`, `service_copilot`) are DELETED. ADR-0063 §2 closed `*.agent.ts` to third parties and cloud#904 made the runtime refuse non-platform agent records, so they were unreachable metadata — carrying them is the ADR-0078 lie. The capability lives in the skills, which attach to the platform assistant by surface affinity. Verified with the merged reference-integrity rule (objectstack#3885 `validate-ai-tool-references`): 22 tool references, 0 findings — was 16 references with 10 dead. typecheck, `objectstack validate`, and the 67-test suite pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BHjroNkLkajskKbJaidko4
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…DR-0011) Follow-up to the skills rewrite in this PR. The first pass routed state changes through `action_escalate_case` / `action_close_case` / `action_send_email` — but the runtime materialises `action_<name>` ONLY for an Action that opts in (`ai.exposed` + `ai.description`) AND has a headless path. All three are `type: 'modal'`: they collect a reason / resolution / final copy from a person, so they are UI-only and no tool is ever generated. Referencing them would have re-created the bug this PR fixes, one layer down. - `convert_lead` and `schedule_followup` are flow-typed with targets, so they only needed the opt-in: both now carry `ai.exposed` + an LLM-facing description. `convert_lead` keeps its `confirmText`, which the runtime reads as approval-required — an agent invocation lands in the HITL queue instead of converting a lead unattended, which is what we want for an irreversible outcome. - `case_triage` and `email_drafting` no longer claim to act. They read, reason, and hand a ready-to-paste `reason` / `resolution` / draft to the user, pointing at the Escalate / Close / Send Email button. The instructions say plainly that the assistant cannot press it — a modal Action staying human-driven is the right shape for these, not a gap. Re-verified with the corrected rule (objectstack#3894, which now models AI exposure): 19 tool references, 0 findings. Both directions hold — the two opted-in flow Actions resolve, the three modal ones would not. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BHjroNkLkajskKbJaidko4
os-zhuang
marked this pull request as ready for review
July 28, 2026 15:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HotCRM is the corpus that motivated objectstack-ai/objectstack#3820: 6 skills declaring 16 tool references, 10 of which named tools that exist nowhere. The copilot's instructions claimed abilities it did not have — the runtime silently drops an unresolved tool, so the model improvised or failed when asked to use one. Every instance passed
objectstack validateandlintcleanly. This is the dogfood fix, and the first app to run on ADR-0109's authoring model.The fix is mostly deletion
The fictional references were a category error: "analyze the pipeline", "generate email copy", "score this lead" are reasoning, not tools. What genuinely needed a tool was the handful of steps that change state — and those already exist as HotCRM Actions, reachable through the
action_<name>tools the runtime materialises from them. That is ADR-0109's default path: no tool records to author.case_triagetriage_caseaction_escalate_case/action_close_casedo the workemail_draftinggenerate_email_copy,optimize_subject_line,personalize_content,generate_emailaction_send_emailsends itlead_qualificationanalyze_lead,suggest_next_actionget_record;action_convert_lead/action_schedule_followupactrevenue_forecastinganalyze_pipeline,identify_at_risk,forecast_revenueaggregate_data/query_records/visualize_data+ the forecasting method spelled outlive_data,customer_360Because every state change now goes through the same Action the UI button uses, permissions and audit are identical whether a human clicks or the assistant calls.
case_triagealso handed off toresponse_drafting— a skill that never existed (the original audit's "hand-off to a nonexistent skill", §7 non-goal because it lives in free-form prompt text). Repointed at the realemail_draftingwhile rewriting the instructions.The two agents are deleted
sales_copilotandservice_copilotwere unreachable metadata: ADR-0063 §2 closed*.agent.tsto third parties, and objectstack-ai/cloud#904 made the runtime refuse non-platform agent records. Carrying them is exactly the ADR-0078 lie the whole issue is about. The capability lives in the skills, which attach to the platform assistant by surface affinity — so this removes a fiction without removing a feature.Prose updated to match (
src/docs/crm_admin.md"AI copilots" → "AI skills", plus two stale comments naming the copilots).Verification
validate-ai-tool-references(the rule merged in feat(spec,lint): ADR-0109 revised + Phase 1 — skills-only default path, tool-name registry, advisory skill.tools lint (#3820 R7) objectstack#3885) run against the rewritten stack: 22 tool references, 0 findings — was 16 references with 10 dead.pnpm typecheckclean;objectstack validateclean (stack summary no longer lists agents); test suite 8 files / 67 tests pass.Refs objectstack-ai/objectstack#3820 · ADR-0109 · ADR-0063 · ADR-0078
🤖 Generated with Claude Code
https://claude.ai/code/session_01BHjroNkLkajskKbJaidko4
Generated by Claude Code