Skip to content

[rig-tasks] Daily rig evaluation — 2026-07-24 — 5/5 passed #89

Description

@github-actions

Summary

Task Description Typecheck Key finding
1 (reused) Git blame hotspot analyzer ✅ pass Initial attempt used repair({ maxTurns: 3 }) — invalid; repair is not callable
2 (reused) Code complexity scorer with subagent delegation ✅ pass Clean first pass
3 (reused) Commit message linter (conventional commit format) ✅ pass Clean first pass
4 (new) Workspace config drift detector ✅ pass defineTool handler initially had unused filename param → TS6133 error
5 (new) Git commit message conventional-format suggester ✅ pass steering("custom string") is invalid; steering() takes no arguments

All 5 tasks passed after corrections. Three errors were caught during generation, revealing real API ergonomics and documentation gaps.


Problems encountered

Problem 1 — repair mistakenly called as a function with options

What was tried: addons: [repair({ maxTurns: 3 })]

What went wrong: repair is an AgentAddon value, not a factory function. Calling it returns void | Promise<void>, which is not assignable to AgentAddon. The maxTurns option belongs in the agent spec directly.

Error:

error TS2554: Expected 2 arguments, but got 1.
error TS2322: Type 'void | Promise<void>' is not assignable to type 'AgentAddon'.

Minimal reproduction:

import { agent } from "rig";
import { repair } from "rig/addons";
// Wrong:
agent({ addons: [repair({ maxTurns: 3 })] });
// Correct:
agent({ maxTurns: 3, addons: repair });

Root cause: The API pattern of repair as a bare value (not a factory) is unintuitive for developers accustomed to addon systems that accept options (e.g., repair({ maxAttempts: 3 })). The SKILL.md shows the correct usage but not a "what NOT to do" example.


Problem 2 — defineTool handler unused parameter triggers TS6133

What was tried: handler: async ({ filename, content }) => { ... } where filename was destructured but only content was used.

What went wrong: TypeScript strict mode emits TS6133 for the unused filename parameter, causing typecheck failure.

Error:

error TS6133: 'filename' is declared but its value is never read.

Fix: Either use _filename convention or omit unused destructured keys.

Root cause: defineTool handlers run under strict TypeScript compilation. This is correct behavior, but generators (human or LLM) frequently destructure full parameter objects for readability. A note in SKILL.md about prefixing unused params with _ would help.


Problem 3 — steering() does not accept a string argument

What was tried: steering("Classify each commit as one of: feat, fix, chore, ...")

What went wrong: steering() accepts SteeringOptions (an object), not a plain string. Passing a string literal causes a type mismatch.

Error:

error TS2559: Type '"Classify each commit..."' has no properties in common with type 'SteeringOptions'.

Fix: Use steering() with no arguments (applies generic final-turn warning), or pass SteeringOptions.

Minimal reproduction:

// Wrong:
addons: [repair, steering("use only valid JSON")]
// Correct:
addons: [repair, steering()]

Root cause: The steering() API is easy to misuse — its purpose (appending a last-chance warning to the repair prompt) is not obvious from the name alone. The SKILL.md example correctly shows steering() but the "what it does" description might invite the belief that custom warning text is injectable as a positional argument.


Improvement opportunities

Missing schema helpers (s.*)

  • s.int vs s.integer: Both appear to work, but SKILL.md lists s.integer/s.int as aliases. Clarifying in one canonical name (or ensuring both appear in autocomplete) would reduce confusion.
  • No s.nullable shorthand visible in SKILL.md reference table — it appears in samples but the table omits it. Add s.nullable(shape) to the schema table in SKILL.md.

Missing prompt helpers (p.*)

  • p.writeOutput vs p.write: The distinction (write known content vs write LLM-generated output field) is critical but easily confused. A one-liner in SKILL.md differentiating them would prevent misuse.
  • p.readInput discoverability: Used in subagents that take typed input, but not mentioned explicitly in the "Choose prompt intents" table in SKILL.md.

Error message quality

  • repair called as function: The error Expected 2 arguments, but got 1 is not actionable — it doesn't hint that repair is not a factory. A custom diagnostic or a JSDoc @deprecated on incorrect overloads would help.
  • steering(string) mismatch: The error names SteeringOptions but doesn't say what that type is or where to find it. Including a usage hint in the type (via JSDoc) would make it self-documenting.

API ergonomics

  • repair is not a factory: The pattern of addons: repair vs addons: [repair, steering()] is inconsistent-feeling. repair is bare, steering() requires a call. A note clarifying why this asymmetry exists (or a repairAddon() factory alias) would reduce confusion.
  • maxTurns coupling with repair: When repair is added, the maxTurns budget must be manually set on the agent. It's not obvious that repair without maxTurns > 1 is effectively a no-op. A warning or a minimum-maxTurns enforcement in repair would help.
  • steering() purpose: The name suggests it "steers" the model, but it actually appends a warning on the final retry turn. Renaming to finalTurnWarning() or documenting with an example of the injected text would clarify intent.

Documentation gaps

  • SKILL.md doesn't show a defineTool handler with unused params and the _param convention for TypeScript compliance.
  • The "what NOT to do" section is absent: Adding a small anti-patterns table (e.g., repair(opts) ❌, steering("text") ❌) would catch the most common mistakes.
  • steering() without repair: SKILL.md says "Do not use steering() without repair" but this is not enforced at runtime or compile time. A lint or JSDoc note would make the constraint machine-checkable.

Tasks run today

  • (reused) Task 1: Git blame hotspot analyzer using p.bash git log with stat, s.record output keyed by file path with changeCount/lastAuthor/risk s.enum, repair addon maxTurns:3
  • (reused) Task 2: Code complexity scorer with subagent delegation: coordinator uses p.bash find + p.read per file via fileAnalyzer subagent, aggregates scores with s.enum verdict
  • (reused) Task 3: Commit message linter that checks conventional commit format using p.bash git log, repair addon with maxTurns:3, and s.enum status output
  • (new) Task 4: Workspace config drift detector using p.read/p.readOptional for config files, defineTool for JSON parsing, s.record(s.object) output with s.enum status, repair addon
  • (new) Task 5: Git commit message conventional-format suggester using p.bash git log, repair+steering addons, p.writeOutput for report, s.array(s.object) with category s.enum

Generated by Daily Rig Task Generator · sonnet46 75.7 AIC · ⌖ 4.78 AIC · ⊞ 5.6K ·

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions