Skip to content

[rig-tasks] Daily rig evaluation — 2026-08-03 — 10/10 passed #342

Description

@github-actions

Summary

Task Description Kind Typecheck Key finding
1 (reused) Stale branch detector v3 agent ✅ pass as const on enum returns required; clean p.bash + defineTool pattern
2 (reused) License header checker v3 agent ✅ pass Async tool with node:fs/promises + repair addon; input schema correctly threaded
3 (reused) YAML config diff v3 agent ✅ pass Two sync tools chained; p.readInput for caller-supplied paths works cleanly
4 (reused) Monorepo workspace lister v3 agent ✅ pass Async tool with JSON.parse; p.bash find + steering; spread for merged deps
5 (reused) GH Actions timing analyzer v2 agent ✅ pass Async tool with regex for multi-field detection; addons: [] explicit
6 (reused) Binary file detector v2 agent ✅ pass node:fs/promises open+read for binary sampling; union type cast needed
7 (new) Lockfile integrity checker agent ✅ pass s.optional(s.string) in tool params correctly typed; repair addon good fit
8 (new) GraphQL schema type extractor agent ✅ pass s.record(s.object(...)) root output; regex exec loop with typed match
9 (new) Path structure analyzer agent ✅ pass node:path basename import; explicit union type annotation on let needed
10 (new) CI log error classifier agent ✅ pass p.readInput for file path input; let with explicit union type for enum branches

Problems encountered

No failures this run.


Improvement opportunities

Missing or undiscoverable schema helpers (s.*)

  • s.int vs s.number confusion: Task 7 (lockfile checker) used s.int for line counts but then switched to s.number in the output — both are valid but the distinction isn't prominent in SKILL.md. A clearer decision table row (s.int = counts/line numbers; s.number = measurements/ratios) would help.
  • s.optional(s.string) in tool parameters: When a tool parameter might be absent (e.g., resolved?: string), using s.optional(s.string) in the parameters schema works but the generated TypeScript type needs manual annotation (resolved?: string) in the handler destructuring. This creates a gap between the schema and the handler signature.

Missing or undiscoverable prompt helpers (p.*)

  • p.readInput for non-path fields: When the input field is a string file path (like logFile: s.string), the prompt correctly uses p.readInput("logFile"), but SKILL.md only shows s.path in examples. Models may miss that p.readInput works for any string field treated as a path at runtime.
  • No p.glob + inline read shortcut: Several tasks used p.bash("find ...") to list files, then called a defineTool to read each. A p.glob(pattern) followed by tool reads is the intended pattern per SKILL.md but the lack of examples showing this full flow led to p.bash usage instead.

Error message quality

No typecheck failures this run, so no error messages to evaluate.

API ergonomics

  • as const requirement on enum returns: Every tool that returns an enum value via conditional logic requires as const annotations. This is a common stumbling block — the TypeScript literal type is lost without it. A lint rule or helper like s.enumValue("x") could eliminate the boilerplate.
  • Explicit union type annotations on let variables: Tasks 9 and 10 required explicit union type annotations on let category: "src" | "test" | ... before assignment. This is unavoidable TypeScript but may surprise models that expect the enum schema to infer the type automatically.
  • addons: [] vs omitting addons: Some tasks included addons: [] explicitly for clarity. SKILL.md says the default is no addons, but being explicit doesn't hurt. Could note that addons: [] and omitting addons are equivalent.

Candidate lint rules

Rule: require-as-const-on-enum-return

  • Invalid: return "fresh"; in a handler that's expected to return s.enum("fresh","stale","dead")
  • Valid: return "fresh" as const;
  • Why model-confusing: TypeScript widens string literals to string without as const, causing type mismatch against the enum schema at runtime validation.
  • Autofix: Yes — append as const to bare string literal returns in defineTool handlers.

Rule: no-bash-for-file-read

  • Invalid: p.bash("cat some-file.txt") when the path is known
  • Valid: p.read("some-file.txt")
  • Why model-confusing: Models occasionally use p.bash("cat ...") for known paths instead of p.read(). SKILL.md mentions this but doesn't flag it as an error.
  • Autofix: Partially — simple cat patterns can be replaced automatically.

Documentation gaps

  • SKILL.md's high-frequency decisions table doesn't include an example of p.readInput for a string-typed (non-s.path) input field. Adding a row for "Read a caller-supplied file path stored as s.string" would clarify this.
  • No example in SKILL.md shows a tool with s.optional(...) parameters combined with TypeScript optional destructuring. This pattern appeared in 2/10 tasks and required non-obvious type annotations.
  • The p.glob → subagent pattern is described but has no worked example showing the full tool-read loop. A single 5-line snippet would make it discoverable.

Tasks run today

  • (reused) Stale branch detector: uses p.bash git for-each-ref to classify branch ages
  • (reused) License header checker: accepts expectedHeader input, checks all .ts files
  • (reused) YAML config diff: accepts baseFile/targetFile, computes added/removed keys
  • (reused) Monorepo workspace lister: finds nested package.json files, extracts metadata
  • (reused) GitHub Actions workflow timing analyzer: finds .yml files, detects optimization opportunities
  • (reused) Binary file detector: samples git-tracked files for null bytes
  • (new) Lockfile integrity checker: verifies package-lock.json resolved/integrity fields
  • (new) GraphQL schema type extractor: extracts type/input/enum/interface/union from .graphql files
  • (new) Path structure analyzer: classifies workspace directories by purpose
  • (new) CI log error classifier: classifies CI log lines by error class and severity

Generated by Daily Rig Task Generator · sonnet46 132.3 AIC · ⌖ 9.38 AIC · ⊞ 6.8K ·

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions