Skip to content

skill_discovered event re-emitted on every step, inflating telemetry counts #68

@byapparov

Description

@byapparov

Problem

The skill_discovered NDJSON event is emitted once per skill every time the skill tool schema is registered — which happens once per step. This inflates downstream telemetry counts by a factor equal to the number of steps in the session.

Evidence

A recent execution bundled 2 skills into the workspace and ran 19 steps. Telemetry recorded:

skill_discovered: 38
step_start: 19

38 = 2 skills × 19 steps. Exact match.

Root cause

src/tool/skill.ts (SkillTool):

export const SkillTool = Tool.define("skill", async (ctx) => {
  const skills = await Skill.all()

  // Emit per-skill discovery events when descriptions are registered in tool schema
  if (ctx?.sessionID) {
    for (const skill of skills) {
      Bus.publish(Session.Event.SkillDiscovered, {
        sessionID: ctx.sessionID,
        name: skill.name,
        description: skill.description,
        location: skill.location,
      })
    }
  }
  ...
})

The factory runs on every tool-schema registration, so each step re-publishes the full skill list.

Impact

  • Downstream consumers (Session Control executor telemetry, dashboards) misreport "skills used" as a huge number — e.g. a 2-skill bundle appears as 38 "discoveries".
  • Makes it hard to distinguish genuine skill-loading anomalies from normal multi-step runs.
  • The event name suggests one-time discovery, which is what consumers assume.

Suggested fix

Emit skill_discovered once per (sessionID, skillName) — dedupe on the CLI side, either:

  1. Move the emission out of the tool factory into session start / skill-registry initialisation, OR
  2. Track emitted (sessionID, name) pairs in a Set and skip duplicates, OR
  3. Rename the event to skill_available (per-step) and add a new one-shot skill_loaded at session start — but this is a breaking schema change.

Option 1 is cleanest and matches the name's intent.

Version

Observed on @aictrl/cli@0.3.3. Same code path exists in 0.3.2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions