Skip to content

hints: expand success-path hint catalog — most tool invocations return empty hints #163

@HumanBean17

Description

@HumanBean17

Summary

In real-codebase testing, the hints field is empty on the majority of successful tool invocations. The current v1+v2+v3 catalog is weighted toward error/empty-result recovery (wrong kind, wrong direction, no matches) with only a few success-path hints (describe rollups, describe route/client, weak search scores). This leaves the agent without road signs during the most common workflow: successful calls where the next step is non-obvious.

The road-sign metaphor from the v1 propose is explicitly about all outputs, not just errors:

"A hint is a road sign attached to a tool output: it tells the agent the next reachable call, not what that call means or why to take it."

The v1 propose (§5) deferred HTTP_CALLS / ASYNC_CALLS / IMPLEMENTS / EXTENDS hint rows as "plausible v2 candidates." Now that the full SCHEMA-V2 has landed (ontology 14, Client/Producer endpoints, EDGE_SCHEMA), and real-codebase testing confirms agents benefit from success-path hints where they do exist (e.g., the describe(class_id) → hint about clients via members workflow), it's time to expand.

Current state of hints by tool

describe — has success-path hints (but incomplete, see #161)

neighbors — success-path hints are almost nonexistent

  • Non-empty results: only fuzzy-strategy hint (v2)
  • Empty results: structural mismatch hints (v3)
  • No hint on success with results for most calls

find — only error-path hints

  • Empty results → try resolve
  • Page full → paginate
  • No hint on success with results

search — only weak-score hint

  • Score spread within 10% → "results look weak"
  • No hint on normal success

resolve — only error-path hints

  • status=none → try search or find
  • status=many → tighten identifier
  • status=one → no hint (correct — the agent has the node id)

Candidate success-path hints to evaluate

These are not prescriptive — each needs evaluation against the road-sign principles (grounded in observable state, ≤120 chars, single call shape, no graph re-query).

neighbors success-path candidates

Trigger Hint shape Rationale
neighbors(sym_id, 'out', ['DECLARES']) returns methods "to see clients/routes on members, follow up with neighbors(member_ids,'out',['DECLARES_CLIENT']) or ['EXPOSES']" The 2-hop DECLARES → DECLARES_CLIENT pattern is the #1 source of agent confusion on real codebases. Currently the agent only gets this hint if it calls describe on the class first.
neighbors(sym_id, 'out', ['DECLARES_CLIENT']) returns client nodes "follow HTTP resolution: neighbors(client_ids,'out',['HTTP_CALLS'])" Client nodes are intermediate — the agent almost always wants the resolved route target next.
neighbors(sym_id, 'out', ['DECLARES_PRODUCER']) returns producer nodes "follow async resolution: neighbors(producer_ids,'out',['ASYNC_CALLS'])" Same pattern for async.
neighbors(route_id, 'in', ['EXPOSES']) returns handler method "handler's callers: neighbors(handler_ids,'in',['CALLS'])" After finding who handles a route, the agent often wants to trace callers.
neighbors(route_id, 'in', ['HTTP_CALLS']) returns client nodes "declaring methods: neighbors(client_ids,'in',['DECLARES_CLIENT'])" Reverse cross-service resolution: route → client → method.

find success-path candidates

Trigger Hint shape Rationale
find(kind='route', ...) returns routes "to see handlers: describe(id=route_id) or neighbors([route_id],'in',['EXPOSES'])" Routes are intermediate — the agent needs the handler method.
find(kind='client', ...) returns clients "to see resolved targets: neighbors([client_id],'out',['HTTP_CALLS'])" Clients are intermediate.

search success-path candidates

Trigger Hint shape Rationale
Top hit has symbol_id "to inspect: describe(id='sym:...')" Agent often doesn't know what to do after search returns chunks.

Design constraints

Any expansion must respect:

  1. Pure function of payload — no graph re-query in hint generation
  2. ≤5 cap, ≤120 chars — existing guardrails
  3. Road-sign discipline — one call per hint, no prose
  4. v1 catalog lock — new templates need a propose-doc amendment per HINTS-ROAD-SIGNS §7.11

The neighbors success-path candidates are the most impactful but the hardest: they require inspecting the result NodeRef.kind to know what was returned (e.g., "results are all client nodes → suggest HTTP_CALLS follow-up"). This is still observable from the payload without graph re-query, so it satisfies the purity constraint.

Proposal

Write a short propose under propose/ covering:

  1. Which success-path templates to add (evaluate the candidates above plus any others)
  2. Priority assignment (success-path hints should probably be PRIORITY_LEAF_FOLLOWUP or a new level)
  3. Whether neighbors success hints should be conditioned on result node kind
  4. Cap interaction — will success hints crowd out the existing rollup/override hints?

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions