Skip to content

Flagged skills are ignored and cannot recover from repeated failures #103

Description

@Nivesh353

Problem

Skills are learned once and then trusted indefinitely, even after repeated failures.

Although the reinforcement system tracks confidence and flags skills whose confidence drops below 0.4, that information is never used when selecting or executing skills.

This creates a feedback loop where low-quality skills continue to be enforced despite accumulating evidence that they are unreliable.

Current behavior

  1. Flagged skills are still treated as mandatory

    task_tracker begin always instructs the model to use the matched skill, regardless of its confidence.

    A skill that has failed repeatedly receives the same mandatory guidance as a highly successful skill.

  2. Failure history is never used

    Failed executions append entries to negative_examples, but those lessons are never consumed to improve the skill.

    As a result, incorrect steps remain unchanged and the same failures repeat.

  3. There is no recovery path

    Once a skill becomes unreliable, the only options are to:

    • manually edit it,
    • delete it, or
    • continue using it.

    The framework provides no built-in mechanism for repairing low-confidence skills.

  4. Existing skills can be silently overwritten

    crystallize can recreate an existing skill, resetting its confidence and discarding accumulated failure history without explicit approval.

Expected behavior

The framework should:

  • respect skill confidence during execution,
  • provide a mechanism to repair low-confidence skills,
  • require explicit user approval before applying repairs,
  • preserve accumulated learning history,
  • prevent accidental overwriting of existing skills.

Additional context

This issue is addressed by PR #102.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions