Problem
Skills are learned once and then trusted indefinitely, even after repeated failures.
Although the reinforcement system tracks confidence and flags skills whose confidence drops below 0.4, that information is never used when selecting or executing skills.
This creates a feedback loop where low-quality skills continue to be enforced despite accumulating evidence that they are unreliable.
Current behavior
-
Flagged skills are still treated as mandatory
task_tracker begin always instructs the model to use the matched skill, regardless of its confidence.
A skill that has failed repeatedly receives the same mandatory guidance as a highly successful skill.
-
Failure history is never used
Failed executions append entries to negative_examples, but those lessons are never consumed to improve the skill.
As a result, incorrect steps remain unchanged and the same failures repeat.
-
There is no recovery path
Once a skill becomes unreliable, the only options are to:
- manually edit it,
- delete it, or
- continue using it.
The framework provides no built-in mechanism for repairing low-confidence skills.
-
Existing skills can be silently overwritten
crystallize can recreate an existing skill, resetting its confidence and discarding accumulated failure history without explicit approval.
Expected behavior
The framework should:
- respect skill confidence during execution,
- provide a mechanism to repair low-confidence skills,
- require explicit user approval before applying repairs,
- preserve accumulated learning history,
- prevent accidental overwriting of existing skills.
Additional context
This issue is addressed by PR #102.
Problem
Skills are learned once and then trusted indefinitely, even after repeated failures.
Although the reinforcement system tracks confidence and flags skills whose confidence drops below 0.4, that information is never used when selecting or executing skills.
This creates a feedback loop where low-quality skills continue to be enforced despite accumulating evidence that they are unreliable.
Current behavior
Flagged skills are still treated as mandatory
task_tracker beginalways instructs the model to use the matched skill, regardless of its confidence.A skill that has failed repeatedly receives the same mandatory guidance as a highly successful skill.
Failure history is never used
Failed executions append entries to
negative_examples, but those lessons are never consumed to improve the skill.As a result, incorrect steps remain unchanged and the same failures repeat.
There is no recovery path
Once a skill becomes unreliable, the only options are to:
The framework provides no built-in mechanism for repairing low-confidence skills.
Existing skills can be silently overwritten
crystallizecan recreate an existing skill, resetting its confidence and discarding accumulated failure history without explicit approval.Expected behavior
The framework should:
Additional context
This issue is addressed by PR #102.