fix(core): emit warn logs when LLM unavailable + configurable tool-heavy skip thresholds#1557
Merged
Conversation
…egradation All LLM-dependent modules were silently falling back to heuristic/skip when the LLM client was null (e.g. host bridge not attached), making it very hard to diagnose why the self-evolution pipeline produced no results. This was reported in the TaskCLI demo test (2026-04-27). Changes: 1. human-scorer.ts: upgrade `score.heuristic` debug → `score.llm_unavailable` warn with explicit reason (config disabled vs client null) 2. induce.ts (L2): add `l2.induce.llm_unavailable` warn when llm is null (previously returned silently with no log at all) 3. crystallize.ts (Skill): upgrade `skill.crystallize.llm_disabled` info → `skill.crystallize.llm_unavailable` warn with reason 4. abstract.ts (L3): add `l3.abstract.llm_unavailable` warn when llm is null (previously returned silently with no log at all) 5. reward.ts: add one-time `reward.llm_unavailable` warn at runner creation explaining the full downstream impact 6. reward.ts + types.ts + schema.ts + defaults.ts: extract hardcoded `toolTurns >= totalTurns * 0.7` and `assistantContentChars < 80` into configurable `algorithm.reward.toolHeavyRatio` (default 0.7) and `algorithm.reward.minAssistantCharsForToolHeavy` (default 80) 7. Fix test mocks to include the two new RewardConfig fields
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
warn级别日志,便于诊断"自我进化链路未跑通"的根因decideSkipReason中硬编码的工具密集型判断阈值(toolTurns >= totalTurns * 0.7和assistantContentChars < 80)提取为可配置参数背景
TaskCLI demo 测试(2026-04-27)发现:当 OpenClaw 的 HostLlmBridge 未注入时,
memos-local-plugin内部 LLM 客户端为空,导致 reward/L2/Skill/L3 全部静默退化为 heuristic/跳过,但日志中几乎看不到任何提示。此外decideSkipReason的工具密集型启发式在 OpenClaw 大量使用工具的场景下误判过多 episode 为 skip。Changes
core/reward/human-scorer.tsdebug→warnscore.llm_unavailable,附原因core/memory/l2/induce.tswarnl2.induce.llm_unavailable(之前无日志)core/skill/crystallize.tsinfo→warnskill.crystallize.llm_unavailable,附原因core/memory/l3/abstract.tswarnl3.abstract.llm_unavailable(之前无日志)core/reward/reward.tswarnreward.llm_unavailable;decideSkipReason读取可配阈值core/reward/types.tsRewardConfig新增toolHeavyRatio+minAssistantCharsForToolHeavycore/config/schema.tscore/config/defaults.tstests/unit/reward/*.test.tsTest plan
llm.provider: host但不注入 bridge → 日志应出现reward.llm_unavailable、score.llm_unavailable、l2.induce.llm_unavailable等 warnconfig.yaml设置algorithm.reward.toolHeavyRatio: 0.9→ 工具密集型 episode 不再被误判 skiptests/unit/reward/)