Skip to content

fix(plugin): reject conversational acts in L2 policy induction (v3) - #2324

Open
smoryan wants to merge 2 commits into
MemTensor:mainfrom
smoryan:fix/l2-induction-conversational-acts
Open

fix(plugin): reject conversational acts in L2 policy induction (v3)#2324
smoryan wants to merge 2 commits into
MemTensor:mainfrom
smoryan:fix/l2-induction-conversational-acts

Conversation

@smoryan

@smoryan smoryan commented Sep 2, 2026

Copy link
Copy Markdown

Summary

Conversational acts — asking clarifying questions, requesting confirmation,
notifying, reporting status — are one-off dialogue behaviours, not reusable
procedures. When the L2 induction LLM writes them as a policy action anyway,
they later crystallize into dead skills that no agent can invoke. This PR adds
an explicit negative list to the induction system prompt and bumps the prompt
to v3 so the re-induction pass rewrites affected policies.

Problem

An internal 25-day production audit (2026-08-28) of the crystallized-skill
pool identified conversational-act policies as a recurring failure mode: they
are induced and stored like any other policy, but no agent can ever invoke
them, so they accumulate as dead weight. In the same window the pool received
207 fresh generations of which 98% of candidates were never invoked — the
quality of what enters the pool is the upstream half of that problem, and the
induction prompt had no guidance separating reusable checks from dialogue.

Change

  • core/llm/prompts/l2-induction.ts — new "Conversational acts are not
    policies" section: a negative list (clarify / confirm-with-user / notify /
    report status) plus a wrong/right example pair steering the model to express
    a reusable CHECK as the action instead (e.g. resolve the skill id via list
    search rather than ask the user to confirm it). Doc comment records the v3
    history. version: 23 — induction op names embed the prompt version
    (l2.${id}.v${version}), so the bump drives re-induction onto the new
    prompt on the next pass.
  • tests/unit/memory/l2/induce.test.ts,
    tests/unit/memory/l2/l2.integration.test.tsfakeLlm mock keys
    l2.l2.induction.v2.v3 (9 occurrences). Without this the fake LLM has
    no mock for the new op and 8 induction tests fail with llm_failed
    which doubles as confirmation the version bump takes effect.

Tests

  • npx vitest run tests/unit/llm/prompts.test.ts tests/unit/memory/l2/57 passed (10 files)
  • npx vitest run tests/unit/1549 passed, 2 skipped, 0 failed
  • npx tsc --noEmit → clean (exit 0)

Related

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g. code style improvements, lint)
  • Documentation update

How Has This Been Tested?

  • Unit tests (npx vitest run tests/unit/llm/prompts.test.ts tests/unit/memory/l2/ — 57 passed, 10 files)
  • Full unit suite (npx vitest run tests/unit/ — 1549 passed, 0 failed)
  • Type-check (tsc --noEmit clean)

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have created related documentation issue/PR in MemOS-Docs (if applicable)
  • I have linked the issue to this PR (if applicable)
  • I have mentioned the person who will review this PR

Policies whose core action is talking to the user (asking clarifying
questions, requesting confirmation, reporting status) are one-off
dialogue behaviours, not reusable procedures — once induced they
crystallize into dead skills that no agent can invoke.

Add a "Conversational acts are not policies" section to the L2
induction system prompt with a wrong/right example pair: if the
underlying trace really contains a reusable check, express the CHECK
itself as the action. Bump the prompt version v2 -> v3 so the
re-induction pass rewrites affected policies.
@Memtensor-AI Memtensor-AI added area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 2, 2026
@Memtensor-AI

Memtensor-AI commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2324
Task: f13e586fd64f9c82
Base: main
Head: fix/l2-induction-conversational-acts

🔍 OpenCodeReview found 1 issue(s) in this PR.


1. apps/memos-local-plugin/core/llm/prompts/l2-induction.ts (L90-L91)

The continuation line of the wrapped action string is misaligned compared to the established convention used in the existing Wrong/Right example block above (lines 72–75), where continuation lines are indented to align under the opening quote. Here it; on miss, ... is only indented by 4 spaces, while the opening quote starts at column 12. This inconsistency affects the prompt text that the LLM receives — in a template literal the indentation is preserved verbatim — so the continuation line will appear with leading whitespace that makes the example look malformed in the rendered prompt.

Suggested fix: indent the continuation to align with the opening ", matching the pattern used throughout the rest of the template.

💡 Suggested Change

Before:

    action: "resolve the skill id via skill list search before invoking
    it; on miss, fall back to keyword search instead of prompting"

After:

    action: "resolve the skill id via skill list search before invoking
             it; on miss, fall back to keyword search instead of prompting"

Generated by cloud-assistant via Open Code Review.

Co-Authored-By: LamzQ <linxlam@foxmail.com>
@smoryan

smoryan commented Sep 2, 2026

Copy link
Copy Markdown
Author

Thanks — confirmed and fixed in 42d68d5: the version-bump sentence now reads v3 (the v2 sentence was upstream history left stale by the v3 edit). Pure JSDoc, no behavior change; domain tests still 57/57.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (14/14 executed). memos_local_plugin/unit: 14/14. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-f13e586fd64f9c82-20260902095939: 16/17 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: fix/l2-induction-conversational-acts

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:plugin OpenClaw & Hermes status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: L2 policy induction crystallizes conversational acts into dead skills

3 participants