fix(sdk): pair the first-task rename_agent call to rename_session - #5908
Merged
Conversation
…o rename_session A live composer session showed the residual name-03 also measures: a post-reinforcement agent (new persona verbatim, rename_agent mounted and auto-allowed) called rename_session perfectly and still left 'Create an agent that sends me a telegram message' as its name. As a standalone 'when your purpose changes' judgment call the model reliably forgets rename_agent, so the persona now pairs it to the reliable call: the rename_session bullet adds 'if this is also your first task since you were created — your agent name is still a raw request or a placeholder — also call rename_agent in the same turn', and the standalone bullet shrinks to the later repurposing case. Both drift-locked copies (SDK _DEFAULT_AGENTS_MD and the services config.py fallback) move together. name-06-composer-first-task measures the pairing on the real composer shape: the workflow seeded with the raw request as its name, the session seeded with the client-auto-title truncation of the first message, both rename ops mounted, and rename_session AND rename_agent each expected exactly once — max_rename_calls is a single combined counter across both rename tools, so 2 plus the two min-1 tool checks pins one call each. The stored workflow name must lose the seeded raw request (a negative lookahead on the per-trial token) and keep is_application. name-03's header-seed shortcut stays.
dosubot
Bot
added
the
size:S
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe default persona now pairs first-task ChangesFirst-task self-naming
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Composer
participant rename_agent
participant rename_session
participant Scenario
Composer->>rename_agent: Rename the placeholder agent
Composer->>rename_session: Rename the raw session
Scenario->>Composer: Verify the combined two-call budget
Scenario->>rename_agent: Verify the workflow name
Scenario->>rename_session: Verify the session name
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: c4205f64-27ae-4f89-b4b1-d13d0fe9ecd7
📒 Files selected for processing (4)
benchmarks/agent-config-editing/scenarios/09-self-naming.jsonsdks/python/agenta/sdk/utils/types.pysdks/python/oss/tests/pytest/unit/agents/test_default_persona_self_naming.pyservices/oss/src/agent/config.py
Contributor
Railway Preview Environment
|
The session pattern passed on the seeded truncation (it contains 'standup summary') and the workflow check only rejected the token segment, so a failed persist could still pass. Both stored names must now differ from their seeds: the session name must not open with the seeded truncation, and the workflow name must not contain the raw request stem.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Live debugging of a real session showed a freshly created agent renaming its session perfectly ("Daily Arabic Poetry") while never once attempting
rename_agent, leaving the agent named "Create an agent that sends me a telegram message". The tool was available and auto-allowed, and the guidance even named "you were just created" as a trigger; as an independent judgment call the model reliably forgot it. The benchmark had already measured this residual (1 of 3 genuine misses on the agent-naming scenario).Changes
rename_agentis no longer a standalone clause. The default persona'srename_sessionbullet now carries the pairing: if this is the agent's first task since creation (its name still a raw request or a placeholder), also callrename_agentin the same turn. The standalone bullet shrinks to the later repurposing case. Both drift-locked copies (SDK_DEFAULT_AGENTS_MDand the services fallback) move together, and the drift-lock tests pin the pairing phrase.New benchmark scenario
name-06-composer-first-taskmirrors the real composer path: the workflow seeded with the raw request as its name (carrying a per-trial token), the session seeded with the client auto-title, both ops mounted, verbatim persona. It requires each rename exactly once and proves the raw-request name was replaced (the token must disappear).Validation (full self_naming run, 18 trials, results in the repo)
name-06's stored-name check passed 1 of 3: the two failures are NOT behavior — both trials attempted correctly and hit a newly exposed backend race where the second side-effecting call's result goes unobserved when the turn's pause ends (#5907, characterization in progress). The scenario stays in as the standing guard and goes fully green when that race is fixed.
Tests
SDK unit 2024 passed (including the extended drift locks), services unit 101 passed, scenario JSON validated, CI-pinned ruff clean.