feat: Budget skill metadata and surface trimming as a warning by xl-openai · Pull Request #18298 · openai/codex

xl-openai · 2026-04-17T06:45:32Z

Cap the model-visible skills section to a small share of the context window, with a fallback character budget, and keep only as many implicit skills as fit within that budget.

Emit a non-fatal warning when enabled skills are omitted, and add a new app-server warning notification

Record thread-start skill metrics for total enabled skills, kept skills, and whether truncation happened

xl-openai · 2026-04-17T06:49:48Z

+                    )
+                })
+                .or_else(|| implicit_skills.is_empty().then_some((0, 0, false)));
+            emit_thread_start_skill_metrics(&self.services.session_telemetry, skill_counts);


will move into rendered_skills 1 sec.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce11cd4e1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+    if !skill_lines.is_empty() {
+        lines.extend(skill_lines);
    }


Surface skill-list truncation inside the rendered skills section

When entries are dropped by budget, render_skills_section silently omits them and leaves ### Available skills without any truncation notice. That makes the model treat the list as complete and miss that additional enabled skills exist. The new behavior should include an explicit in-prompt trim warning whenever report.omitted_count > 0.

Useful? React with 👍 / 👎.

I don't think it makes any difference. Model can't do anything here.

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+    if !skill_lines.is_empty() {
+        lines.extend(skill_lines);
    }


Emit a trim warning inside the skills prompt

When report.omitted_count > 0, the rendered ### Available skills list is shortened but no in-prompt notice is added. The model sees an apparently complete list and cannot infer missing enabled skills, which breaks discoverability of trimmed skills.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+    let (skill_lines, report) = render_skill_lines(skills, budget);
    let mut lines: Vec<String> = Vec::new();
    lines.push("## Skills".to_string());
    lines.push("A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.".to_string());
    lines.push("### Available skills".to_string());
-
-    for skill in skills {
-        let path_str = skill.path_to_skills_md.to_string_lossy().replace('\\', "/");
-        let name = skill.name.as_str();
-        let description = skill.description.as_str();
-        lines.push(format!("- {name}: {description} (file: {path_str})"));
+    if !skill_lines.is_empty() {
+        lines.extend(skill_lines);
    }

    lines.push("### How to use skills".to_string());


Count fixed skills-section text in the metadata budget

The budget is only applied to per-skill lines, while the long static header/how-to text is always emitted. On small context windows this can exceed the intended context share even when zero skills are included, so the section is not actually capped by the configured budget.

Useful? React with 👍 / 👎.

current budget intentionally applies only to the per-skill metadata entries

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+fn prompt_scope_rank(scope: SkillScope) -> u8 {
+    match scope {
+        SkillScope::System => 0,
+        SkillScope::Admin => 1,
+        SkillScope::Repo => 2,
+        SkillScope::User => 3,


Keep repo/user skills ahead of system/admin in trimming

Budget ordering now prioritizes System/Admin before Repo/User. Existing skill precedence treats repo-local skills as highest priority, so truncation can drop the most task-relevant repo skills first and keep less relevant global ones.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+    if !skill_lines.is_empty() {
+        lines.extend(skill_lines);
    }


Emit a trim warning inside the skills prompt

When skill entries are omitted by budget, the rendered ### Available skills section is silently shortened. The model sees an apparently complete list and cannot infer that more enabled skills exist, which undermines discoverability of trimmed skills.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+    let (skill_lines, report) = render_skill_lines(skills, budget);
    let mut lines: Vec<String> = Vec::new();
    lines.push("## Skills".to_string());
    lines.push("A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.".to_string());
    lines.push("### Available skills".to_string());
-
-    for skill in skills {
-        let path_str = skill.path_to_skills_md.to_string_lossy().replace('\\', "/");
-        let name = skill.name.as_str();
-        let description = skill.description.as_str();
-        lines.push(format!("- {name}: {description} (file: {path_str})"));
+    if !skill_lines.is_empty() {
+        lines.extend(skill_lines);
    }

    lines.push("### How to use skills".to_string());


Count fixed skills text against metadata budget

Budgeting is applied only to per-skill lines. The long static header/how-to text is always emitted, so small context windows can still get a large skills block even when zero skills fit. This violates the intended cap for model-visible skill metadata.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+fn prompt_scope_rank(scope: SkillScope) -> u8 {
+    match scope {
+        SkillScope::System => 0,
+        SkillScope::Admin => 1,
+        SkillScope::Repo => 2,
+        SkillScope::User => 3,


Prioritize repo/user skills before system/admin when trimming

Trimming order now prefers System/Admin skills first. Existing skill precedence elsewhere is repo/user-first, so budget pressure can drop the most task-relevant local skills while keeping less relevant global ones.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-17T07:05:06Z

+        explicit_invocation_only: vec![],
+        allowlisted: vec![],
+        denylisted: vec![],
+        activated: vec![


Construct SkillLoadOutcome with current fields in test

This test initializes SkillLoadOutcome with fields like explicit_invocation_only and activated, which do not match the struct shape used by core-skills (skills, errors, disabled_paths, ...). The test target will not compile as written.

Useful? React with 👍 / 👎.

Co-authored-by: Codex <noreply@openai.com>

sayan-oai · 2026-04-17T08:30:40Z

+pub struct WarningNotification {
+    /// Concise warning message for the user.
+    pub message: String,
+}
+


should we add thread_id and make the warning thread-scoped? otherwise the notification isnt bound to a single thread and would show up globally regardless of what the active thread is

Why should it be tied to a specific thread? The backend should decide when to send it, but the UI should just render it regardless?

i think there's a notion of global vs thread server notification targets here. if that distinction is not relevant for how we plan to render it in app-server clients, global is fine

Let me make thread_id optional. In general, WarningNotification can be global, so tying it to a thread should be an option rather than a requirement. For the current case it does apply.

sayan-oai · 2026-04-17T08:52:41Z

    pub(crate) guardian_review_session: GuardianReviewSessionManager,
    pub(crate) services: SessionServices,
    pub(super) js_repl: Arc<JsReplHandle>,
+    pub(super) thread_start_skill_reported: AtomicBool,


i dont like tracking this in top-level Session. IIUC this is necessary because we can render skills multiple times in a thread (compaction), but we dont want to emit the warning multiple times.

i cant think of a better alternative though, other than making build_initial_context take or return a bool for the same reason (which also sucks).

@aibrahim-oai any ideas?

I will remove it for now.

sayan-oai · 2026-04-17T08:58:42Z

+pub const SKILL_METADATA_CONTEXT_WINDOW_PERCENT: usize = 2;
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum SkillMetadataBudget {


nit: does this need to be pub?

This one need to be used in core but I clean up other ones.

sayan-oai

i think the changes to how we render skills make sense. my main concern is how we track warning state.

also should have an integration test

Co-authored-by: Codex <noreply@openai.com>

jgershen-oai · 2026-04-18T00:07:49Z

@codex review

sayan-oai · 2026-04-18T00:10:22Z


+Recoverable configuration and initialization warnings use the existing `configWarning` notification: `{ summary, details?, path?, range? }`. App-server may emit it during initialization for config parsing and related setup diagnostics.
+
+Generic runtime warnings use the `warning` notification: `{ message }`. App-server emits this for non-fatal warnings from the core event stream, including cases where not all enabled skills are included in the model-visible skills list for a session.


update to show optional threadid

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a514dd4cd4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-18T00:11:47Z

    let mut lines: Vec<String> = Vec::new();
    lines.push("## Skills".to_string());
    lines.push("A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.".to_string());
    lines.push("### Available skills".to_string());
-
-    for skill in skills {
-        let path_str = skill.path_to_skills_md.to_string_lossy().replace('\\', "/");
-        let name = skill.name.as_str();
-        let description = skill.description.as_str();
-        lines.push(format!("- {name}: {description} (file: {path_str})"));
+    if !skill_lines.is_empty() {
+        lines.extend(skill_lines);
    }

    lines.push("### How to use skills".to_string());


Budget the full skills section, not only skill lines

render_skill_lines enforces budget only for per-skill bullets, but render_skills_section always appends the long static skills instructions afterward. With very small context windows, this still injects a large fixed block, so the model-visible skills section is not actually capped to the configured budget and can still consume substantial prompt space.

Useful? React with 👍 / 👎.

yes that's the design.

chatgpt-codex-connector · 2026-04-18T00:11:47Z

+            SkillRenderSideEffects::ThreadStart {
+                session_telemetry: &self.services.session_telemetry,
+            },
+        );
+        if let Some(rendered_skills) = rendered_skills {
+            if rendered_skills.emit_warning {
+                self.send_event_raw(Event {
+                    id: String::new(),
+                    msg: EventMsg::Warning(WarningEvent {
+                        message: THREAD_START_SKILLS_TRIMMED_WARNING_MESSAGE.to_string(),
+                    }),


Gate thread-start side effects to true thread start

build_initial_context always passes SkillRenderSideEffects::ThreadStart and emits the trimmed-skills warning on truncation. This helper is reused outside initial thread setup (e.g., context reinjection/compaction), so "thread-start" metrics and warnings can be emitted multiple times in one thread, skewing telemetry and producing repeated warnings mid-session.

Useful? React with 👍 / 👎.

consequence of not tracking metric emission state in Session, which i still think is worth it. should follow up with an improvement to this

sayan-oai · 2026-04-18T00:18:43Z

+        warning.message,
+        "Some enabled skills were not included in the model-visible skills list for this session. Mention a skill by name or path if you need it."


nit; this uses "warning emitted" as a proxy for "skills trimmed". can we check that the skills are actually trimmed/omitted from the outgoing request?

you can extend the app-server integration test or use test_codex, whichever is clearer.

mzeng-openai and others added 4 commits April 16, 2026 14:52

Add skill metadata budget handling

47f5026

Warn on trimmed skills at thread start

82e5ae9

Resolve merge conflicts after pulling main

64cd78e

Add warning + telemetry.

ce11cd4

xl-openai requested review from aibrahim-oai, mzeng-openai and sayan-oai April 17, 2026 06:45

xl-openai commented Apr 17, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Apr 17, 2026

View reviewed changes

xl-openai and others added 2 commits April 17, 2026 00:22

Refactor emit_thread_start_skill_metrics.

f0ad9fc

Merge branch 'main' into xl/skills

69c32a8

Co-authored-by: Codex <noreply@openai.com>

xl-openai force-pushed the xl/skills branch from 2dbda4e to 69c32a8 Compare April 17, 2026 08:10

sayan-oai reviewed Apr 17, 2026

View reviewed changes

Comment thread codex-rs/tui/src/chatwidget.rs Outdated

sayan-oai reviewed Apr 17, 2026

View reviewed changes

xl-openai force-pushed the xl/skills branch from 01ebacd to 0a76411 Compare April 17, 2026 17:58

xl-openai and others added 2 commits April 17, 2026 11:12

Address comments.

99e1fc8

Merge origin/main into xl/skills

fb7f9c0

Co-authored-by: Codex <noreply@openai.com>

xl-openai force-pushed the xl/skills branch from 0a76411 to fb7f9c0 Compare April 17, 2026 18:18

xl-openai requested a review from sayan-oai April 17, 2026 18:18

xl-openai force-pushed the xl/skills branch from 727be3a to 2e0a8d5 Compare April 17, 2026 19:07

cleanup.

a514dd4

xl-openai force-pushed the xl/skills branch from 2e0a8d5 to a514dd4 Compare April 17, 2026 19:28

pakrym-oai reviewed Apr 17, 2026

View reviewed changes

Comment thread codex-rs/core/src/session/tests.rs

sayan-oai reviewed Apr 18, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Apr 18, 2026

View reviewed changes

sayan-oai reviewed Apr 18, 2026

View reviewed changes

sayan-oai approved these changes Apr 18, 2026

View reviewed changes

More cleanup.

0d1de00

xl-openai merged commit 3f7222e into main Apr 18, 2026
25 checks passed

xl-openai deleted the xl/skills branch April 18, 2026 01:11

github-actions bot locked and limited conversation to collaborators Apr 18, 2026


		Recoverable configuration and initialization warnings use the existing `configWarning` notification: `{ summary, details?, path?, range? }`. App-server may emit it during initialization for config parsing and related setup diagnostics.

		Generic runtime warnings use the `warning` notification: `{ message }`. App-server emits this for non-fatal warnings from the core event stream, including cases where not all enabled skills are included in the model-visible skills list for a session.

		warning.message,
		"Some enabled skills were not included in the model-visible skills list for this session. Mention a skill by name or path if you need it."

Conversation

xl-openai commented Apr 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xl-openai Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xl-openai Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sayan-oai left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jgershen-oai commented Apr 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Apr 18, 2026

Choose a reason for hiding this comment

xl-openai Apr 17, 2026 •

edited

Loading

xl-openai Apr 17, 2026 •

edited

Loading

sayan-oai left a comment •

edited

Loading