NES: experiment-driven global token budget for the inline-edits prompt#323948
Merged
Conversation
Under the opt-in global budget experiment, the current file now draws its clip budget from the shared pool (its `shares.currentFile` slice of `totalTokens`) and donates whatever it does not use to the cascade as the initial surplus. When the global budget is disabled (prod default) behavior is byte-identical, and the new defaults (totalTokens 8000, shares in eighths) reproduce today's per-part `maxTokens` caps exactly. - xtabPromptOptions: add `GlobalBudgetSharePart`, `currentFileBudget()` and `validate()`; grow `shares` to include `currentFile`; bump `DEFAULT_TOTAL_TOKENS` to 8000 and rebalance `DEFAULT_SHARES`. - promptCrafting: seed the cascade's initial surplus from the current file's leftover budget; delegate validation to the namespace helper. - xtabProvider: size and clip the current file from the pool and compute the surplus, gated on `globalBudget` being defined. - Tests: volume-neutral invariant, `currentFileBudget`/`validate` units, and provider-level regression + new-behavior coverage. - Docs: rewrite globalBudgetCascade.md for the seeded-surplus design. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ring
Replace the two experiment-driven global-budget settings
(globalBudget.enabled boolean + globalBudget.totalTokens number) with a
single JSON-encoded string setting, modelled after modelConfigurationString,
that defines totalTokens, order, and shares together.
- configurationService: remove InlineEditsXtabGlobalBudgetEnabled and
InlineEditsXtabGlobalBudgetTotalTokens; add InlineEditsXtabGlobalBudget
(string | undefined, ExperimentBased, default undefined).
- xtabPromptOptions: add GlobalBudgetOptions.VALIDATOR (all top-level fields
optional; shares must be complete when present) and a pure
GlobalBudgetOptions.fromConfigString(): Result<GlobalBudgetOptions, string>
that JSON-parses, structurally validates, merges over the defaults, then
runs the semantic validate(). Never throws.
- xtabProvider: inject ITelemetryService; resolve the budget via
getGlobalBudget(), returning undefined (disabled, identical to prod) when
the string is unset/empty/invalid and emitting incorrectNesGlobalBudgetConfig
telemetry on parse/validation failure.
- Tests: migrate the provider global-budget tests to the JSON string and add
fromConfigString unit tests.
- Docs: rewrite the globalBudgetCascade.md Wiring section and migration note.
No regression: an unset string keeps the byte-identical legacy path. '{}'
enables the budget with the volume-neutral defaults; {"totalTokens":N} only
overrides the pool size.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e leftover Under a global budget the current file is now clipped LAST, sized to its base share plus whatever the budget cascade left unused (`currentFileBudget + finalSurplus`), so it trims less and reuses leftover budget. The cascade runs first (seeded with 0, the current file never donates) and is threaded into `getUserPrompt` via `precomputedCascade` so it renders exactly once. This replaces the earlier two-mode design: the opt-in `reuseLeftoverForCurrentFile` flag and the seeded-surplus "donate-forward" path are dropped, along with all `currentFileBudgetSurplus`/`initialSurplus` threading. Clip-last is now the single behavior under a global budget. The validator hardening (rejecting non-finite or negative `totalTokens`/shares) is kept. No production regression: when `globalBudget` is undefined the current file is still clipped to its own `currentFile.maxTokens` with the legacy await ordering, byte-identical to before. The global budget remains experiment-gated and off in prod. Tests and globalBudgetCascade.md updated for the clip-last-only behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…iming - xtabProvider: record setNLinesOfCurrentFileInPrompt inside both budget branches. In the legacy/prod branch the clip happens before the context-gathering awaits, so the telemetry call is moved back there to match prod timing exactly (it is still emitted when a request is cancelled mid-gathering). The clip-last branch records it after the cascade, as the clip is inherently last there. - xtabPromptOptions: fix two stale JSDoc blocks that still described the removed donate-forward behavior; describe clip-last instead. - globalBudgetCascade.md: correct the conservation bound to T·(Σ shares) and document the share-sum tolerance (over-allocation ≤ ~1e-3·T). - promptCrafting.spec: add a cascade test asserting finalSurplus shrinks as the cascade consumes budget (current file reuses less leftover). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an experiment-driven global token budget for the NES (inline next-edit suggestions) prompt, replacing fixed per-section maxTokens caps with a pooled totalTokens budget that is allocated by ordered “shares”, with unused budget cascading forward and the current file clipped last to reuse leftover.
Changes:
- Add
GlobalBudgetOptionsparsing/validation (single JSON config string) and update configuration to a single experiment-driven setting. - Implement clip-last behavior by precomputing the cascade in the provider and clipping the current file to
currentFileBudget + finalSurplus. - Add/extend unit and integration tests plus documentation for the cascade model and guarantees.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/platform/inlineEdits/test/common/xtabPromptOptions.spec.ts | Adds unit tests for GlobalBudgetOptions defaults, validation, and JSON parsing behavior. |
| extensions/copilot/src/platform/inlineEdits/common/dataTypes/xtabPromptOptions.ts | Defines global-budget types/defaults and implements validation + config-string parsing. |
| extensions/copilot/src/platform/configuration/common/configurationService.ts | Replaces the prior global-budget settings with a single JSON-string experiment setting. |
| extensions/copilot/src/extension/xtab/test/node/xtabProvider.spec.ts | Adds provider-layer tests to confirm clip-last causes the current-file region to grow with pooled budget and leftover. |
| extensions/copilot/src/extension/xtab/test/common/promptCrafting.spec.ts | Adds cascade-layer tests for surplus carry-forward, final surplus behavior, and precomputed cascade identity. |
| extensions/copilot/src/extension/xtab/node/xtabProvider.ts | Implements global-budget resolution/parsing + telemetry on invalid configs; runs cascade first then clips current file last. |
| extensions/copilot/src/extension/xtab/common/promptCrafting.ts | Adds exported runGlobalBudgetCascade/CascadeResult with finalSurplus and supports precomputed cascades in getUserPrompt. |
| extensions/copilot/src/extension/xtab/common/globalBudgetCascade.md | Updates design documentation with clip-last semantics, validation rules, and config wiring/migration notes. |
Review details
- Files reviewed: 8/8 changed files
- Comments generated: 1
- Review effort level: Low
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pulls the ~55-line global-budget/legacy if-else out of doGetNextEditWithSelection
into a private gatherContextAndClipCurrentFile method returning
Result<{ clippedTaggedCurrentDoc, areaAroundCodeToEdit, precomputedCascade,
langCtx, neighborSnippets }, NoNextEditReason>. Behavior, clip-first/clip-last
ordering, and nLinesOfCurrentFileInPrompt telemetry timing are unchanged.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ulugbekna
marked this pull request as ready for review
July 2, 2026 15:12
ulugbekna
enabled auto-merge (squash)
July 2, 2026 15:12
main (#322382) lowered currentFile.maxTokens 2000->1500, which broke the global-budget 'volume-neutral defaults' invariant (floor(total*share) must equal each part's legacy cap). Recalibrate DEFAULT_SHARES to cap/7500 and set DEFAULT_TOTAL_TOKENS to 7500 (the sum of the merged per-part caps: 1500 + 2000 + 2000 + 1000 + 1000), so shares still sum to exactly 1 and every part's base allocation reproduces its cap. finalSurplus stays 6000 (7500 - 1500). Update dependent specs (literal 8000 -> DEFAULT_TOTAL_TOKENS, stale 2000 cap comments -> 1500) and the design doc (defaults table, worked examples, effective caps, ordering caveat, migration note) for T=7500 and currentFile base 1500. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
vijayupadya
approved these changes
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The NES (inline next-edit suggestions) prompt is built from several sections: the current file around the cursor, language context, recently viewed documents, neighbor files, and diff history. Today each section has its own fixed
maxTokenscap, so budget unused by one section cannot be reused by another, and the current file is clipped independently of everything else. This makes it hard to experiment with how the token budget is split across sections and how much of the current file we keep.What
This PR introduces an experiment-driven global budget that pools all sections into a single
totalTokensbudget with a configurableorderand per-sectionshares, and reuses leftover budget instead of wasting it.Key pieces:
modelConfigurationString. This keeps it easy to roll out and tune via experimentation.order; whatever budget a section does not consume cascades forward as surplus to the next section, so no allocation is stranded.sharesentry but is clipped LAST, after the cascade runs. It receives its base share plus whatever the cascade left unused (the final surplus), so it reuses leftover budget and trims less when the other sections are light.Safety / prod behavior
The entire feature is gated behind the
globalBudgetoption. When it isundefined(the production default), every section keeps its ownmaxTokenscap and there is no cross-section reuse. The legacy code path is preserved so prod output is unchanged. Reviewers can focus on theglobalBudget !== undefinedbranch.One subtlety worth a careful look: in the global-budget path the current file is clipped after the context-gathering awaits (required for clip-last), so the
nLinesOfCurrentFileInPrompttelemetry timing was deliberately kept in its original position on the legacy branch so that field is still emitted when a request is cancelled mid-gathering.Tests
promptCrafting.spec.tscover surplus carry-forward, the clip-last reuse behavior (final surplus shrinks as the cascade consumes budget), and precomputed-cascade identity.xtabProvider.spec.tsverify the current-file region grows with the total budget and absorbs leftover versus the legacy path.xtabPromptOptions.spec.ts.The behavior, guarantees (including the conservation bound and share-sum tolerance), and worked examples are documented in
globalBudgetCascade.md.