Skip to content

executor: move window executors into windows subpackage#67993

Merged
ti-chi-bot[bot] merged 2 commits into
pingcap:masterfrom
hawkingrei:executor-windows-package-only
Apr 23, 2026
Merged

executor: move window executors into windows subpackage#67993
ti-chi-bot[bot] merged 2 commits into
pingcap:masterfrom
hawkingrei:executor-windows-package-only

Conversation

@hawkingrei
Copy link
Copy Markdown
Member

@hawkingrei hawkingrei commented Apr 23, 2026

What problem does this PR solve?

Issue Number: ref #67989

Problem Summary:

The previous executor split was still too large for review. This PR narrows the
scope to the generic window executor move only, so later planner or ordered-input
work can be reviewed separately on top of the new package boundary.

What changed and how does it work?

  • move WindowExec and PipelinedWindowExec from pkg/executor to
    pkg/executor/windows
  • keep pkg/executor/builder.go as the entry point and delegate window executor
    construction to windows.Build
  • move the window SQL behavior tests from pkg/executor/window_test.go to
    pkg/executor/windows/window_sql_test.go
  • keep the focused executor tests in
    pkg/executor/windows/window_executor_test.go
  • remove the old root copies after the new package is wired in

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Test

go test -json -tags=intest ./pkg/executor/windows -run 'TestWindowExecutorsBasic|TestWindowFunctions|TestWindowFunctionsDataReference|TestSlidingWindowFunctions|TestIssue45964And46050|TestVarSampAsAWindowFunction|TestWindowReturnColumnNullableAttribute'
make bazel_prepare
make lint

Release note

None

Summary by CodeRabbit

  • Tests

    • Added tests validating pipelined vs non-pipelined window execution and column nullability for window functions.
  • Refactor

    • Moved and reorganized window executor implementation into a dedicated windows package.
  • Chores

    • Added build and test configuration for the new windows package and updated executor build wiring.

@ti-chi-bot ti-chi-bot Bot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 23, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 23, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: d13ecedf-1951-4057-985e-306c847c44e8

📥 Commits

Reviewing files that changed from the base of the PR and between 9ad10fa and 7493170.

📒 Files selected for processing (1)
  • pkg/executor/windows/window_executor_test.go

📝 Walkthrough

Walkthrough

Moves window executor code into a new pkg/executor/windows package with Bazel build/test targets, implements windows.Build to construct pipelined or non‑pipelined window executors, updates pkg/executor to delegate window construction, and adds/adjusts window tests under the new package.

Changes

Cohort / File(s) Summary
Bazel for windows package
pkg/executor/windows/BUILD.bazel
Adds go_library and go_test targets for the new windows package; test target configured flaky, sharded (7 shards), with parser/mysql/testkit dependencies.
Window executor builder (new package)
pkg/executor/windows/builder.go
New Build function that creates window executors: initializes base executor, computes partition/order columns, builds per-function agg funcs and result slots, and selects/configures pipelined (PipelinedWindowExec) or non‑pipelined (WindowExec + processors) execution with frame/range compare setup and error propagation.
Window executor implementation moved
pkg/executor/windows/pipelined_window.go, pkg/executor/windows/window.go
Implementation files moved/renamed to package windows (package declaration changed); no other functional changes in these files shown.
Executor package wiring changed
pkg/executor/BUILD.bazel, pkg/executor/builder.go
Removes in‑package window sources/tests from pkg/executor library, adds dependency on //pkg/executor/windows, and replaces inline window construction with a call to windows.Build.
Window tests added/updated
pkg/executor/windows/window_executor_test.go, pkg/executor/windows/window_sql_test.go
Adds tests for pipelined vs non‑pipelined execution and nullable attribute checks; test package updated to windows_test where applicable.

Sequence Diagram(s)

sequenceDiagram
    participant Builder as Builder
    participant Session as sessionctx.Context
    participant Child as ChildExecutor
    participant WindowsPkg as windows.Build
    participant Pipelined as PipelinedWindowExec
    participant NonPipelined as WindowExec/Processor

    Builder->>Session: read settings (EnablePipelinedWindowExec)
    Builder->>Child: provide child executor
    Builder->>WindowsPkg: Build(plan, Child, forcePipelined)
    WindowsPkg->>WindowsPkg: compute partition/order cols\nbuild agg funcs & result slots
    alt pipelined chosen
        WindowsPkg->>Pipelined: construct PipelinedWindowExec\nconfigure sliding-window metadata
        WindowsPkg-->>Builder: return PipelinedWindowExec
    else non-pipelined chosen
        WindowsPkg->>NonPipelined: select processor (agg/row/range)\nconfigure frame compare cols
        WindowsPkg-->>Builder: return WindowExec wrapping processor
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested labels

size/XXL

Suggested reviewers

  • windtalker
  • YangKeao
  • xzhangxian1008

Poem

🐰 I hopped through frames and leapt a refactor tree,
Windows now nest in a tidy new den for me,
Pipelined or patient, I stitch each plan seam,
Tests hum like carrots, bright as spring’s dream. 🥕✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the primary change: moving window executors from pkg/executor to a new windows subpackage.
Description check ✅ Passed The description includes all required sections: problem statement with issue number, detailed explanation of changes, completed test checklist, and release notes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@hawkingrei hawkingrei changed the title pkg/executor: add windows executor subpackage pkg/executor: move window executors into windows subpackage Apr 23, 2026
@hawkingrei hawkingrei force-pushed the executor-windows-package-only branch from f75f9fc to 5bb21f6 Compare April 23, 2026 06:13
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/executor/windows/BUILD.bazel`:
- Around line 28-38: The go_test target "windows_test" is not depending on the
windows library so it won't verify that the :windows package compiles; update
the go_test rule (name "windows_test", srcs "window_test.go") to either add
embed = [":windows"] or add ":windows" to deps so the test target depends on the
windows library and the build will fail if that package doesn't compile.

In `@pkg/executor/windows/pipelined_window.go`:
- Around line 103-108: OpenSelf currently resets many counters but leaves
PipelinedWindowExec.partialResults and PipelinedWindowExec.emptyFrame from
previous runs, causing stale aggregate state to be reused; update OpenSelf to
reinitialize partialResults to an empty slice (or nil) and set emptyFrame to
true (or its initial empty-state value) so each reuse starts with a fresh
partial result; modify the PipelinedWindowExec.OpenSelf implementation to
explicitly reset the fields partialResults and emptyFrame along with the other
initialized fields.

In `@pkg/executor/windows/window_test.go`:
- Around line 25-71: The tests currently drive SQL through testkit and therefore
exercise the existing SQL executor path instead of the new executor package;
update the tests to directly exercise the new builders/executors by invoking
windows.Build and instantiating/validating WindowExec, PipelinedWindowExec and
PartitionTopNWindowExec (or else mark these SQL-level tests to be skipped until
pkg/executor/builder.go is wired to pkg/executor/windows). Specifically, replace
the testkit-based assertions with unit tests that construct the logical window
plan (or use the package-level plan helpers), call windows.Build to produce
physical executors, and assert the executors produce the expected rows and
column nullable attributes for the functions tested
(sum/count/row_number/rank/dense_rank); include checks that both pipelined and
non-pipelined code paths are exercised.

In `@pkg/executor/windows/window.go`:
- Around line 287-317: The code path that handles empty ROWS frames (start >=
end) reuses stale p.partialResults from the previous non-empty frame; before
calling windowFunc.AppendFinalResult2Chunk you must reset the aggregate state
for each aggregate so the empty-frame semantics are produced. Locate
rowFrameWindowProcessor.appendResult2Chunk and, in the start >= end branch (and
the analogous RANGE branch mentioned around lines 417-453), clear or
reinitialize p.partialResults[i] for each windowFunc (and if a
slidingWindowAggFunc exists, reset its sliding state or mark
initializedSlidingWindow=false) so that AppendFinalResult2Chunk receives an
empty-frame partial result rather than the prior aggregation state.
- Around line 50-72: WindowExec retains mutable state across Open/Next calls
(fields executed, resultChunks, remainingRowsInChunk and internal processor
cursors) but never resets them, causing stale results on reuse; add/override an
Open method on WindowExec that calls BaseExecutor.Open(...) and then resets
executed=false, clears and nils resultChunks and remainingRowsInChunk, and
reinitializes any internal processor cursor/index fields used by consumeOneGroup
(e.g. partition/peer cursors) so the executor starts fresh on reopen. Ensure the
new Open signature matches the executor lifecycle and preserves the
BaseExecutor.Open call before clearing WindowExec-specific state.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: fe819d90-0d10-453e-b0ce-3b07ca70398a

📥 Commits

Reviewing files that changed from the base of the PR and between 2956483 and f75f9fc.

📒 Files selected for processing (6)
  • pkg/executor/windows/BUILD.bazel
  • pkg/executor/windows/builder.go
  • pkg/executor/windows/partition_topn_window.go
  • pkg/executor/windows/pipelined_window.go
  • pkg/executor/windows/window.go
  • pkg/executor/windows/window_test.go

Comment thread pkg/executor/windows/BUILD.bazel
Comment thread pkg/executor/windows/pipelined_window.go Outdated
Comment thread pkg/executor/windows/window_executor_test.go
Comment thread pkg/executor/windows/window.go Outdated
Comment thread pkg/executor/windows/window.go
@hawkingrei hawkingrei force-pushed the executor-windows-package-only branch 2 times, most recently from 4f1e69c to 67bc798 Compare April 23, 2026 06:36
@ti-chi-bot ti-chi-bot Bot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/needs-tests-checked do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 23, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/executor/windows/window_executor_test.go`:
- Around line 46-71: The test TestWindowReturnColumnNullableAttribute runs
window queries but doesn't enable window functions explicitly; before running
queries use the test kit to enable the feature by calling tk.MustExec("set
@@tidb_enable_window_function=1") (or equivalent session-set) at the start of
TestWindowReturnColumnNullableAttribute so checkNullable and the select
executions use a deterministic session with window functions enabled; place the
tk.MustExec call near the top of the test (after tk := testkit.NewTestKit(...))
and keep the change minimal and local to this test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: d7808589-f3d7-4be1-a099-fbfcdcdcfbf3

📥 Commits

Reviewing files that changed from the base of the PR and between f75f9fc and 67bc798.

📒 Files selected for processing (8)
  • pkg/executor/BUILD.bazel
  • pkg/executor/builder.go
  • pkg/executor/windows/BUILD.bazel
  • pkg/executor/windows/builder.go
  • pkg/executor/windows/pipelined_window.go
  • pkg/executor/windows/window.go
  • pkg/executor/windows/window_executor_test.go
  • pkg/executor/windows/window_sql_test.go
✅ Files skipped from review due to trivial changes (1)
  • pkg/executor/windows/window_sql_test.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • pkg/executor/windows/BUILD.bazel
  • pkg/executor/windows/window.go
  • pkg/executor/windows/pipelined_window.go

Comment thread pkg/executor/windows/window_executor_test.go
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 23, 2026

Codecov Report

❌ Patch coverage is 88.23529% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.4387%. Comparing base (2956483) to head (7493170).
⚠️ Report is 14 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #67993        +/-   ##
================================================
- Coverage   77.7924%   77.4387%   -0.3538%     
================================================
  Files          1982       1970        -12     
  Lines        549742     550794      +1052     
================================================
- Hits         427658     426528      -1130     
- Misses       121164     124264      +3100     
+ Partials        920          2       -918     
Flag Coverage Δ
integration 40.8465% <66.6666%> (+1.0493%) ⬆️
unit 76.6910% <88.2352%> (+0.3279%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4888% <ø> (ø)
parser ∅ <ø> (∅)
br 50.1111% <ø> (-12.9838%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hawkingrei hawkingrei force-pushed the executor-windows-package-only branch from 67bc798 to 67a3de4 Compare April 23, 2026 07:01
@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 23, 2026
@hawkingrei hawkingrei force-pushed the executor-windows-package-only branch from 67a3de4 to 0706b97 Compare April 23, 2026 07:02
@hawkingrei hawkingrei force-pushed the executor-windows-package-only branch from 0706b97 to 9ad10fa Compare April 23, 2026 07:19
Copy link
Copy Markdown
Contributor

@windtalker windtalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 23, 2026
@hawkingrei hawkingrei changed the title pkg/executor: move window executors into windows subpackage executor: move window executors into windows subpackage Apr 23, 2026
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Apr 23, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: windtalker, wshwsh12

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [windtalker,wshwsh12]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 23, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Apr 23, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-04-23 07:51:54.117865147 +0000 UTC m=+2238719.323225204: ☑️ agreed by windtalker.
  • 2026-04-23 07:58:44.086606556 +0000 UTC m=+2239129.291966603: ☑️ agreed by wshwsh12.

@hawkingrei
Copy link
Copy Markdown
Member Author

/retest

2 similar comments
@hawkingrei
Copy link
Copy Markdown
Member Author

/retest

@hawkingrei
Copy link
Copy Markdown
Member Author

/retest

@ti-chi-bot ti-chi-bot Bot merged commit 8bde239 into pingcap:master Apr 23, 2026
35 checks passed
premal pushed a commit to premal/tidb that referenced this pull request Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants