[v3-3-test] Fix cursor pagination dropping rows when sorting by a nullable column (#68869) - #70739
Merged
Merged
Conversation
…#68869) * Fix cursor pagination dropping rows when sorting by a nullable column Cursor (keyset) paginated REST list endpoints silently dropped rows when the sort column was nullable. The keyset predicate and the generated ORDER BY disagreed on where NULLs sort, so once a page boundary fell on the NULL/non-NULL edge every row on one side of it was skipped with no error. This is silent data loss from the public API under a common sort such as start_date. Cross-dialect NULLS FIRST/LAST is not portable (unsupported on MySQL and old SQLite), so the cursor path now pins NULL placement with a portable CASE-based null-rank key shared by both the keyset ORDER BY and the keyset predicate, so the two can no longer disagree on any backend. The rank follows the column's sort direction, so NULLs order as the largest value (last when ascending, first when descending), matching PostgreSQL's default. PostgreSQL result order is therefore unchanged; SQLite and MySQL NULL ordering shifts to align with it. The cursor token format is unchanged (the rank is derived, not encoded) and offset pagination is untouched. * Add newsfragment for nullable-column cursor pagination fix * Preserve index usage for nullable-column cursor pagination The keyset ORDER BY ranked NULLs with a portable CASE expression so NULL placement was uniform across backends. A computed ORDER BY key cannot use a B-tree index, so large lists fall back to a full sort. Match each backend's native NULL placement in the keyset predicate instead and keep the ORDER BY a bare column, preserving the index; the fix still returns every row exactly once. EXPLAIN on Postgres confirms an Index Only Scan rather than a Sort. (cherry picked from commit 3b13e59)
vatsrahul1001
requested review from
bugraoz93,
choo121600,
henry3260,
jason810496,
pierrejeambrun,
rawwar and
shubhamraj-git
as code owners
July 30, 2026 09:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #68869 to
v3-3-testfor the Airflow 3.3.1 patch release. Cleangit cherry-pick -x(no conflicts).Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines — automated cherry-pick backport; code is the original author's, unchanged.