Skip to content

fix(migration): support RESP3 key enumeration - #730

Open
Kuang-xianxin wants to merge 1 commit into
redis:mainfrom
Kuang-xianxin:codex/fix-migration-resp3-enumeration
Open

fix(migration): support RESP3 key enumeration#730
Kuang-xianxin wants to merge 1 commit into
redis:mainfrom
Kuang-xianxin:codex/fix-migration-resp3-enumeration

Conversation

@Kuang-xianxin

@Kuang-xianxin Kuang-xianxin commented Sep 6, 2026

Copy link
Copy Markdown

Closes #713
Closes #714

On RESP3 connections, migration key enumeration reads byte-keyed FT.INFO maps as string-keyed dictionaries, treating failed or incomplete indexes as ready. Healthy indexes then hit a KeyError when the aggregate cursor's result map is sliced as a RESP2 row list.

Normalize FT.INFO before checking readiness and extracting SCAN prefixes. Preserve a real zero for percent_indexed instead of treating it as the missing-value default. Share aggregate key extraction between the sync and async executors, handling both RESP2 rows and RESP3 result maps while retaining cursor pagination and cleanup.

Validation:

  • The new regression file fails 16 cases on the original code (12 compatibility/control cases pass).
  • All 120 related enumeration, executor, planner and batch-migration unit tests pass with redis-py 8.1.0 and separately with 6.3.0. Tests cover decoded and byte responses, failed/partial/zero-progress indexes, scoped SCAN fallback, multiple cursor pages, empty pages and cursor cleanup.
  • Full source mypy passes (120 files); repository-wide Black/isort and scoped codespell pass.
  • These tests use protocol-shaped mock responses. Run with pytest --confcutdir=tests/unit to exclude the root Docker startup fixture. The Redis-backed integration suite was not run because Docker is unavailable locally.

Suggested release label: auto:patch.

AI assistance: implementation, tests and this description were prepared with Codex.


Note

Medium Risk
Changes core migration enumeration logic; incorrect parsing could skip or mis-count documents during migrations, though behavior is guarded by SCAN fallbacks and new regression tests.

Overview
Fixes migration document enumeration on RESP3 Redis connections where FT.INFO and FT.AGGREGATE return structured maps instead of RESP2-style lists.

FT.INFO responses are normalized with convert_bytes before readiness checks and prefix extraction, so byte-keyed maps behave like string dicts. percent_indexed now treats an explicit 0 as incomplete indexing (SCAN fallback) instead of defaulting missing values to 1.0.

Adds shared _extract_aggregate_keys for sync and async executors to read __key from both RESP2 cursor pages and RESP3 aggregate result maps, with cursor pagination and cleanup unchanged. New unit tests cover RESP2/RESP3, decoded vs byte wire formats, incomplete indexes, aggregate errors, and cursor release.

Reviewed by Cursor Bugbot for commit d55117f. Bugbot is set up for automated code reviews on this repo. Configure here.

Normalize readiness and prefix metadata, preserve zero indexing progress, and share RESP2/RESP3 aggregate parsing across sync and async executors.

Fixes redis#713

Fixes redis#714

Assisted-by: Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant