Migrate Slots & Keys Optimizations

[vazois]

This PR adds following optimizations and fixes to migrate operations

• For both KEYS and SLOTS option, we now use a sketch to avoid keeping track of the actual keys in a dictionary which is expensive.
• For SLOTS option enable support parallel scan tasks to speedup migration.
• For SLOTS option ensure no keys are lost because of copy-update during the scan.
• Expose api to suspend reviv.

NOTES:
When using the SLOTS option, we do not know in advance which keys need to be migrated.
We have to scan the store log and make sure any relevant keys are captured and transmitted to the target node.
Before starting the scan, the migration driver captures the BeginAddress (BA) and CurrentTailAddress (CTA).
This is done after the slot transitions to MIGRATING state and is coordinated under epoch protection.
The migration driver will perform a scan between (BA,CTA). For this reason, we need to ensure that we scan all records created within that region, including those that were copy-updated or deleted.
After gathering the associated keys, we lock them from write operations and perform an additional random read to ensure they are still live.
We transmit every live key to the target node and continue the scan from where we left off otherwise break.

[Copilot]

Pull Request Overview

This PR introduces a bloom-filter–based sketch for key tracking during both KEYS and SLOTS migrations, adds parallel slot-scan tasks to accelerate slot migrations, and extends scan APIs to support cursors, max addresses, and tombstone visibility.

• Replace MigratingKeysWorkingSet with a space-efficient Sketch and update migrate commands to use it
• Enable parallel scan tasks in MigrationDriver for SLOTS option
• Extend IterateLookup and related APIs with cursor, maxAddress, resetCursor, and returnTombstoned parameters

Reviewed Changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/Garnet.test.cluster/ClusterTestUtils.cs	Added MigrateSlotsIndex helper; updated loop variable types
test/Garnet.test.cluster/ClusterTestContext.cs	Added GenerateKeysWithPrefix and GenerateIncreasingSizeValues
test/Garnet.test.cluster/ClusterMigrateTests.cs	Updated migrate calls to use new collection expressions
libs/storage/Tsavorite/cs/src/core/ClientSession/ClientSession.cs	Extended IterateLookup and ScanCursor signatures
libs/storage/Tsavorite/cs/src/core/Allocator/*.cs	Propagated returnTombstoned through scan iterator hierarchy
libs/server/Storage/Session/Common/ArrayKeyIterationFunctions.cs	Extended client APIs for cursor, maxAddress, and tombstones
libs/server/Servers/GarnetServerOptions.cs	Added ParallelMigrateTasks option
libs/host/*	Added defaults and CLI support for ParallelMigrateTasks
libs/common/HashUtils.cs	Added MurmurHash2x64A(Span<byte>) overload
libs/cluster/Session/RespClusterMigrateCommands.cs	Switched non-importing key logic from continue to break
libs/cluster/Session/MigrateCommand.cs	Replaced working set with Sketch in MIGRATE command parsing
libs/cluster/Server/Migration/*	Major refactor to use Sketch, parallelize slot scans
libs/client/ClientSession/GarnetClientSessionIncremental.cs	Removed default param for InitializeIterationBuffer

Comments suppressed due to low confidence (3)

libs/storage/Tsavorite/cs/src/core/Allocator/BlittableAllocatorImpl.cs:279

• Parameter name returneTombstoned has a typo; it should be returnTombstoned to match the rest of the API.

        internal override bool ScanCursor<TScanFunctions>(TsavoriteKV<TKey, TValue, TStoreFunctions, BlittableAllocator<TKey, TValue, TStoreFunctions>> store,

libs/client/ClientSession/GarnetClientSessionIncremental.cs:48

• Removing the = default default parameter breaks existing callers; consider restoring a default value or updating all call sites.

        public void InitializeIterationBuffer(TimeSpan iterationProgressFreq)

test/Garnet.test.cluster/ClusterMigrateTests.cs:602

• The literal [slot] does not create a List<int> and will not match the List<int> parameter; consider using new List<int> { slot } or adjusting the method to accept an array.

            context.clusterTestUtils.MigrateSlots(sourcePort, targetPort, [slot], logger: context.logger);