Scan converts string cursors to a JavaScript number

[fabiancook]

Description

It appears redis can return a cursor larger than the max safe integer of JavaScript.

I was getting a cursor returned 220792619452327820, and when it was converted to a number, it was converted to 220792619452327800

node-redis/packages/client/lib/commands/SCAN.ts

Line 31 in a7d5bc7

cursor: Number(cursor),

When I was reproducing this with ioredis, I first followed the same number type and converted the string to a number, but then realised thats where the problem was happening.

const {
    REDIS_URL
} = process.env;

const options = {
    MATCH: "product::*",
    COUNT: 10
}

{
    const { createClient } = await import("redis");
    const client = createClient({
        url: REDIS_URL,
    });
    client.on("error", console.warn);

    await client.connect();

    {

        let cursor = 0;

        const keys = [],
            cursors = [];
        do {
            cursors.push(cursor);
            const reply = await client.scan(cursor, options);
            cursor = reply.cursor;
            keys.push(...reply.keys);
        } while (cursor !== 0)

        console.log({ keys, cursors, length: keys.length });

    }

    {
        const keys = await client.keys(options.MATCH);

        console.log({ keys, length: keys.length });
    }

}




{
    const { Redis } = await import("ioredis");
    const client = new Redis(REDIS_URL);

    {
        let cursor = "0";

        const keys = [],
            cursors = [];
        do {
            cursors.push(cursor);
            const [cursorString, reply] = await client.scan(cursor, "MATCH", options.MATCH, "COUNT", options.COUNT);
            cursor = cursorString;
            keys.push(...reply);
        } while (cursor !== "0")

        console.log({ keys, cursors, length: keys.length });
    }

    {
        const keys = await client.keys(options.MATCH);

        console.log({ keys, length: keys.length });
    }

}


process.exit(1)

Node.js Version

v18.16.0

Redis Server Version

Server upstash_version:1.7.8 redis_version:6.2.6 redis_git_sha1:4004b61 redis_build_id:20230612 redis_mode:standalone

Node Redis Version

redis@4.6.7

Platform

macOS

Logs

Relevant cursor values:

 cursors: [
    '0',
    '220792619452327820',
    '15453497142749838673',
    '6057887309112734661',
    '174621329877753584',
    '10884524981021954571',
    '7370746790775601805',
    '4348529041547164572',
    '11458793538303765667',
    '15775831359424251828',
    '16508031302366686876',
    '12470010913521533369'
  ]

[leibale]

Will be fixed in v5, see here

[fabiancook]

Cheers! Sounds like you can close this! Thanks for the quick direction!

[leibale]

I'll leave it open for now...

[leibale]

@fabiancook BTW, you can work around that by using .sendCommand directly:

const [cursor, keys] = await client.sendCommand(['SCAN', cursor, '...']);
// cursor = string, keys = Array<string>
const { cursor, keys } = await client.scan(cursor, options);
// cursor = number, keys = Array<string>