Running two different queries concurrently with Promise.all returns the same results

[tomanagle]

We upgraded from ^6.0.4 to ^7.1.2 and noticed some really weird results when running queries concurrently with Promise.all.

The queries were returning the same results, sometimes from query A and sometimes from query B. We would get different execution ids and use those to make the query, but the results were the same.

Let me know if you need a code sample to explain it better.

[ghdna]

yes, please provide sample code. And are you saying the issue didn't exist on previous version

[piyushk96]

Problem for me too. I'm getting same results for multiple SELECT queries that are running concurrently. @ghdna just put 10-15 unique select queries inside Promise.all and you will get the same results for all the queries.

[tomanagle]

Yeah, we rolled back to ^6.0.4 and it works perfectly now.

As for testing it, no need for 15 select queries, we just had 2.

[ghdna]

That's good context for me to check what's going on between the 2 versions. Thanks. Will look into this.

[piyushk96]

Just to mention Somentimes I'm also getting following error when running queries concurrently

{
    "type": "Error",
    "message": "Query has not yet finished. Current state: QUEUED",
    "stack":
        InvalidRequestException: Query has not yet finished. Current state: QUEUED
            at Request.extractError (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/protocol/json.js:52:27)
            at Request.callListeners (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
            at Request.emit (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
            at Request.emit (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:688:14)
            at Request.transition (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:22:10)
            at AcceptorStateMachine.runTo (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/state_machine.js:14:12)
            at /home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/state_machine.js:26:10
            at Request.<anonymous> (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:38:9)
            at Request.<anonymous> (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:690:12)
            at Request.callListeners (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
            at Request.emit (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
            at Request.emit (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:688:14)
            at Request.transition (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:22:10)
            at AcceptorStateMachine.runTo (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/state_machine.js:14:12)
            at /home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/state_machine.js:26:10
            at Request.<anonymous> (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:38:9)
            at Request.<anonymous> (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/request.js:690:12)
            at Request.callListeners (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
            at callNextListener (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
            at IncomingMessage.onEnd (/home/ec2-user/sapphire-pm2/source/node_modules/aws-sdk/lib/event_listeners.js:313:13)
            at IncomingMessage.emit (events.js:327:22)
            at IncomingMessage.EventEmitter.emit (domain.js:467:12)
            at endReadableNT (internal/streams/readable.js:1327:12)
            at processTicksAndRejections (internal/process/task_queues.js:80:21)
    "code": "InvalidRequestException",
    "time": "2022-01-11T13:27:54.215Z",
    "requestId": "9acfd970-bc05-42d8-8e1b-4b318e1ee6ce",
    "statusCode": 400,
    "retryable": false,
    "retryDelay": 66.02900664883433
}

According to my understanding since all concurrent queries are returning same QueryExecutionId when one of them gets finished other may be in RUNNING or QUEUED state and when it tries to get results for queued or running query it throws this unhandled promise rejection

[piyushk96]

@ghdna I have opened a PR for the fix. Please check.

[IagorSs]

Same problem here, please accept this PR 😭

[filipemarruda]

Same here! +1

[ghdna]

Merged. Thanks for sharing the issue and submitting the PR.