PynamoDB Inconsistent Read Results in High-Concurrency Queue Processing

[baryaner]

PynamoDB's get() method occasionally returns stale/incorrect data when multiple queue messages are processed simultaneously, leading to data consistency issues.

Context: Worker processes consuming from a queue, with multiple messages published within the same second
Problem:
The DynamoDB query
CommitTask.get(hash_key=event.commit_id, range_key=event.task_id)
sometimes returns a record where commit_task.task_id doesn't match the expected event.task_id

Inconsistency: Retrying the same get() operation immediately after returns the correct/expected result

Trigger: Issue occurs specifically when queue payloads are published simultaneously (within the same second)

i try to use consistent_read = True but i don't think it's the case here.
any idea where is the problem?

[ikonst]

1. There's very little this library does in-between getting the response from AWS and returning it to you.

2. I don't think "consistent reads" are something that you ever need with regard to HK and RK, since those are literally the keys you use to look up the item.

In short, I have no idea where the problem may be. If there's any complexity in your code between CommitTask.get and where you're checking those values, I'd look there.