S3 provides only eventual-consistency for list() operation after writes. This, according to my experience, might result in 20-30 seconds of delay for discovering the results of a finished task. One solution is to use get to fetch results after, say, 90% of tasks are finished.
I have a patch to fix this issue. Will make a PR.
The text was updated successfully, but these errors were encountered:
According to this post:
https://forums.aws.amazon.com/thread.jspa?messageID=687028#687028
S3 provides only eventual-consistency for list() operation after writes. This, according to my experience, might result in 20-30 seconds of delay for discovering the results of a finished task. One solution is to use
get
to fetch results after, say, 90% of tasks are finished.I have a patch to fix this issue. Will make a PR.
The text was updated successfully, but these errors were encountered: