Repro stashed upload blob issue when processing reuploaded blob#22981
Merged
Conversation
github-actions
Bot
added
area: runtime
msfluid-bot
reviewed
Nov 5, 2024
Collaborator
There was a problem hiding this comment.
Code Coverage Summary
↑ packages.runtime.container-runtime.src.blobManager:
Line Coverage Change: 0.02% Branch Coverage Change: 0.14%
| Metric Name | Baseline coverage | PR coverage | Coverage Diff |
|---|---|---|---|
| Branch Coverage | 96.66% | 96.80% | ↑ 0.14% |
| Line Coverage | 99.14% | 99.16% | ↑ 0.02% |
Baseline commit: 3c27751
Baseline build: 305767
Happy Coding!!
Code coverage comparison check passed!!
Collaborator
⯅ @fluid-example/bundle-size-tests: +2.22 KB
Baseline commit: 3c27751 |
Contributor
There was a problem hiding this comment.
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 suggestions.
Comments skipped due to low confidence (1)
packages/runtime/container-runtime/src/test/blobManager.stashed.spec.ts:159
- The line contains a syntax error. The 'await' keyword should not be used with 'let'. It should be 'await letUploadsComplete;'.
await letUploadsComplete.promise;
Tip: Copilot code review supports C#, Go, Java, JavaScript, Markdown, Python, Ruby and TypeScript, with more languages coming soon. Learn more
| private readonly localBlobIdGenerator: () => string; | ||
| private readonly pendingStashedBlobs: Map<string, Promise<ICreateBlobResponse | void>> = | ||
| new Map(); | ||
| private stashedBlobsUploadP: Promise<(void | ICreateBlobResponse)[]> | undefined = undefined; |
Contributor
There was a problem hiding this comment.
nit. this could be a lazy promise, which would make the method simpler, and allow this to be immutable (readonly)
Contributor
There was a problem hiding this comment.
you could also embed the clear into the lazy promise as well
Contributor
Author
There was a problem hiding this comment.
I'll take this nit into consideration on a follow up PR
anthony-murphy
approved these changes
Nov 8, 2024
dannimad
added a commit
that referenced
this pull request
Nov 15, 2024
This change fixes issue covered in PR #22981 After several simplifications, I realized we could reuse pendingStashedBlobs map to identify the bug scenario, which is reuploading a stashed blob that was already processed. If while onUploadResolve we don't see the blob in the pending list but it's on the stashed one, then just ignore the new upload. trackPendingStashedUploads is replaced with waitForStashedBlobs which also implies there are further simplifications to the blob manager logic, mainly we could also remove the stashedUpload property on the PendingBlob type since now we keep track of stashed blobs separately .
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a test repro of AB#4630.
To reproduce such issue, I had to fake the localId generation using uuidOverride constructor method in the blob manager so we could aim at a unique pending blob: such pending blob was: 1 stashed, 2 reapplied to a different blob manager simulating rehydration, 3 processed and 4 reuploaded by stashing workflow, which ends up in the issue described in the ADO item.
In order to wait for the reupload I had to create a different blob manager method waitForStashedBlobs() since the one we have hasPendingStashedUploads() uses pendingBlobs array as reference, which blob entry would be removed by the processing step.
A fix will follow up this PR.