New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
User/Track update ipfs bug fix #1029
Conversation
…ws, add ipfs + disk validation during image upload
…to get stronger guarantees
…ocol into ss-fix-update-route-image-checks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some questions but the one I have some follow up q's on is what do we do in validateStateForImageDirCIDAndReturnFileUUID in the case of a non dir image? it seems like that flow is broken currently as well cause that would error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's go, this looks great
Trello Card Link
https://trello.com/c/wQ26N0RE
Description
Replace ipfs validation with disk validation in user/track update flows, add ipfs + disk validation during image upload
Problem: currently, anytime someone updates user or artist data, CN performs validation on associated images. the function that performed this validation,
getFileUUIDForImageCID()
would callipfs.cat()
andipfs.ls()
on the image dir CIDs with no timeouts or try-catch. this is bad for many reasons, and likely the primary cause for repeated/audius_users
calls timing out at 10minSolution: On a state update, we should do minimal validation, and move the heavy-weight validation to first time upload to guarantee correctness before returning to user and writing CID to chain. image upload now performs a full byte-level validation of the files written to disk. user updates now perform a much lighter disk/db check.
Services
Does it touch a critical flow like Discovery indexing, Creator Node track upload, Creator Node gateway, or Creator Node file system?
Delete an option.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide repro instructions & any configuration.
Include log analysis if applicable.
/audius_users
and/tracks
, pullensurePrimaryMiddleware
duration from logs and subtract. on staging i'm repeatedly observing ~0.25s which is fantasticPlease list the unit test(s) you added to verify your changes.