New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add via queueing pin op on pinMgr for handleAdd and handleAddCar #886

Merged

en0ma merged 8 commits into application-research:dev from neelvirdy:nvirdy/add-with-pinmgr

Jan 24, 2023

Contributor

neelvirdy commented Jan 19, 2023 •

edited

Loading

Have handleAdd and handleAddCar leverage the existing pin op codepaths for pinning contents, instead of maintaining a separate pinning codepath for those handlers
Create contents in "queued" state instead of "pinning". Have pinmgr set the status to pinning once a worker is on it
Use onPinStatusUpdate for all types of pin status updates, not just failures

neelvirdy commented

View reviewed changes

api/v1/handlers.go Outdated Show resolved Hide resolved

neelvirdy force-pushed the nvirdy/add-with-pinmgr branch from 0987dbe to fd6c0d8 Compare

January 19, 2023 19:55

neelvirdy requested review from en0ma, anjor and alvin-reyes

January 19, 2023 19:56

en0ma reviewed

View reviewed changes

api/v1/handlers.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

api/v1/handlers.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

cmd/estuary-shuttle/main.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

cmd/estuary-shuttle/main.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

cmd/estuary-shuttle/main.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

cmd/estuary-shuttle/main.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

api/v1/handlers.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

api/v1/handlers.go Show resolved Hide resolved

en0ma reviewed

View reviewed changes

content/pinning.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

content/queue/queue.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

content/pinning.go Show resolved Hide resolved

en0ma reviewed

View reviewed changes

shuttle/rpc/rpc.go Outdated Show resolved Hide resolved

en0ma reviewed

View reviewed changes

shuttle/rpc/rpc.go Show resolved Hide resolved

neelvirdy requested a review from en0ma

January 23, 2023 20:01

neelvirdy and others added 8 commits

January 23, 2023 15:03


          Use pinmgr for apiv1 handleAdd and handleAddCar

564545a


          handleAdd by queueing pin op

f2d2d79


          Re-org imports

61361f6


          docs: update swagger docs

a90b9aa


          Err on pinop fail, re-add collections bit, remove getOrigins

5a2a736


          Re-add replication param, comment on block objects

d7a56dd


          Dump blocks before create content on shuttle, return errs without log…

c124cbd

…ging


          Re-add size check, fix zone stuck updates, remove offloaded update

c986746

neelvirdy force-pushed the nvirdy/add-with-pinmgr branch from d49539d to c986746 Compare

January 23, 2023 20:04

en0ma approved these changes

View reviewed changes

Contributor

en0ma left a comment

LGTM

en0ma merged commit 74d255a into application-research:dev

neelvirdy deleted the nvirdy/add-with-pinmgr branch

January 31, 2023 15:19

neelvirdy mentioned this pull request

Render depending on pinning status in upload view application-research/estuary-www#174

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment