Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Colossus] Serve objects from pending folder #5060

Open
mnaamani opened this issue Jan 24, 2024 · 2 comments
Open

[Colossus] Serve objects from pending folder #5060

mnaamani opened this issue Jan 24, 2024 · 2 comments
Labels

Comments

@mnaamani
Copy link
Contributor

Currently on request for an object that is still being processed in the pending folder, we return a 404.
We were concerned with what would happen if file was being streamed to the response, while the AcceptPendingObjects service tries to move/copy the file from pending folder to the uploads folder.

If we find that it is safe and we are okay with the resulting behaviour, and the potential latency between uploading a new object and being able to fetch it from the same node is not acceptable, then implement this change.

@kdembler
Copy link
Member

kdembler commented Feb 2, 2024

If an operator runs out of balance, they could build up a stack of uploaded objects that don't get accepted. If a node like that disappears for some reason, currently all those objects would get lost since other operators won't sync until object is moved to main uploads/.

@kdembler
Copy link
Member

kdembler commented Feb 2, 2024

This is possibly blocked by the fact that storage nodes will currently not synch objects that haven't been accepted on-chain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

No branches or pull requests

2 participants