Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go all-in on HTTP API #53

Merged
merged 3 commits into from Dec 7, 2022
Merged

Go all-in on HTTP API #53

merged 3 commits into from Dec 7, 2022

Conversation

lukechampine
Copy link
Member

worker and bus are proper microservices now: the only way to access their functionality is with a Client. This spares us from reimplementing every method twice. This does introduce some overhead for local setups, but we'll wait until we've measured it before making any decisions around refactoring.

@peterjan
Copy link
Member

peterjan commented Dec 7, 2022

looks good to me but it's only part of the puzzle right? in a single-node setup this code won't create a bus/worker client to pass into the autopilot

worker/worker.go Outdated Show resolved Hide resolved
}
return s, nil
return s, length, nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason you are returning the length here? Seems like we're never using it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's used in objectsKeyHandlerPUT.
It's not strictly necessary -- we could wrap the io.Reader in a counter to determine the total length of the object -- but this is easier.

@@ -190,3 +190,11 @@ type MigrationContract struct {
HostIP string `json:"hostIP"`
ID types.FileContractID `json:"id"`
}

// UploadParams contains the metadata needed by a worker to upload an object.
type UploadParams struct {
Copy link
Member

@peterjan peterjan Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slightly different than what I had in mind. I figured the worker would be using contract sets? I also figured the redundancy would be optionally provided (defaults to 10-30) by the user on the upload endpoint.

Seeing as you fetch it from the bus I assume you have in mind to add it to the bus config? So if I want to upload an object with a different redundancy I'd have to manually use the slabs endpoint or change my bus config?

Copy link
Member

@peterjan peterjan Dec 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The redundancy is also part of the autopilot config, so the migrate endpoints at least should expose some way of specifying the required redundancy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, we can allow overriding the redundancy settings. I'll wait to add that until it's needed, though.

The bus should store the default redundancy. If the autopilot is currently storing the redundancy settings, we should change it to fetch the settings from the bus.

bus/api.go Outdated Show resolved Hide resolved

// UploadParams contains the metadata needed by a worker to upload an object.
type UploadParams struct {
CurrentHeight uint64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it a bit awkward to have consensus logic in the upload endpoint? What happens if the height changes during the upload?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current height is required in order to calculate how much time is left in the contract, and thus how many blocks of storage we're paying for. If the height changes mid-upload, we might slightly overpay or underpay; IIRC the renter has some leeway to account for this.

I have always been annoyed that we have to know the height in order to upload. Would be cool if we could remove it in RHPv4 somehow.

@lukechampine lukechampine merged commit ab0488d into master Dec 7, 2022
@lukechampine lukechampine deleted the worker-api branch December 7, 2022 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants