Skip to content

Make size info available during volume creation without requiring activation#557

Merged
gjcolombo merged 16 commits intomainfrom
gjcolombo/client-supplied-region-info
Dec 12, 2022
Merged

Make size info available during volume creation without requiring activation#557
gjcolombo merged 16 commits intomainfrom
gjcolombo/client-supplied-region-info

Conversation

@gjcolombo
Copy link
Contributor

@gjcolombo gjcolombo commented Dec 7, 2022

Allow a Volume to have a valid block size and total size from its creation by requiring users of VolumeConstructionRequest::Region to pass extent sizes and counts in addition to block sizes. This allows callers to create a Volume and query its sizes before anyone has activated, which in turn allows Propolis to defer volume activation until late in migration, solving the problems outlined in #440 and #543. Callers who want to create an upstairs without a VolumeConstructionRequest (e.g. tools like crudd and hammer) can still do so (they will get the old "accept information from the first downstairs" behavior).

NOTE: Because this changes VolumeConstructionRequest's definition, this PR is a breaking change for Propolis/Omicron. I have a draft of the Propolis changes needed to adopt the new struct format, but still need to do the corresponding Omicron changes. I'm happy to wait to merge until the Omicron bits are also ready--just let me know (I wanted to go ahead and start getting this reviewed regardless).

Open issues to review:

  • I'm unsure about how volume/region UUIDs are supposed to work. I left a REVIEW(gjc) comment in upstairs/src/lib.rs in a spot where this was important (it affects what sort of checks we can do when processing incoming region definitions).

Tests:

  • cargo test
  • PHD tests, including migration-under-load test with Propolis fixes in place that shows that this approach works to allow Crucible-backed VMs to migrate

Fixes #440. Fixes #543.

@gjcolombo gjcolombo requested a review from leftwo December 7, 2022 01:00
Copy link
Contributor

@leftwo leftwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've done some good stuff here, and in a much needed area.

I'm trying to break down where the line should be between CrucibleOpts and RegionExtentInfo as well as getting enough working here that you can go back to migration.

However, after typing and re-typing various ideas, I think what you have here is
probably the right place to draw the line between what you need to move forward right
now that won't require major overhaul of Crucible inputs, and also won't be
creating work we would undo later.

"Block size not available (active: {})",
up.guest_io_ready().await
);
req.send_err(CrucibleError::UpstairsInactive).await;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's return some different error here. Maybe just generic error. It's probably UpstairsInactive, but unless we check it, we don't know for sure.

Same for the two other places where we return error below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7012fb9.

pub async fn up_main(
opt: CrucibleOpts,
gen: u64,
expected_region_def: Option<RegionDefinition>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit: We could probably just call this region_def.
Yes, it is expected, but so is the gen above, and we don't call that expected_gen.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in dcc367f.

@gjcolombo
Copy link
Contributor Author

I've finally managed to test this with the corresponding Propolis/Omicron changes and things look good so far (Propolis tests pass, Omicron instance starts up). I've pushed the drafts to Github: Propolis, Omicron. These are (IMO) really straightforward changes that suggest that we're taking the right approach here.

I'll start incorporating feedback here later today.

@leftwo leftwo self-requested a review December 10, 2022 05:07
Copy link
Contributor

@leftwo leftwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge away. Thanks for you work here.

@gjcolombo gjcolombo merged commit e3a29af into main Dec 12, 2022
@gjcolombo gjcolombo deleted the gjcolombo/client-supplied-region-info branch December 12, 2022 17:54
@gjcolombo
Copy link
Contributor Author

Thanks for reviewing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Volume create wants a regions size before the Upstairs knows what the size is. Activation and volume construction

2 participants