Checking consistent block size on existing pools

[mulkieran]

We enforce a rule on consistent block sizes somewhat stringently. Unfortunately, there is a case where our choice of how we compare causes a problem. When a pool is encrypted, we compare the block sizes of the opened devices to the block sizes of the devices that have been presented for adding to a pool. However, when we encrypt, cryptsetup may change the logical block size of the device, generally to increase it to match the physical block size. So, even though the devices in the pool and the devices being presented to be added have exactly the same block sizes, stratisd will reject the devices.

Note that this problem will disappear if crypt device is moved away from the block devices as we intend w/ our design. So, this is sort of a temporary problem, although it still has to be fixed.

The modified approach can be the following:

For the devices to be added, we only know what their block sizes are now, not what their block sizes will be after cryptsetup is applied to them. So, when pool is encrypted, we must read the block sizes on the opened devices and determine if the encryption would be able to make the devices of compatible size. Likely the rule should be that we can expect that encryption could increase the logical block size to match the physical block size of the devices being added, if necessary to make a match. So, we allow block devices that would allow that, but then we enforce that cryptsetup makes the logical block size to match, and if, for some reason it fails, we do a rollback using existing functionality.

Related bz: https://bugzilla.redhat.com/show_bug.cgi?id=2170318

[mulkieran]

So, the first step is to read the block sizes from the base devices, and not from the opened cryptsetup devices. This allows us to be consistent about the block sizes we report for the devices on creation and after stratisd is restarted.

[mulkieran]

However, we need to store in the StratisBlockdev struct not just the block sizes of the base device, but also the block sizes of the opened crypt device if any.

StratBlockSizes {
   physical: BlockSizes,
   luks: Option<BlockSizes>,
}

when adding devices, we must use the sector_size which has been used in the existing crypt devices to enforce the compatible logical block size for the crypt device.