Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disallow legacy hostNetwork together with non-default provider #13693

Merged
merged 2 commits into from Feb 13, 2024

Conversation

obnoxxx
Copy link
Contributor

@obnoxxx obnoxxx commented Feb 5, 2024

Description of changes:

Since the introduction of the "host" network provider, the legacy
"hostNetwork" setting is intended to be used only in combination with
the default network provider (""), but originally, the code did not enforce this.

This change adds the required validation checks to throw errors
in invalid constellations.

Issue resolved by this Pull Request:
Resolves #13692

Checklist:

  • Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
  • Reviewed the developer guide on Submitting a Pull Request
  • Pending release notes updated with breaking and/or notable changes for the next minor release.
  • Documentation has been updated, if necessary.
  • Unit tests have been added, if necessary.
  • Integration tests have been added, if necessary.

Copy link

mergify bot commented Feb 5, 2024

⚠️ The sha of the head commit of this PR conflicts with #13651. Mergify cannot evaluate rules on this PR. ⚠️

@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 2 times, most recently from f0ef161 to 1513efe Compare February 5, 2024 20:03
@obnoxxx obnoxxx changed the title Disallow legacy and provider Disallow legacy hostNetwork together with non-default provider Feb 5, 2024
@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 2 times, most recently from a12fcfb to b3e0b74 Compare February 5, 2024 20:31
@obnoxxx obnoxxx added the bug label Feb 5, 2024
@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 3 times, most recently from d9aef18 to 210e2a1 Compare February 6, 2024 12:22
@BlaineEXE
Copy link
Member

This looks good. Let's also add a validating admission policy to reflect this in types.go. That will probably happen in between two of the lines here. And there is an example of how to use the VAPs with // +kubebuilder:validation:XValidation.

With that set, it should mean that k8s won't let you set hostNetwork: true at the same time as provider != "" in the CRD when creating/updating it.

@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 2 times, most recently from 64d94eb to a02473d Compare February 6, 2024 18:32
@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 6 times, most recently from 02e5703 to 127d5dd Compare February 6, 2024 21:05
@obnoxxx
Copy link
Contributor Author

obnoxxx commented Feb 7, 2024

@BlaineEXE wrote:

This looks good. Let's also add a validating admission policy to reflect this in types.go. That will probably happen in between two of the lines here. And there is an example of how to use the VAPs with // +kubebuilder:validation:XValidation.

With that set, it should mean that k8s won't let you set hostNetwork: true at the same time as provider != "" in the CRD when creating/updating it.

Thanks for this hint! I found the place in the code.
It looks a bit as if one should essentially repeat the logic of the Validate function here.

I will soon update the PR accordingly

@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch from 729d9b3 to 75b15fc Compare February 7, 2024 17:13
@obnoxxx
Copy link
Contributor Author

obnoxxx commented Feb 7, 2024

@BlaineEXE wrote:

This looks good. Let's also add a validating admission policy to reflect this in types.go. That will probably happen in between two of the lines here. And there is an example of how to use the VAPs with // +kubebuilder:validation:XValidation.

With that set, it should mean that k8s won't let you set hostNetwork: true at the same time as provider != "" in the CRD when creating/updating it.

I think I have now added what you suggested. And added the updated crds resulting from that.

I guess now it "just" need some testing ...

@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch from 75b15fc to 1f643d1 Compare February 7, 2024 19:01
@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 2 times, most recently from ff38153 to ff2e341 Compare February 9, 2024 18:14
@@ -2316,6 +2316,7 @@ type SSSDSidecarAdditionalFile struct {

// NetworkSpec for Ceph includes backward compatibility code
// +kubebuilder:validation:XValidation:message="at least one network selector must be specified when using multus",rule="!has(self.provider) || (self.provider != 'multus' || (self.provider == 'multus' && size(self.selectors) > 0))"
// +kubebuilder:validation:XValidation:message="the legacy hostNetwork setting can only be set if the network.provider is set to the empty string",rule="(self.hostNetwork != true) || (self.provider == '')"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to look up CEL documentation to learn that we do need has() statements when JSON omitempty is used.

The expression here is failing because it does not allow the empty-omitted case to succeed. The expression must be modified to allow !has(self.hostNetwork) to be interpreted the same as self.hostNetwork == false.

Ditto for !has(self.provider) being interpreted the same as self.provider == "".

Please also use self.hostNetwork == false instead of self.hostNetwork != true. == is always easier to read than !=.

The parens surrounding the statements between || are also unnecessary and don't improve legibility. Let's remove those before final merge.

Copy link
Member

@BlaineEXE BlaineEXE Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I certainly find that sometimes the required logic needed to explain to VAPs when to pass validation can be hard to grok. I imagine this may be the case for you as well.

I have been finding it easier to mentally write out the failure case and then use DeMorgan's Laws to get the inverse expression. I learned this in school, but I'm not sure how common it is in other curriculums.

For example, the failure case is has(self.hostNetwork) && self.hostNetwork == true && has(self.provider) && self.provider != "". The inverse of this is !(<the whole previous expression>), and then DeMorgan's Laws allow simplifying the expression into || statements.

FWIW, this is also often how I go about reading/interpreting the existing validation statements. I find it easier to mentally grasp the failure cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BlaineEXE thanks fore the detailed explanation above! I agree with everything you wrote. fwiw, application of Boolean algebra and de Morgan's law is how I originally arrived at the rule I initially had in the code (involving || instead of &&.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After these latest updates, the CI looks a lot better. One failing check looks unrelated.

Locally everything builds and deploys fine. got cluster-test.yaml to deploy to status HEALTH_OK ! 🎉

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inverse of the "failure case" expression I mentioned would be this:

!has(self.hostNetwork) || self.hostNetwork == false || !has(self.provider) || self.provider == ""

Note that I am using == false as the inverse of == true and == "" as the inverse of != "" to reduce the statement's layers to their simplest boolean algebra forms.

deploy/charts/rook-ceph/templates/resources.yaml Outdated Show resolved Hide resolved
deploy/examples/crds.yaml Outdated Show resolved Hide resolved
pkg/apis/ceph.rook.io/v1/types.go Outdated Show resolved Hide resolved
@obnoxxx obnoxxx force-pushed the disallow-legacy-and-provider branch 2 times, most recently from 4166dfb to c7689c6 Compare February 12, 2024 11:53
@@ -2316,6 +2316,7 @@ type SSSDSidecarAdditionalFile struct {

// NetworkSpec for Ceph includes backward compatibility code
// +kubebuilder:validation:XValidation:message="at least one network selector must be specified when using multus",rule="!has(self.provider) || (self.provider != 'multus' || (self.provider == 'multus' && size(self.selectors) > 0))"
// +kubebuilder:validation:XValidation:message="the legacy hostNetwork setting can only be set if the network.provider is set to the empty string",rule="!has(self.hostNetwork) || !has(self.provider)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rule="!has(self.hostNetwork) || !has(self.provider)
-> rule="has(self.hostNetwork) && !has(self.provider)" might be this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parth-gr wrote:

rule="!has(self.hostNetwork) || !has(self.provider) -> rule="has(self.hostNetwork) && !has(self.provider)" might be this

Actually no: We want to prevent hostNetwork from being set together with a non-empty provider, so the rule has to be this:

!(has(self.hostNetwork) && has(self.provider))

which Boolean algebra simplifies to:

!has(self.hostNetwork) || !has(self.provider)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is actually even more complex since we do need to check the values in addition to using has() I'll update shortly.

Signed-off-by: Michael Adam <obnox@samba.org>
Copy link
Member

@BlaineEXE BlaineEXE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but please squash the commits into one. It's fine to leave the first (ci: add "network" to the allowed commit message prefixes) commit separate if you like.

@obnoxxx
Copy link
Contributor Author

obnoxxx commented Feb 12, 2024

@BlaineEXE wrote:

Looks good, but please squash the commits into one. It's fine to leave the first (ci: add "network" to the allowed commit message prefixes) commit separate if you like.

Thanks!

I'll do the squashing tomorrow

Fixes: rook#13692

Since the introduction of the "host" network provider, the  legacy
"hostNetwork" setting is intended to be used only in combination with
the default network provider (""), but the code did not enforce this.

This change adds the required validation checks to throw errors
in invalid constellations.

These checks are added both in the operator's input  validation code
and as kubernetes x-validation admission policies in the Cepcluster CRD.

Signed-off-by: Michael Adam <obnox@samba.org>
@obnoxxx
Copy link
Contributor Author

obnoxxx commented Feb 12, 2024

@BlaineEXE wrote:

Looks good, but please squash the commits into one. It's fine to leave the first (ci: add "network" to the allowed commit message prefixes) commit separate if you like.

Thanks!

I'll do the squashing tomorrow

I actually already did it now 😬

@travisn travisn merged commit 1a3d242 into rook:master Feb 13, 2024
50 of 51 checks passed
travisn added a commit that referenced this pull request Feb 13, 2024
Disallow legacy hostNetwork  together with  non-default provider (backport #13693)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

disallow legacy hostNetwork setting with non-default network provider
6 participants