-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
network: improve the IsHost() function #14253
Conversation
One quick thing: |
go.sum
Outdated
github.com/portworx/sched-ops v0.20.4-openstorage-rc3 h1:tXnHsjZT2wZ2BCXf8avDoya7zGyCgLNUC8Upt+WEQrY= | ||
github.com/portworx/sched-ops v0.20.4-openstorage-rc3 h1:46EZ+vYCJ3qmQolvgDCrGuPz8Tf0vIds41RuF0dqVEw= |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this changing? I don't see anything that appears related in the code changes.
This also seems to be the reason why CI is failing. Here's what I see when I look at the CI output:
...
verifying github.com/portworx/sched-ops@v0.20.4-openstorage-rc3: checksum mismatch
downloaded: h1:tXnHsjZT2wZ2BCXf8avDoya7zGyCgLNUC8Upt+WEQrY=
go.sum: h1:46EZ+vYCJ3qmQolvgDCrGuPz8Tf0vIds41RuF0dqVEw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@BlaineEXE wrote:
Why is this changing? I don't see anything that appears related in the code changes.
To be honest, I am not quite certain. I was puzzled myself when I saw go.sum in the list of changed files. 🤷🏼
Here is my guess:
Earlier today, I wanted to build and unit-test locally and this was failing on go mod errors. So I tidied the dependencies and built/tested again, successfully. I assume that this has introduced the change to go.sum.
This also seems to be the reason why CI is failing.
Right, this seems plausible. Let me try and get rid of the go.sum change in the commit
Here's what I see when I look at the CI output:
... verifying github.com/portworx/sched-ops@v0.20.4-openstorage-rc3: checksum mismatch downloaded: h1:tXnHsjZT2wZ2BCXf8avDoya7zGyCgLNUC8Upt+WEQrY= go.sum: h1:46EZ+vYCJ3qmQolvgDCrGuPz8Tf0vIds41RuF0dqVEw= SECURITY ERROR This download does NOT match an earlier download recorded in go.sum. The bits may have been replaced on the origin server, or an attacker may have intercepted the download attempt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interestingly, when I locally revert the change to go.sum
and run make clean
and go mod tidy
, I get the same error:
verifying github.com/portworx/sched-ops@v0.20.4-openstorage-rc3: checksum mismatch
downloaded: h1:46EZ+vYCJ3qmQolvgDCrGuPz8Tf0vIds41RuF0dqVEw=
go.sum: h1:tXnHsjZT2wZ2BCXf8avDoya7zGyCgLNUC8Upt+WEQrY=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
so I am still unsure how to properly fix this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have undone the change to go.sum
and mod-check
passes in the ci now, but locally, make mod.check
still fails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@obnoxxx Can you revert this file? It seems to be causing build issues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done. no changes to go.sum
in the PR anymore, even though I can't build locally without them.
pkg/apis/ceph.rook.io/v1/network.go
Outdated
if !spec.IsMultus() && !spec.IsHost() { | ||
if !spec.IsMultus() && spec.Provider != NetworkProviderHost { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand why this change is needed here.
I think that this condition no longer works correctly if the user has HostNetwork: true
, because the legacy behavior is no longer also supported.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also note that the signature of IsHost()
has changed to include an additional Error return
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this has gone now.
@@ -670,7 +670,8 @@ func (c *Cluster) failoverMon(name string) error { | |||
return errors.Wrap(err, "failed to place new mon on a node") | |||
} | |||
|
|||
if c.spec.Network.IsHost() { | |||
isHost, _ := c.spec.Network.IsHost() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldn't make it a habit to ignore error return values.
Ditto for other similar changes as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@BlaineEXE wrote:
We shouldn't make it a habit to ignore error return values. Ditto for other similar changes as well.
ok, good point. Agreed. I will address this, but it might make the code flow more complicated and less natural.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The complication is that IsHost()
returns an error, but I don't think it should. The flow will be much simpler if IsHost()
only returns a bool. There can be a single check in the reconcile for valid network settings, instead of every time IsHost()
is called. This will also allow you to revert a lot of changes in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@travisn wrote:
The complication is that
IsHost()
returns an error, but I don't think it should.
The decision to have IsHost return error on invalid network spec was a esult of the discussion between @BlaineEXE and myself in the original PR #13878
#13878 (comment)
The flow will be much simpler if
IsHost()
only returns a bool.There can be a single check in the reconcile for valid network settings, instead of every time
IsHost()
is called.
Agreed.
This will also allow you to revert a lot of changes in this PR.
This sounds like it is desirable to have as few changes as possible in one PR.
Of course, fewer changes tend to be simpler and therefore less likely to break things. So I do see the point. 😄
While it does seem kind of sad to me to revert a major piece of the work put into this so far, I will do it as I see the logic behind the reasoning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@travisn starting to undo the changes that add the eroor return to IsHost(), I am wondering if it would make sense to still validate the network spec in IsHost() but just issue an informational log message and return false for host network instead of erroring out. wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since IsHost()
is called from multiple locations, the log entry could easily be too verbose and repetitive. If there is an error condition to check for, I would recommend have a separate method that checks the error once during a reconcile, then IsHost()
can assume there is no error condition and just return bool
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
net := NetworkSpec{HostNetwork: false} | ||
|
||
net.Provider = NetworkProviderHost | ||
ret, err := net.IsHost() | ||
assert.NoError(t, err) | ||
assert.True(t, ret) | ||
|
||
net.Provider = NetworkProviderDefault | ||
net.HostNetwork = true | ||
ret, err = net.IsHost() | ||
assert.NoError(t, err) | ||
assert.True(t, ret) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good unit test hygiene is to make sure each test sets up its own inputs so that future test modifications don't have unintended effects. It is hard to read and modify unit tests when test inputs and side-effects are propagated from between test cases.
Please fully specify net = Network{}
before each test to ensure tests are independent from each other.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done!
@BlaineEXE wrote:
fixed, thanks! |
I pulled the branch and ran
SO I think the error is coming from network API |
pkg/apis/ceph.rook.io/v1/network.go
Outdated
// which is incompatible with other network providers: HostNetwork set to true | ||
// together with an empty or unset network provider has the same effect as | ||
// network.Provider set to "host" | ||
func (n *NetworkSpec) IsHost() (ret bool, err error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see there a lot of bugs with this implementation
func (n *NetworkSpec) IsHost() (bool, error) {
ret := false
err := ValidateNetworkSpec("", *n)
if err != nil {
return ret, errors.Wrapf(err, "Unable to determine host network setting from Invalid network spec. not setting host network.")
}
return (n.HostNetwork && n.Provider == NetworkProviderDefault) || n.Provider == NetworkProviderHost, err
}
Fixing this make the make build
worked
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@parth-gr this has been fixed.
@parth-gr wrote:
right, and that is code from this PR. I see the same error locally now, but the confusing part is that this is a vet error And the error message shows which code is wrong but not what aspect. ... |
wrong it is in fact a gofmt error.
manually running
I am surprised that it complains only about line 31: There are many more occurrences like this. |
The go.sum file has been removed entirely in the commit. The build can't proceed without that |
896de79
to
3576ae3
Compare
@BlaineEXE wrote:
interesting. For me, locally, the build would not proceed or even succeed with the checked-in Now I was able to fix the problem locally and build again. I committed the resulting diff and updated the PR. |
pkg/operator/ceph/file/mds/spec.go
Outdated
HostNetwork: c.clusterSpec.Network.IsHost(), | ||
HostNetwork: useHost, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If IsHost()
is being reverted to only return a bool (even when there are configuration inconsistencies that Rook has to guess about), I don't think it makes sense to keep these changes that add useHost := ...IsHost()
, as it's just code churn with no real changes. These should probably be reverted also.
At that point, the focus of the changes is better unit testing and docs, which is still valuable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done now
f1293d1
to
8b6f486
Compare
@@ -1186,6 +1186,11 @@ func (in *CephFilesystemSubVolumeGroupList) DeepCopyObject() runtime.Object { | |||
func (in *CephFilesystemSubVolumeGroupSpec) DeepCopyInto(out *CephFilesystemSubVolumeGroupSpec) { | |||
*out = *in | |||
in.Pinning.DeepCopyInto(&out.Pinning) | |||
if in.Quota != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes under pkg/apis
actually are from master, not needed for your changes to IsHost()
. I'd suggest just add these generated files in a separate commit in this PR. It seems our CI action is not working to catch that this generated code is out of date.
@@ -670,7 +670,7 @@ func (c *Cluster) failoverMon(name string) error { | |||
return errors.Wrap(err, "failed to place new mon on a node") | |||
} | |||
|
|||
if c.spec.Network.IsHost() { | |||
if c.spec.Network.IsHost() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert this space, note that vscode would automatically remove these extra spaces for you, so you don't have to think about whitespace much in golang.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done (manually 😉 )
pkg/operator/ceph/cluster/mon/mon.go
Outdated
@@ -189,7 +189,7 @@ func (c *Cluster) Start(clusterInfo *cephclient.ClusterInfo, rookImage string, c | |||
c.spec = spec | |||
|
|||
// fail if we were instructed to deploy more than one mon on the same machine with host networking | |||
if c.spec.Network.IsHost() && c.spec.Mon.AllowMultiplePerNode && c.spec.Mon.Count > 1 { | |||
if c.spec.Network.IsHost() && c.spec.Mon.AllowMultiplePerNode && c.spec.Mon.Count > 1 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert this space
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
pkg/operator/ceph/cluster/mon/mon.go
Outdated
@@ -923,7 +923,7 @@ func (c *Cluster) assignMons(mons []*monConfig) error { | |||
// placement is not being made. otherwise, the node choice will map | |||
// directly to a node selector on the monitor pod. | |||
var schedule *controller.MonScheduleInfo | |||
if c.spec.Network.IsHost() || c.monVolumeClaimTemplate(mon) == nil { | |||
if c.spec.Network.IsHost() || c.monVolumeClaimTemplate(mon) == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert this space
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -108,7 +108,7 @@ func (r *ReconcileNode) createOrUpdateCephCron(cephCluster cephv1.CephCluster, c | |||
getCrashPruneContainer(cephCluster, *cephVersion), | |||
}, | |||
RestartPolicy: corev1.RestartPolicyNever, | |||
HostNetwork: cephCluster.Spec.Network.IsHost(), | |||
HostNetwork: cephCluster.Spec.Network.IsHost(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert this space
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
pkg/operator/ceph/file/mds/spec.go
Outdated
@@ -136,7 +135,7 @@ func (c *Cluster) makeMdsDaemonContainer(mdsConfig *mdsConfig, fsName string) v1 | |||
"--foreground", | |||
) | |||
|
|||
if !c.clusterSpec.Network.IsHost() && !c.clusterSpec.Network.IsMultus() { | |||
if !c.clusterSpec.Network.IsHost() && !c.clusterSpec.Network.IsMultus() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert this space
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -707,7 +707,8 @@ func TestMakeRGWPodSpec(t *testing.T) { | |||
if tt.isSet { | |||
assert.Equal(t, *c.store.Spec.Gateway.HostNetwork, podTemplateSpec.Spec.HostNetwork) | |||
} else { | |||
assert.Equal(t, c.clusterSpec.Network.IsHost(), podTemplateSpec.Spec.HostNetwork) | |||
isHost := c.clusterSpec.Network.IsHost() | |||
assert.Equal(t, isHost, podTemplateSpec.Spec.HostNetwork) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can revert this file as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
c425cae
to
e4ba750
Compare
81f99d5
to
7e09856
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
addressed a few review requests and updated.
net := NetworkSpec{HostNetwork: false} | ||
|
||
net.Provider = NetworkProviderHost | ||
ret, err := net.IsHost() | ||
assert.NoError(t, err) | ||
assert.True(t, ret) | ||
|
||
net.Provider = NetworkProviderDefault | ||
net.HostNetwork = true | ||
ret, err = net.IsHost() | ||
assert.NoError(t, err) | ||
assert.True(t, ret) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done!
@@ -670,7 +670,8 @@ func (c *Cluster) failoverMon(name string) error { | |||
return errors.Wrap(err, "failed to place new mon on a node") | |||
} | |||
|
|||
if c.spec.Network.IsHost() { | |||
isHost, _ := c.spec.Network.IsHost() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
pkg/operator/ceph/file/mds/spec.go
Outdated
HostNetwork: c.clusterSpec.Network.IsHost(), | ||
HostNetwork: useHost, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
pkg/apis/ceph.rook.io/v1/network.go
Outdated
if !spec.IsMultus() && !spec.IsHost() { | ||
if !spec.IsMultus() && spec.Provider != NetworkProviderHost { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this has gone now.
pkg/operator/ceph/file/mds/spec.go
Outdated
HostNetwork: c.clusterSpec.Network.IsHost(), | ||
HostNetwork: useHost, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done now
@@ -108,7 +108,7 @@ func (r *ReconcileNode) createOrUpdateCephCron(cephCluster cephv1.CephCluster, c | |||
getCrashPruneContainer(cephCluster, *cephVersion), | |||
}, | |||
RestartPolicy: corev1.RestartPolicyNever, | |||
HostNetwork: cephCluster.Spec.Network.IsHost(), | |||
HostNetwork: cephCluster.Spec.Network.IsHost(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
pkg/operator/ceph/file/mds/spec.go
Outdated
@@ -136,7 +135,7 @@ func (c *Cluster) makeMdsDaemonContainer(mdsConfig *mdsConfig, fsName string) v1 | |||
"--foreground", | |||
) | |||
|
|||
if !c.clusterSpec.Network.IsHost() && !c.clusterSpec.Network.IsMultus() { | |||
if !c.clusterSpec.Network.IsHost() && !c.clusterSpec.Network.IsMultus() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -707,7 +707,8 @@ func TestMakeRGWPodSpec(t *testing.T) { | |||
if tt.isSet { | |||
assert.Equal(t, *c.store.Spec.Gateway.HostNetwork, podTemplateSpec.Spec.HostNetwork) | |||
} else { | |||
assert.Equal(t, c.clusterSpec.Network.IsHost(), podTemplateSpec.Spec.HostNetwork) | |||
isHost := c.clusterSpec.Network.IsHost() | |||
assert.Equal(t, isHost, podTemplateSpec.Spec.HostNetwork) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -670,7 +670,7 @@ func (c *Cluster) failoverMon(name string) error { | |||
return errors.Wrap(err, "failed to place new mon on a node") | |||
} | |||
|
|||
if c.spec.Network.IsHost() { | |||
if c.spec.Network.IsHost() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done (manually 😉 )
pkg/operator/ceph/cluster/mon/mon.go
Outdated
@@ -189,7 +189,7 @@ func (c *Cluster) Start(clusterInfo *cephclient.ClusterInfo, rookImage string, c | |||
c.spec = spec | |||
|
|||
// fail if we were instructed to deploy more than one mon on the same machine with host networking | |||
if c.spec.Network.IsHost() && c.spec.Mon.AllowMultiplePerNode && c.spec.Mon.Count > 1 { | |||
if c.spec.Network.IsHost() && c.spec.Mon.AllowMultiplePerNode && c.spec.Mon.Count > 1 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
6b23ceb
to
f6a3cd0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a couple small whitespace changes that could be reverted if desired
@@ -49,7 +49,6 @@ func ApplyCephNetworkSettings( | |||
clusterInfo *cephclient.ClusterInfo, | |||
) error { | |||
netSpec := clusterSpec.Network | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could revert
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
pkg/operator/ceph/file/mds/spec.go
Outdated
@@ -101,7 +101,6 @@ func (c *Cluster) makeDeployment(mdsConfig *mdsConfig, fsNamespacedname types.Na | |||
}, | |||
}, | |||
} | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could revert
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@travisn wrote:
Thanks! I addressed these |
This change is a follow-up to PR rook#13651 . It is also meant to provide a continuation of the closed, stale PR rook#13878 . It is furthermore intended as a preparation for the introduction of a way to enforce host networking in the next round of updates to PR rook#13651 . This change improves the documentation, implementation, and unit testing for the method of the spec. Signed-off-by: Michael Adam <obnox@samba.org> Co-authored-by: Blaine Gardner <b.blaine.gardner@gmail.com> Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
This change is a follow-up to PR #13693 .
It is also meant to provide a continuation of the closed, stale PR #13878 .
Finally, it is intended as a preparation for the introduction of a way to enforce host networking in the next round of updates to PR
#13651 .
This change improves the documentation comment and the unit testing for the
IsHost()
method of theCephCluster.Network
spec.The implementation of IsHost() is changed to validate the Network Spec and return error if the spec is invalid.
Checklist: