ceph: Allow an even number of mons

[travisn]

Description of your changes:
While an even number of mons can cause lower availability of mon quorum, it also can provide higher durability for the cluster. Mon quorum can be restored from a single mon according to the disaster recovery guide, so there may be scenarios where an even number of mons may be preferable.

Reverts the check for an even number of mons added in #6420.

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
• Skip Tests for Docs: Add the flag for skipping the build if this is only a documentation change. See here for the flag.
• Skip Unrelated Tests: Add a flag to run tests for a specific storage provider. See test options.
• Reviewed the developer guide on Submitting a Pull Request
• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• Integration tests have been added, if necessary.
• Pending release notes updated with breaking and/or notable changes, if necessary.
• Upgrade from previous release is tested and upgrade user guide is updated, if necessary.
• Code generation (make codegen) has been run to update object specifications, if necessary.

[BlaineEXE]

I like the doc changes.

[BlaineEXE]

While we are revising this, should we set // +kubebuilder:validation:Maximum=9 in types.go so the validation will stop users from choosing more than 9?

[parth-gr]

rook/pkg/apis/ceph.rook.io/v1/cluster.go

Line 58 in d577fea

if updatedCephCluster.Spec.Mon.Count > 0 && updatedCephCluster.Spec.Mon.Count%2 == 0 {

Is this API should also be updated with the change?

[parth-gr]

What for the case for automatic adding of mons if new nodes are added, is this we need to keep the scenario the same? add mons auto if there are an odd number of nodes present.

[travisn]

rook/pkg/apis/ceph.rook.io/v1/cluster.go

Line 58 in d577fea

if updatedCephCluster.Spec.Mon.Count > 0 && updatedCephCluster.Spec.Mon.Count%2 == 0 {

Is this API should also be updated with the change?

Not sure what you mean? We are relaxing the check so we allow an even number of mons now.

What for the case for automatic adding of mons if new nodes are added, is this we need to keep the scenario the same? add mons auto if there are an odd number of nodes present.

If new nodes are added, the behavior would still be the same as before. We usually don't add new mons with new nodes though.

[parth-gr]

Not sure what you mean? We are relaxing the check so we allow an even number of mons now.

Sorry, I just thought this place was missed.

If new nodes are added, the behavior would still be the same as before. We usually don't add new mons with new nodes though.

Okay :)

And just Check if this warning needs to be updated

rook/pkg/operator/ceph/cluster/mon/health.go

Lines 600 to 602 in 8af24cf

	if monCount%2 == 0 {
	logger.Warningf("external cluster mon count is even (%d), should be uneven, continuing.", monCount)
	}

[travisn]

@parth-gr Yeah, removed the message from the external cluster, thanks!