object: check obc provisioner for bucket notification

[thotz]

Description of your changes:
obc-label-controller needs to check only the obc provisioned by the Rook operator.
Check other OBCs result in errors from the controller and keep on reconcilling

Which issue is resolved by this Pull Request:
Resolves #

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide).
• Skip Tests for Docs: If this is only a documentation change, add the label skip-ci on the PR.
• Reviewed the developer guide on Submitting a Pull Request
• Pending release notes updated with breaking and/or notable changes for the next minor release.
• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• Integration tests have been added, if necessary.

[galexrt]

LGTM

[yuvalif]

@thotz maybe a simpler way is to use the error returned from the name parsing function as an indication that the bucket was provisioned by an external provisioner, and let the bucket notification controller ignore it without an error?

[travisn]

@thotz The object integration tests are failing with this error:

=== RUN   TestCephObjectSuite/TestWithTLS/perform_s3_operations_and_check_for_notifications/check_for_put_bucket_notification
2023-03-27 12:10:33.431665 D | exec: Running command: kubectl logs --selector=app=sample-http-server --tail 5
    ceph_bucket_notification_test.go:142: 
        	Error Trace:	/home/runner/work/rook/rook/tests/integration/ceph_bucket_notification_test.go:142
        	Error:      	Should be true
        	Test:       	TestCephObjectSuite/TestWithTLS/perform_s3_operations_and_check_for_notifications/check_for_put_bucket_notification

[BlaineEXE]

Thanks for the good test updates in the PR. I think it looks good other than the small integration test bug Travis pointed out

[thotz]

@thotz The object integration tests are failing with this error:

=== RUN   TestCephObjectSuite/TestWithTLS/perform_s3_operations_and_check_for_notifications/check_for_put_bucket_notification
2023-03-27 12:10:33.431665 D | exec: Running command: kubectl logs --selector=app=sample-http-server --tail 5
    ceph_bucket_notification_test.go:142: 
        	Error Trace:	/home/runner/work/rook/rook/tests/integration/ceph_bucket_notification_test.go:142
        	Error:      	Should be true
        	Test:       	TestCephObjectSuite/TestWithTLS/perform_s3_operations_and_check_for_notifications/check_for_put_bucket_notification

I thought the provisioner's name was the combination of Rook Operator Namespace and "ceph.rook.io/bucket". But the current code uses the ceph cluster namespace. I am confused about how to get that here. In the Bucketprovisioner the are finding the ceph cluster details using the following,

cephCluster := &cephv1.CephCluster{}
	err := r.client.Get(r.opManagerContext, request.NamespacedName, cephCluster)
	if err != nil {
		if kerrors.IsNotFound(err) {
			logger.Infof("no ceph cluster found in %+v. not deploying the bucket provisioner", request.NamespacedName)
			return reconcile.Result{}, nil
		}
		// Error reading the object - requeue the request.
		return opcontroller.ImmediateRetryResult, errors.Wrap(err, "failed to get the ceph cluster")
	}

	if !cephCluster.DeletionTimestamp.IsZero() {
		logger.Debug("ceph cluster is being deleted, no need to reconcile the bucket provisioner")
		return reconcile.Result{}, nil
	}

	if !cephCluster.Spec.External.Enable && cephCluster.Spec.CleanupPolicy.HasDataDirCleanPolicy() {
		logger.Debug("ceph cluster has cleanup policy, the cluster will soon go away, no need to reconcile the bucket provisioner")
		return reconcile.Result{}, nil
	}

The request.NamespacedName should be the namespace of OBC right? AFAIK obc can exist in any namespace, not bind to the namespace of ceph cluster or object store

[travisn]

The namespace of the cephcluster is the same as the namespace of the object store. Would this method help? getNSNameFromAdditionalState()?

[thotz]

The namespace of the cephcluster is the same as the namespace of the object store. Would this method help? getNSNameFromAdditionalState()?

That function was erroring out while handling OBC which were not provisioned by Rook. So I thought handling this check before fetching details about object store in the OBC

[subhamkrai]

Just adding a comment that lots of CI are failing.