Update provider-api-server to be deployed in internal and provider mode

[rewantsoni]

Allow provider-server to be deployed in intenal and provider mode

Why?
This is a part of RDR for Provider Mode RHSTOR-4886.
With this epic, we are introducing rpc calls for odf to odf communication, the RPC's we have for this epic are limited to Onboarding a Storage Cluster on different Openshift clusters and Getting the Mirroring Info required for setting up RDR (refer #2671).

This can also be used to replace the mechanism we have in the internal mode for setting up Mirroring for blockpool using MCO's Agents.

Hence moving the ocs-provider-server deployment to be deployed in both modes to enable both client to provider communication and cross odf communication.

The PR does the following:

1. Create service, deployment, onboarding job for both modes
2. Update the variable from watchnamespace to podnamespace
3. Remove hardcoded name for storagecluster
4. Move client configmap in storageclient
5. Update the unit tests

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[nb-ohad]

@rewantsoni Please provide a proper PR title and description

[iamniting]

Can the provider handle requests for multiple storage clusters?

Do we want to run only one ocs-provider-server deployment in case of the multiple storagecluster?

If yes then we can have it as part of the ocs initialization otherwise it should be part of the storagecluster reconciliation.

[iamniting]

Can you pls add deprication warning in the storage cluster API in first commit?

Also pls separate the generated changes in the 3rd commit and rename the deployment to the ocs-server.

[rewantsoni]

Can the provider handle requests for multiple storage clusters?

Yes, the server should be able to handle multiple storage clusters, but the client will onboard to only the storageCluster in the same namespace as the provider-server

Do we want to run only one ocs-provider-server deployment in case of the multiple storagecluster?

Yes, one provider-server for multiple storageClusters

If yes then we can have it as part of the ocs initialization otherwise it should be part of the storagecluster reconciliation.

I have added the deployment as a part of csv and secret, service as part of ocsinit

[iamniting]

Can you pls resolve merge conflict

[rewantsoni]

/retest-required

[leelavg]

client-op needs an update after this gets merged, isn't it?

[rewantsoni]

client-op needs an update after this gets merged, isn't it?

@leelavg I don't think we need any update in client o/p after merging this, let me know if I missed anything.

[nb-ohad]

LGTM

[rewantsoni]

rebased the PR

[nb-ohad]

@iamniting Can you please take a look as well?

[rewantsoni]

moved unit-tests into a separate commit

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: iamniting, rewantsoni

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [iamniting]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[iamniting]

/unhold