feat: Allow user instance monitoring

[jaideepr97]

What type of PR is this?

Uncomment only one /kind line, and delete the rest.
For example, > /kind bug would simply become: /kind bug

/kind bug
/kind chore
/kind cleanup
/kind failing-test
/kind enhancement
/kind documentation
/kind code-refactoring

What does this PR do / why we need it:
This PR allows users to enable workload status monitoring for a given argo-cd instance. Users can enable monitoring for their instance by setting .spec.monitoring.enabled to true, which creates a PrometheusRule for alerts (with 7 opinionated alert rules within it - one for each workload). The rules are configured to fire when a workload has not had the required number of ready replicas for a certain duration of time out of the box. Users are free to delete or make changes to the alert rule and the operator would not overwrite the action.

Have you updated the necessary documentation?
Not yet

• Documentation update is required by this PR.
• Documentation has been updated.

Which issue(s) this PR fixes:

Fixes #?

How to test changes / Special notes to the reviewer:

1. Launch operator locally against an openshift cluster or any cluster that contains the prometheus API
2. create a namespace and argo-cd instance within that namespace
3. If using openshift - enable user workload monitoring for your cluster by running oc -n openshift-monitoring edit configmap cluster-monitoring-config and setting data.config.yaml.enableUserWorkload = true (https://docs.openshift.com/container-platform/4.6/monitoring/enabling-monitoring-for-user-defined-projects.html?extIdCarryOver=true&sc_cid=701f2000001Css5AAC#enabling-monitoring-for-user-defined-projects_enabling-monitoring-for-user-defined-projects)
4. Set .spec.monitoring.enabled=true in CR
5. Check that prometheusRule is created in the namespace
6. On OpenShift you can go to Admin > Observe > Alerting > Alerting Rules to ensure that all 7 alerting rules show up in the list (Remove any pre-applied filters0)
7. Set .spec.applicationSet.image=test-image to create a situation where # desired replicas for appset controller != # ready replicas
8. Go back to Admin > Observe > Alerting > Alerts and you should see the ApplicationSetControllerNotReady alert in this list (remove all pre-applied filters) with state initially set as "pending" which should change to "firing" after 10m

[drpaneas]

Great work @jaideepr97 💪 🚀 🚀 I've added a couple of comments, but I wouldn't consider any of those as blockers.

• Some corner test-cases are missing, but in general awesome work for adding unit-tests for the alerts!!!
• This PR adds a new YAML manifest. Is this auto-generated or you wrote it by hand?
• This can be done in separate story as well: a doc listing the alerts we support.

[jaideepr97]

@drpaneas Thanks for taking a look :) I don't see any comments on here though, did they get saved?

[drpaneas]

@drpaneas Thanks for taking a look :) I don't see any comments on here though, did they get saved?

sorry I forgot to submit them. They should be visible now ;)

[drpaneas]

/lgtm

[wtam2018]

Please add documentation in the same PR.

[iam-veeramalla]

LGTM, Thanks for adding the documentation @jaideepr97

Approved.