feat(monitoring): add servicemonitor and port to exporter sidecar

[KarstenSiemer]

⚠️ Note on feature completeness ⚠️

We are narrowing the scope of acceptable enhancements to DefectDojo in preparation for v3. Learn more here:
https://github.com/DefectDojo/django-DefectDojo/blob/master/readme-docs/CONTRIBUTING.md

Description

Describe the feature / bug fix implemented by this PR.
If this is a new parser, the parser guide may be worth (re)reading.

Test results

Ideally you extend the test suite in tests/ and dojo/unittests to cover the changed in this PR.
Alternatively, describe what you have and haven't tested.

Documentation

Please update any documentation when needed in the documentation folder)

Checklist

This checklist is for your information.

• Make sure to rebase your PR against the very latest dev.
• Features/Changes should be submitted against the dev.
• Bugfixes should be submitted against the bugfix branch.
• Give a meaningful name to your PR, as it may end up being used in the release notes.
• Your code is flake8 compliant.
• Your code is python 3.11 compliant.
• If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
• Model changes must include the necessary migrations in the dojo/db_migrations folder.
• Add applicable tests to the unit tests.
• Add the proper label to categorize your PR.

Extra information

Please clear everything below when submitting your pull request, it's here purely for your information.

Moderators: Labels currently accepted for PRs:

• Import Scans (for new scanners/importers)
• enhancement
• performance
• feature
• bugfix
• maintenance (a.k.a chores)
• dependencies
• New Migration (when the PR introduces a DB migration)
• settings_changes (when the PR introduces changes or new settings in settings.dist.py)

Contributors: Git Tips

Rebase on dev branch

If the dev branch has changed since you started working on it, please rebase your work after the current dev.

On your working branch mybranch:

git rebase dev mybranch

In case of conflict:

 git mergetool
 git rebase --continue

When everything's fine on your local branch, force push to your myOrigin remote:

git push myOrigin --force-with-lease

To cancel everything:

git rebase --abort

Squashing commits

git rebase -i origin/dev

• Replace pick by fixup on the commits you want squashed out
• Replace pick by reword on the first commit if you want to change the commit message
• Save the file and quit your editor

Force push to your myOrigin remote:

git push myOrigin --force-with-lease

[cneill]

As mentioned here, we are considering breaking all the 3rd party bits like Prometheus, cloud-sql, etc off on their own to avoid cluttering the chart for people who don't use that functionality. I'm more sympathetic to adding Prometheus functionality than cloud provider-specific functionality, but I think when we start relying on Custom Resource Definitions, we're probably being too expansive in what we include in the main chart. I'm very open to feedback here. If you have any thoughts on how to make this kind of integration easy for deployers without adding additional service-specific bits to the chart, please let us know.

[KarstenSiemer]

@cneill it is indeed a tough problem and I generally do agree with your statements.
With helm it is just very hard to add functionality afterwards. Take a look at the nginx exporter sidecar.
Whoever added this functionality to the chart, did not include port information onto the pod for that container.
I cannot add the port to the pod without creating a PR to the project.

If it is decided to not include third party tools, this chart needs to include a lot of loops to iterate in sidecars, volumes etc.

In regards to serviceMonitor, it is a staple in many opensource projects and included in a lot of charts. If it is already in, I am happy to use it and thought many others might be as well.

Would the PR be acceptable, if the serviceMonitor is removed? I have no problems with adding third party stuff another way, but I certainly need the port on the sidecar.

[dsever]

@KarstenSiemer as you stated this is the common way how people are distributing this days their helm chart, I like see it is improvement.
Another side of the medal is we want to deliver helm in the production ready states, and we have hard times to test all options that are part of the helm.
So we need to find some sweet spot...

[KarstenSiemer]

@DSSever, I'm uncertain if there is a definitive "sweet spot" to be found, as even a term like "production ready" can have different interpretations depending on the observer.

To cater to the broadest range of users, the most suitable production ready state for this chart would likely be a minimal configuration that can be expanded upon. Users would have the option to add more sidecars and their corresponding ports, for instance. By excluding any vendor-specific code for AWS, GCP, or Azure, this chart would cater to bare metal users while still allowing expandability for AWS users who may prefer using their own custom sidecars and code. For example, on AWS, one might opt to use IAM instead of a password for connecting to a managed PostgreSQL database.

In this approach, a serviceMonitor would not be included in the chart based on the provided definition. The primary focus of this chart would be to deploy DefectDojo efficiently, keeping the scope limited and the codebase more manageable and conducive to contributions.

Any additional value that currently resides in the cloud-specific parts of the chart could be extracted and documented separately, providing examples and instructions on how to achieve those specific configurations.

[Maffooch]

Closing due to stale