add additionalScrapeJobs for prometheus

[guerremdq]

Description

We want to scrape other services like ie: Push gateway running on the same cluster and we need to be able to modify the Prometheus configuration.

is this something that could be supported by Astronomer?

[danielhoherd]

@guerremdq thanks for the submission! Please add additionalScrapeJobs: {} to charts/prometheus/values.yaml so the new feature is more discoverable. Also we need to test the new feature, so please add a new test_additional_scrape_jobs test case for it in tests/chart_tests/test_prometheus_config_configmap.py. There are a bunch of test cases in there that would be great as a copy/paste way to start the new test case. If you need help getting started, here is a short tutorial https://github.com/astronomer/astronomer/blob/master/tests/chart_tests/README.md and I'm free to answer any questions, review code, and give suggestions.

[guerremdq]

Thanks @danielhoherd for the quick reply, I added the test and the default values.

[danielhoherd]

This looks good to me. I'd like @pgvishnuram to sign off on it too.

[guerremdq]

@danielhoherd We are currently running 0.32.4 can this maybe cherry-picked and released under 0.32 too?

[danielhoherd]

@guerremdq yeah, this is something that is easy to include in all supported versions, which would be 0.32, 0.33, 0.34. https://docs.astronomer.io/software/release-lifecycle-policy#stable

[pgvishnuram]

This PR looks fine we can merge it

[danielhoherd]

Merged to master, merged master to release-0.34, cherry-picked change to release-0.33 and release-0.32

Thanks for the feature addition @guerremdq !

Internal ticket for this issue: https://github.com/astronomer/issues/issues/6192

[guerremdq]

Hey @danielhoherd when we can expect the new release ?

[danielhoherd]

@guerremdq 0.34.1 will probably be released in early to mid April, and will include this new feature.