Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ec2_sd_configs in ScrapeConfig #5902

Merged
merged 1 commit into from Sep 20, 2023

Conversation

slashpai
Copy link
Contributor

@slashpai slashpai commented Sep 8, 2023

Fixes #5818

Not all fields are implemented, current change will support monitoring using API access and secret key

Description

Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request.
If it fixes a bug or resolves a feature request, be sure to link to that issue.

Add EC2 Service Discovery support in ScrapeConfig

This is how I tested the change:

  • Created an ec2 instance, added security group to open port 9100, installed node-exporter and run it

  • Attached the policy AmazonEC2ReadOnlyAccess to IAM credentials (reference)

  • Created k8s secret with api access key and secret key

  • Created ScrapeConfig with EC2SDConfig

    apiVersion: monitoring.coreos.com/v1alpha1
    kind: ScrapeConfig
    metadata:
      name: scrape-config-ec2
      namespace: default
      labels:
        app.kubernetes.io/name: scrape-config-ec2
    spec:
      ec2SDConfigs:
        - region: us-east-1
          accessKey:
            key: accessKey
            name: aws-api-access
          secretKey:
            key: secretKey
            name: aws-api-access
          port: 9100
      relabelings:
      - sourceLabels: [__meta_ec2_public_ip]
        replacement: ${1}:9100
        targetLabel: __address__

    Relabelling was required because only public ip will be accessible to scrape target metrics

Result:

ec_target

I added unit tests, I am exploring how to do e2e test using localstack but that will take some time to explore in free time

Type of change

What type of changes does your code introduce to the Prometheus operator? Put an x in the box that apply.

  • CHANGE (fix or feature that would cause existing functionality to not work as expected)
  • FEATURE (non-breaking change which adds functionality)
  • BUGFIX (non-breaking change which fixes an issue)
  • ENHANCEMENT (non-breaking change which improves existing functionality)
  • NONE (if none of the other choices apply. Example, tooling, build system, CI, docs, etc.)

Changelog entry

Please put a one-line changelog entry below. This will be copied to the changelog file during the release process.

Add EC2 Service Discovery support in ScrapeConfig

@slashpai slashpai requested a review from a team as a code owner September 8, 2023 20:24
pkg/apis/monitoring/v1alpha1/scrapeconfig_types.go Outdated Show resolved Hide resolved
pkg/apis/monitoring/v1alpha1/scrapeconfig_types.go Outdated Show resolved Hide resolved
pkg/apis/monitoring/v1alpha1/scrapeconfig_types.go Outdated Show resolved Hide resolved
pkg/prometheus/promcfg.go Outdated Show resolved Hide resolved
pkg/prometheus/promcfg_test.go Outdated Show resolved Hide resolved
pkg/prometheus/resource_selector.go Outdated Show resolved Hide resolved
@slashpai
Copy link
Contributor Author

Please have another look on this :)

pkg/apis/monitoring/v1alpha1/scrapeconfig_types.go Outdated Show resolved Hide resolved
pkg/prometheus/promcfg.go Outdated Show resolved Hide resolved
pkg/prometheus/promcfg_test.go Show resolved Hide resolved
Fixes prometheus-operator#5818

Signed-off-by: Jayapriya Pai <slashpai9@gmail.com>
@slashpai
Copy link
Contributor Author

addressed comments

@ArthurSens ArthurSens merged commit f16f1d7 into prometheus-operator:main Sep 20, 2023
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support EC2 service discovery with ScrapeConfig CRD
3 participants