AWS/EKS: Support for IAM authentication

[headcr4sh]

I am trying to connect a postgres_exporter instance running on AWS EKS (managed Kubernetes platform) to a managed AWS/RDS instance using IAM authentication methods as described here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.AWSCLI.PostgreSQL.html

My attempt to use an initContainer to obtain the access token (which is valid for a limited period of time only) works just fine:

      initContainers:
      - args:
        - -c
        - "echo \"Generating RDS authentication token\" && /usr/bin/aws rds generate-db-auth-token --hostname $(RDS_HOST) --port $(RDS_PORT) --username $(POSTGRES_USER) --region $(AWS_DEFAULT_REGION) > /etc/postgresql-exporter/aws-rds/connection && cat /etc/postgresql-exporter/aws-rds/connection"
        command:
        - /bin/sh
        envFrom:
        - configMapRef:
            name: map_that_holds_my_environment_variables
        # This image contains only the AWS CLI and since Amazon hasn't officially released a
        # container image (yet), I had to roll my own... :-(
        image: 1234567890123.dkr.ecr.eu-central-1.amazonaws.com/library/awscli:v1.16.247
        imagePullPolicy: IfNotPresent
        name: generate-db-auth-token
        resources:
          limits:
            cpu: 100m
            memory: 120Mi
          requests:
            cpu: 100m
            memory: 100Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
        volumeMounts:
        - mountPath: /etc/postgresql-exporter/aws-rds
          name: config
          readOnly: false

As you might see, the initContainer writes an RDS connection string into a file which I put onto a shared volume (tmpDir) that is accessible from the main container (postgres-exporter). Now comes the tricky part: This connection string contains the full postgresql connection string.

Example (credentials obfuscated):

my-rds-instance-name.abcdef1x2y3z.eu-central-1.rds.amazonaws.com:5432/?Action=connect&DBUser=prometheus_exporter&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=900&X-Amz-Date=20191104T190218Z&X-Amz-SignedHeaders=host&X-Amz-Security-Token=SfxcvIvYXdzEEQaDGuApMrWVOItKG756iKMA4if966d5Th%2BB9IXSv6ZDGI2DXcxkO9YP1jp8oY189vtQuh1dsG7h45Uv3waOWx76gKXp%2F8D2OT2WaD8iL0P9g36VkXVD1J6RNfsdKwshSw%2B5qOuODo%2F1B%2B6ZKrQ%2BHv5oIHODixnZH8jD9NgyjVPHcqB4PTVYTaelOJtL1tQ484g5d3vT8iGvdEpaRLdLzW3IM2wPuWyvuw9R7eqwbolUHrgEIpWphsRUmIhT3tV%2BIxwjm76y093zQQViDauAEjsGQjUyw4brYN%2B7uuADcfT%2BLlQtczTZUFoobfhw%2F2zN3BqJVozuMOLYCJU9v5zkegZY8ikp%2FwWyVM0cmHF%2BsS4p54OYRGfG8NxObgOjFIxdcVvV%2FiQf%2Ff3F3fYn2WZyod3ERqHQlOvqnO6QYG0TS0kGpTHD6IknlCksVDxf%2F3akpA8O70HIq4XR4qj0jJLoAQZoMGbjDeOzm3QoGo%2FWLysVQBA1K0%2BArcnVpIQZfg6e2LtsBuyG%2BasiIqX52PW2aZ915GNldEJC8cJEKo3hSi664HuBTLyAVzWy%2B36zzWfFTx%2FKZqmPljoIdf5pIyGPhr9HN5Srv2qEBB6939Eueb1F7t2HLCQPlhZF8TrdEnAeW8qM3JihgICQldUxcpjxBFsDxhTgMe%2Fdvftl%2BCL7W74N2%2FLNofVEwDrVTwAa4XZw9TUNkrTxPVKXJl%2B7vwqPvGjfmCljANUgNW9JPg2ctAn0O7T%2F2ZoJdnFessUJ17%2F73bgQVaFJyQNq9n87J8pILqf1YUlU%2FS7eePOcVEO0Rn%2FTQyKQhcatTwAoebPgHEC04%2FbEw3PQFNtdd5C5J2y29FQEN9Pe%2BOKNCNaQAtMgq9Jlul2ljkr98tC&X-Amz-Credential=ADIAWZGGSLFRQZT43XHR%2F20191104%2Feu-central-1%2Frds-db%2Faws4_request&X-Amz-Signature=8df675ac12530eb757bffb3818681d0ff361b4e79226c0a31aac546b92e1080a

Unfortunately, it is not possible, to use that string when using the official Docker image of the postgres_exporter, because there is no way to construct the DATA_SOURCE_URI environment variable from a file like that. Overriding the container's entrypoint would be an option:

  command:
  - sh
  args:
  -c
  - "DATA_SOURCE_URI=\"postgresql://$(cat /etc/postgresql-exporter/aws-rds)\" /postgres_exporter"

But right now it is not an option, since the container does not contain a shell (which is actually a good thing when considering best container security practices, I assume).

... any help/ideas appreciated.

PS:
After having thought about it for quite some time, I suppose, a DATA_SOURCE_URI_FILE environment variable might solve all my issues. Would that make for an acceptable PR?

[robbiet480]

@headcr4sh Your PR is great but i'm wondering how you are triggering a token update every 15 minutes since the password expires every 15?

[headcr4sh]

@robbiet480 Unfortunately, I was not able to solve any further issues with the postgres_exporter in conjunction with EKS and IAM-based ServiceAccounts.

So... I was not able to trigger the token update. :-(

(Using the posgres_exporter was just a proof-of-concept and I did not proceed any further, unfortunately,...)