vmagent: Support AWS's sigv4 signing process for remote_write

[chrissng]

Is your feature request related to a problem? Please describe.

I would like vmagent to be used as a standalone metric scraper that is able to write to any backend via the remote_write protocol, such as a managed service like the AWS Managed Service for Prometheus (AMP). vmagent is preferred over the prometheus server because it typically requires lower amounts of RAM, CPU and network bandwidth.

Describe the solution you'd like

Support the sigv4 AWS's Signature Verification 4 signing process. This should support writing metrics to AMP with sigv4 authentication. remote_write sigv4 configuration is specified here.

Some possible command-line flags could be:

-remoteWrite.sigv4Region
    Specifies the AWS region

-remoteWrite.sigv4Profile
    Specifies the AWS profile used to authenticate

-remoteWrite.sigv4RoleArn
    Specifies AWS Role ARN as an alternative to using AWS API keys

Describe alternatives you've considered

• Deploy a sigv4 proxy sidecar https://github.com/awslabs/aws-sigv4-proxy and specify the --host=HOST host to proxy to
• Deploy a VM cluster with vminsert, vmselect and vmstorage on your own.

Additional context

AWS official doc on remote_write for AMP: here

[f41gh7]

Hello, as temporary workaround, its possible to use sigv4 proxy https://github.com/awslabs/aws-sigv4-proxy, configure it with needed params and point -remoteWrite.url=http://localhost:8080 - actual local address of sigv4 proxy.

[chrissng]

@f41gh7 thanks for the info. This is useful and I think it should work. Will include this as an alternative

[valyala]

vmagent gained the ability to send data to -remoteWrite.url with sigv4 signature starting from v1.77.0. This can be done with the following command-line flags:

  -remoteWrite.aws.accessKey array
     Optional AWS AccessKey to use for -remoteWrite.url if -remoteWrite.aws.useSigv4 is set. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
     Supports an array of values separated by comma or specified via multiple flags.
  -remoteWrite.aws.region array
     Optional AWS region to use for -remoteWrite.url if -remoteWrite.aws.useSigv4 is set. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
     Supports an array of values separated by comma or specified via multiple flags.
  -remoteWrite.aws.roleARN array
     Optional AWS roleARN to use for -remoteWrite.url if -remoteWrite.aws.useSigv4 is set. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
     Supports an array of values separated by comma or specified via multiple flags.
  -remoteWrite.aws.secretKey array
     Optional AWS SecretKey to use for -remoteWrite.url if -remoteWrite.aws.useSigv4 is set. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
     Supports an array of values separated by comma or specified via multiple flags.
  -remoteWrite.aws.useSigv4 array
     Enables SigV4 request signing for -remoteWrite.url. It is expected that other -remoteWrite.aws.* command-line flags are set if sigv4 request signing is enabled. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
     Supports array of values separated by comma or specified via multiple flags.

Closing the feature request as done.