IAM auth_ec2 doesn't work, and seems like it can't possibly work

[kenrestivo-stem]

According to the Vault documentation for the iam/ec2 auth endpoints, it works like this:

curl -X POST "http://127.0.0.1:8200/v1/auth/aws/login" -d '{"role":"dev", "iam_http_request_method": "POST", "iam_request_url": "aHR0cHM6Ly9zdHMuYW1hem9uYXdzLmNvbS8=", "iam_request_body": "QWN0aW9uPUdldENhbGxlcklkZW50aXR5JlZlcnNpb249MjAxMS0wNi0xNQ==", "iam_request_headers": "eyJDb250ZW50LUxlbmd0aCI6IFsiNDMiXSwgIlVzZXItQWdlbnQiOiBbImF3cy1zZGstZ28vMS40LjEyIChnbzEuNy4xOyBsaW51eDsgYW1kNjQpIl0sICJYLVZhdWx0LUFXU0lBTS1TZXJ2ZXItSWQiOiBbInZhdWx0LmV4YW1wbGUuY29tIl0sICJYLUFtei1EYXRlIjogWyIyMDE2MDkzMFQwNDMxMjFaIl0sICJDb250ZW50LVR5cGUiOiBbImFwcGxpY2F0aW9uL3gtd3d3LWZvcm0tdXJsZW5jb2RlZDsgY2hhcnNldD11dGYtOCJdLCAiQXV0aG9yaXphdGlvbiI6IFsiQVdTNC1ITUFDLVNIQTI1NiBDcmVkZW50aWFsPWZvby8yMDE2MDkzMC91cy1lYXN0LTEvc3RzL2F3czRfcmVxdWVzdCwgU2lnbmVkSGVhZGVycz1jb250ZW50LWxlbmd0aDtjb250ZW50LXR5cGU7aG9zdDt4LWFtei1kYXRlO3gtdmF1bHQtc2VydmVyLCBTaWduYXR1cmU9YTY5ZmQ3NTBhMzQ0NWM0ZTU1M2UxYjNlNzlkM2RhOTBlZWY1NDA0N2YxZWI0ZWZlOGZmYmM5YzQyOGMyNjU1YiJdfQ==" }'

https://www.vaultproject.io/docs/auth/aws.html

After un-base64-ing that, and formatting it to make it somewhat readable, it looks like:

   curl -X POST "http://127.0.0.1:8200/v1/auth/aws/login" -d '{"role":"dev", \
              "iam_http_request_method": "POST", \
               "iam_request_url": "https://sts.amazonaws.com/", \
                "iam_request_body": "Action=GetCallerIdentity&Version=2011-06-15", \
                "iam_request_headers": "{"Content-Length": ["43"], \
               "User-Agent": ["aws-sdk-go/1.4.12 (go1.7.1; linux; amd64)"], \
               "X-Vault-AWSIAM-Server-Id": ["vault.example.com"], \
              "X-Amz-Date": ["20160930T043121Z"], \
              "Content-Type": ["application/x-www-form-urlencoded; charset=utf-8"], \
               "Authorization": ["AWS4-HMAC-SHA256 Credential=foo/20160930/us-east-1/sts/aws4_request,                       SignedHeaders=content-length;content-type;host;x-amz-date;x-vault-server, \
Signature=a69fd750a3445c4e553e1b3e79d3da90eef54047f1eb4efe8ffbc9c428c2655b"]}" }'

But I'm looking through the hvac code, and auth_ec2 does none of those things.

In experimenting with it, I'm getting explosions like:

>> client.auth_ec2(requests.get("http://169.254.169.254/latest/dynamic/instance-identity/pkcs7").text)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/ops/venv/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 562, in auth_ec2
    return self.auth('/v1/auth/aws-ec2/login', json=params, use_token=use_token)
  File "/opt/ops/venv/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 787, in auth
    response = self._post(url, **kwargs).json()
  File "/opt/ops/venv/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 947, in _post
    return self.__request('post', url, **kwargs)
  File "/opt/ops/venv/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 986, in __request
    self.__raise_error(response.status_code, text, errors=errors)
  File "/opt/ops/venv/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 992, in __raise_error
    raise exceptions.InvalidRequest(message, errors=errors)
hvac.exceptions.InvalidRequest: missing client token

Missing client token is not what it should be responding. But then, hvac doesn't appear to actually be even trying to authenticate properly, so the server appears to be trying to authenticate it with the default (token) auth.

Does this auth_ec2 even work? Or am I missing something very obvious and fundamental?

[ianwestcott]

It sounds like this might be related to #108

[kenrestivo-stem]

Orthogonal issues.

The way I read #108 is that it relates to creating and managing EC2/IAM roles from the library, as a superuser, for example..

This ticket is about what appears to be the complete lack of library support for its use as an unprivileged client, to log in using IAM.

[jthompson75]

Vault changed the aws auth backend completely in one of the later releases and this doesn't seem to be reflected in hvac yet.

[kenrestivo-stem]

1. Is anyone working on that?
2. Do you want help/contributions/patches?

[otakup0pe]

1. Maybe? Does anything in Support new parameters on Vault aws-ec2 role creation #109 or Support specifying period when creating EC2 roles #140 cover it?
2. yes please 💖

[myoung34]

What ive done is used requests to authenticate manually and get a token, then create an hvac client with that token. Its tedious but works

[rberlind]

I would love to see the relevant section of your code, @myoung34, especially if you are using the iam option for the Vault AWS authentication method rather than the ec2 method. I need to use the iam option with a Lambda function.

When you said you did requests manually, did you mean by doing calls equivalent to those in https://github.com/hashicorp/vault/blob/master/builtin/credential/aws/cli.go?

[rberlind]

Actually, it seems that #126 provides some code to do this outside the hvac code.

[myoung34]

@rberlind I use this then we just call client with the token and use some wrapped methods

In our containers:

get_vault_token() {
  python3 -c "from stratatilities.auth import return_token;print(return_token())"
}

export VAULT_TOKEN=${VAULT_TOKEN:-$(get_vault_token)}

and in lambda:

from stratatilities.auth import get_vault_client, read_vault_secret
def lambda_handler(event, context):
    vault_client= get_vault_client()
    some_token = read_vault_secret(vault_client, 'secret/ops/some_token')

[rberlind]

Thanks for clarifying what you are doing @myoung34. That is interesting, but you are not using the Vault AWS iam authentication method. I was able to do that by making a modification to code provided by @ltm in #170. See my comment #170 (comment)

[jeffwecan]

It appears to me that the latest AWS auth backend expects a route of /v1/auth/aws/login where the current auth_ec2() method implicated here is sending the request to /v1/auth/aws-ec2/login. I'll try to track down the specifics behind that change (e.g., which Vault release introduced it?, etc.) and submit a patch if it is indeed that simple.

[rberlind]

You raise a good point, @jeffwecan. The actual mountpoint for any Vault auth method can vary depending on how it was created with the Enable Auth Method (https://www.vaultproject.io/api/system/auth.html#enable-auth-method) or auth enable CLI command (https://www.vaultproject.io/docs/commands/auth/enable.html). This is pointed out at the top of https://www.vaultproject.io/api/auth/aws/index.html.

So, one could create an instance of the AWS auth method with type ec2 at path aws-ec2 and the methods in this library that refer to /v1/auth/aws-ec2 should work fine.

However, it would be much better if the methods in this library that do refer to "aws-ec2" were instead parameterized like other methods in the library to use a mount_point argument that would then be referenced in the actual call to the HTTP API instead of "aws-ec2".

[jeffwecan]

Ahh that makes total sense! In light of that, Ill see about adding a parameter to that effect (perhaps with the current hard-coded value as a default for backwards compatibility?) and see what I can come up with.

[hayderimran7]

@myoung34 your wrapper-methods worked like a charm! thanks for sharing that.

[jeffwecan]

This is ideally fixed in #181 and will be included in the next release. Feel free to reopen this issue if its not quite resolved however.

[lrodri29]

@jeffwecan Is this update in the pip installable version of hvac?
Or should I install this module through the github repo? I seem to still have the same problem as first described in this issue

[jeffwecan]

This issue should be closed and the fix was released in version 0.6.0 which is published on Pypi. If you're getting a "missing client token" exception it's quite possibly a configuration issue. However if you are able to provide some details and/or replication steps I'd be happy to look into your specific issue.

[lrodri29]

Might be something really simple, but I am currently using this:

response = client.auth_ec2(pkcs7=pkcs7, mount_point='aws-ec2')

where client = hvac.Client(url=url)

My mistake may be not specifying the role argument? If so would the role be the one created during vault configuration by writing to /auth/aws-ec2/role?

[jeffwecan]

Correct. Or at least I believe the role name defaults to the instance's AMI ID derived from the pkcs7 string:

Name of the role against which the login is being attempted. If role is not specified, then the login endpoint looks for a role bearing the name of the AMI ID of the EC2 instance that is trying to login if using the ec2 auth method, or the "friendly name" (i.e., role name or username) of the IAM principal authenticated. If a matching role is not found, login fails.

[lrodri29]

How would I authenticate if I am trying to log in to a role configured in /auth/aws/role. According to the instructions here https://www.vaultproject.io/docs/auth/aws.html, when configuring authentication for an EC2 instance we create the role in the /auth/aws/role path, but when using auth_ec2() I am taken to the aws-ec2 path

[lrodri29]

Nevermind, just realized I can just exclude the mounting path