-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding support for AWS creds via instance metadata with session token #42
base: master
Are you sure you want to change the base?
Conversation
adding aws session token support
I can't tell if build is failing because Coveralls is down or if there's a problem with my change. Either way, thanks! |
Thanks! Can you sign our CLA please: https://github.com/snowplow/snowplow/wiki/CLA |
Sent, thanks again. |
Confirmed! Thanks so much |
Any luck in rerunning tests? We've had only minor success compiling from our fork, running into issues with Fog version compatibility. Fog 1.24 returns a 'net-ssh missing' error, 1.36 an AWS request header region mismatch issue (config.yml passes us-west-2, Fog is defaulting to us-east-1). |
So interesting collision...it seems the Snowplow Storage Loader IAM handler and this PR are redundant. The Storage Loader handler actually rewrites the creds before Sluice receives them, so sluice is getting the actual instance profile creds instead of 'iam'. EMR-ETL-Runner is working great, but the double translation is failing Storage Loader. I still think this is best handled by Sluice, which leverages fog-aws's credentials fetcher naturally, but am not sure if there are other downstream dependencies or user applications outside of Snowplow. |
Right - this problem should go away when we do: snowplow/snowplow#2548 |
It looks like Fog is able to fetch credentials via IAM/instance profile. Adding a case to new_fog_s3_from() to check for use_iam_profile => true to prep for Snowplow EmrEtlRunner IAM/EC2 instance metadata support.