Default empty constructor for AmazonS3Client (and others)

[andyscott]

In some circumstances, leveraging the default constructor for the java S3 client is more appealing than the default constructor provided in AWScala.

Both default constructors look for credentials stored in system properties. However, the scala API fails if the parameters aren't present. The java API won't fail. And if you're running the code on an EC2 instance with an IAM role that grants permissions to S3 buckets, you'll be in the clear. See http://blogs.aws.amazon.com/security/post/Tx1XG3FX6VMU6O5/A-safer-way-to-distribute-AWS-credentials-to-EC2

At the moment, this is how I initialize my S3 client.

val s3 = new AmazonS3Client with S3 {
  override def createBucket(name: String): Bucket = super.createBucket(name)
}

Can we add more direct support for the default constructors of the underlying java APIs?

[seratch]

I'm afraid that I don't understand your point, Credentials.defaultEnv is available.

val s3 = S3() // = S3(Credentials.defaultEnv)

For instance, following environment variables are available, they will be loaded by default.

export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=yyy

Isn't it enough?

[andyscott]

When you're running on an ec2 instance, the IAM role of the instance can grant it access to S3 resources even when the AWS env vars aren't set. The default constructor for AWScala fails if the env vars aren't set.

On Feb 2, 2014, at 15:20, Kazuhiro Sera notifications@github.com wrote:

I'm afraid that I don't understand your point, Credentials.defaultEnv is available.

val s3 = S3() // = S3(Credentials.defaultEnv)
For instance, following environment variables are available, they will be loaded by default.

export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=yyy
Isn't it enough?

—
Reply to this email directly or view it on GitHub.

[seratch]

Thank you for your kind explanation. I got it.

I didn't know about it but it's pretty cool.
I'd like to consider supporting this.

[andyscott]

Gladly! It's very useful-- please add it in!

[seratch]

Now we can read EC2 instance profiles. Would you try this out?

resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"

libraryDependencies += "com.github.seratch" %% "awscala" % "0.2.0-SNAPSHOT"

[andyscott]

The AmazonS3Client constructor is still called with a cred arg at S3.scala#L207:

class S3Client(credentials: Credentials = CredentialsLoader.load())
    extends aws.AmazonS3Client(credentials)
    with S3 { /* ... */ }

Checking the AWS SDK javadoc here, if you pass in the cred param it won't fallback to the IAM role. Also, the default behavior is to check the env vars anyway, so it may be advantageous to let the java layer do the lifting there.

I can look into a fix and submit a pull request if you'd like.

[seratch]

I see.

I can look into a fix and submit a pull request if you'd like.

Yes, please! Currently, AWScala expects different environment varibales (AWS_SECRET_ACCESS_KEY instead of AWS_SECRET_KEY) but it's not reasonable. It's also OK to remove that and switch to use only Java SDK's credentials loading.

[apatrida]

This is a big problem, IAM roles are common, not using them is a very bad practice

[iceberg901]

I've just implemented a fix, pull request coming in a jiffy

[iceberg901]

So I've discovered that my fix isn't really a complete fix. Even though DefaultAwsCredentialsProviderChain is now used by deafult, awscala grabs the credentials once and then holds them forever. This doesn't work for IAM roles, because the credentials are regularly invalidated and regenerated, so once this happens once the credentials held by awscala stop working.

@seratch can you please reopen and I will write a real fix and create a PR.

[iceberg901]

From version 0.5.0 onward the default constructors of all clients in AWScala use the DefaultAWSCredentialsProviderChain, which will fallback to IAM role as the last choice in its chain. They will also automatically refresh IAM role credentials so they don't get stuck with expired ones.