connection from ycsb to dynamoDB not working

[daljeetsingh23]

Hi team,

We are trying to use ycsb to connect to dynamodb to perform some benchmarking.

In our environment we are only allowed to use AWS STS service to get access and secret key, when i pass this in AWSCredentials file, the connection is failing with error security token invalid.

It seem STS functionality is missing from ycsb to connect to dynamoDB.
Also in AWSCredentials file, only access and secret variables are given to pass the value but there is not option to pass security token which is kind of mandatory in our aws environment.

[billkarwin]

I had the same issue with session security token not being recognized. I was able to get it to work with a minor code modification.

In dynamodb/src/main/java/site/ycsb/db/DynamoDBClient.java delete five lines:

22,23d21
< import com.amazonaws.auth.AWSStaticCredentialsProvider;
< import com.amazonaws.auth.PropertiesCredentials;
32d29
< import java.io.File;
84d80
< String credentialsFile = getProperties().getProperty("dynamodb.awsCredentialsFile", null);
147d142
< .withCredentials(new AWSStaticCredentialsProvider(new PropertiesCredentials(new File(credentialsFile))))

With those lines deleted, AWS client defaults to inspect the credentials chain, so it will look for credentials in environment variables or ~/.aws/credentials or instance metadata. This modification doesn't allow me to specify an AWS profile, only to use the default profile. But that was enough for me.

Also with this modification, there's no need to specify the AWS credential file in the ycsb properties.

[nyh]

I have the same problem. The AWS SDK's PropertiesCredentials, which YCSB uses, for some unknown reason (I would consider it a bug in the SDK...) only allows a key and secret, and not the third "session token" needed for STS-returned credentials.

[daljeetsingh23]

you need too make chhangs in how is authenticated.

…

On Tue, Sep 6, 2022 at 7:26 PM nyh ***@***.***> wrote: I have the same problem. The AWS SDK's PropertiesCredentials, which YCSB uses, for some unknown reason (I would consider it a bug in the SDK...) only allows a key and secret, and not the third "session token" needed for STS-returned credentials. — Reply to this email directly, view it on GitHub <#1496 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMLVIWVEKLRYXJUEBN3VS7LV45ER5ANCNFSM4U6FPIZQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nyh]

@billkarwin's patch above works perfectly! Instead of a silly weirdly-formatted and half-featured "credential property file" and a YCSB option dynamodb.awsCredentialsFile pointing to it, it lets you use the normal AWS configuration file (e.g., $HOME/.aws/credentials, or environment variables) - and those have the full capabilities of AWS credentials (e.g., STS's aws_session_token).

What would be the appropriate process to get this patch (probably with the corresponding documentation patch) in? Should I (or @billkarwin) send a pull request? Do we care about breaking backward compatibility (i.e., the dynamodb.awsCredentialsFile option will stop working)?

[fruch]

@nyh @billkarwin

Seems like this PR also address this issue (and do few other things):
#1624

It also remove the credentials file property, I don't think anyone using AWS would actually welcome such a change.

[fee-mendes]

Bump - still hurts!