Where to put aws_session_token information

[AcidLeroy]

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.

• Version of s3fs being used (s3fs --version): Amazon Simple Storage Service File System V1.82(commit:f4515b5) with OpenSSL

• Version of fuse being used (pkg-config --modversion fuse): 2.9.2

• System information (uname -a): Linux 3.10.0-693.el7.x86_64 Using %20 instead of the plus (+) sign for encoding spaces #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

• Distro (cat /etc/issue): Red Hat Enterprise Linux Workstation release 7.4 (Maipo)

Details about issue

We currently use temporary credentials using aws sts, from the documentation it is not clear to me how or where I would put the aws_session_token. Without setting this, I cannot access aws s3 buckets (i.e. I have to refresh this credential each day to gain access to the s3 buckets again). Can someone please point me to where I need to set this value in s3fs? Thanks!

[ifnull]

Ultimately it needs to be passed in the request header as x-amz-security-token. Which seems to be set in S3fsCurl::AWSAccessToken. I attempted to add support by adding the AWS SessionToken to /etc/passwd-s3fs as a key/value pair but found that it was going to be more work that just adding it to parse_passwd_file(). I don’t really know C++ so I’m not having much luck. This would be a great feature if someone want to add it. For now, I'm going to have to use aws s3 sync and keep a cached copy of everything locally.

Using Temporary Security Credentials to Request Access to AWS Resources
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html

[gaul]

Please test with -o use_session_token and reopen if this does not address your use case.

[ekarmazin]

s3fs gaves me:
fuse: unknown option `use_session_token'

[gaul]

Which version did you test with? No release version yet supports this; you have to compile from master.

[ekarmazin]

Oh ok, I've used 1.85, let me try master and get back to here.

[ekarmazin]

s3fs built from master branch, see latest commit:

Amazon Simple Storage Service File System V1.85 (commit:80972aa) with OpenSSL
Copyright (C) 2010 Randy Rizun rrizun@gmail.com
License GPL2: GNU GPL version 2 https://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

s3fs error:

[CRT] s3fs.cpp:set_s3fs_log_level(297): change debug level from [CRT] to [INF]
[INF] s3fs.cpp:set_mountpoint_attribute(4379): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
fuse: unknown option `use_session_token'

[gaul]

I believe use_session_token goes in $HOME/.aws/credentials.

[ekarmazin]

No, ~/.aws/credentials has different naming:

aws_access_key_id = KEY_GOES_HERE
aws_secret_access_key = SECRET_GOES_HERE
aws_session_token = TOKEN_GOES_HERE

I've tried to pass 'aws_session_token' in s3fs options but no luck, same error: fuse: unknown option `aws_session_token'

weird, but I see that feature in cpp listing:

if(profile != aws_profile){
return EXIT_FAILURE;
}
if (session_token.empty()) {
if (is_use_session_token) {
S3FS_PRN_EXIT("AWS session token was expected but wasn't provided in aws/credentials file for profile: %s.", aws_profile.c_str());
return EXIT_FAILURE;
}

But I am getting an error about 'no such option' instead of missing token or something.

[gaul]

Sorry, $HOME/.aws/credentials should contain aws_session_token and the command-line flags should contain -o use_session_token.

[ekarmazin]

Thats what I did, and got error not about incorrect or missing token but:

fuse: unknown option `use_session_token'

while specifying cli flag for s3fs built form master branch.

[gaul]

How exactly do you involve s3fs? I see the following:

$ src/s3fs BUCKET PATH -f -o use_session_token
s3fs: AWS session token was expected but wasn't provided in aws/credentials file for profile: default.
s3fs: could not determine how to establish security credentials.

[juliogonzalez]

Wild suggestion:

@ekarmazin keep in mind that if you are using /etc/fstab and you have both 1.85 and master build installed at the same time, most probably the system is using s3fs 1.85 and not master.

[ekarmazin]

I am using docker containers and each build is a fresh one, so no version overlapping.

Here is my configs and how I run s3fs (git clone and make were done at /tmp):

/tmp/s3fs-fuse/src/s3fs MY_BUCKET_HERE /mnt -o use_session_token -o allow_other -o use_cache=/tmp -o profile="PROFILE_FORM_AWS_CREDS" -o use_path_request_style -o dbglevel=info -f -o curldbg

cat ~/.aws/credentials
[ACCOUNT-PROFILENAME]
aws_access_key_id     = ASOMEKEYHERE723B
aws_secret_access_key = wxENSOMESECURITYKEYHEREhw67dQW
aws_session_token     = FQoGLONGLONGTOKENHEREhTkFYNEYt

See commit #, the latest from master, so it was build from master branch:

s3fs --version
Amazon Simple Storage Service File System V1.85 (commit:80972aa) with OpenSSL
Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Error I am getting:

[CRT] s3fs.cpp:set_s3fs_log_level(297): change debug level from [CRT] to [INF]
[INF]     s3fs.cpp:set_mountpoint_attribute(4379): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
fuse: unknown option `use_session_token'

Point me, what I am doing wrong. Here is how I compiled from src:

cd /tmp && \
git clone https://github.com/s3fs-fuse/s3fs-fuse.git && \
cd s3fs-fuse && \
./autogen.sh && \
./configure  && \
make && \
make install

fuse version: 2.9.x

[saurabhdas]

Can this issue be re-opened? I'm facing the same problem: I get:

fuse: unknown option `use_session_token'

when I invoke s3fs with -o use_session_token.

[gaul]

Which version of s3fs do you use? This option only exists in master, not 1.85.

[saurabhdas]

For the s3fs installation, I am following the same procedure as @ekarmazin – compiling from source.

I later discovered that doing this works:

export AWSACCESSKEYID=${AWS_ACCESS_KEY_ID}
export AWSSECRETACCESSKEY=${AWS_SECRET_ACCESS_KEY}
export AWSSESSIONTOKEN=${AWS_SESSION_TOKEN}
s3fs -o retries=20 $S3_BUCKET $MOUNT_PATH

The -o use_session_token option causes a failure as documented above. Perhaps this is correct/intentional, however something in the documentation led me to believe that I needed to use -o use_session_token to enable this behaviour.

[wwhurley]

I'm having the same issue. I'm pulling master from the repo, using Amazon Simple Storage Service File System V1.85 (commit:e0712f4) with OpenSSL with Fuse 2.9.7 and when running the command s3fs [BUCKET]:[PATH] [MOUNT] -f -o nonempty -o use_session_token I'm getting the error fuse: unknown option use_session_token'`

[gaul]

Could someone test with the latest master which includes a flag parsing fix?

[llitz]

Working fine for me with c58c91f

[tiilikainen]

Still does not work for me. I'm using the brew install version which bills itself as 1.87 with FUSE 2.9.7. However, --version displays commit:unknown.

[gaul]

@tiilikainen You will have to compile from master until 1.88 is released.