Skip to content

Add options to force S3 address style.#1249

Merged
daviesrob merged 2 commits intosamtools:developfrom
whitwham:path_style_s3
Mar 4, 2021
Merged

Add options to force S3 address style.#1249
daviesrob merged 2 commits intosamtools:developfrom
whitwham:path_style_s3

Conversation

@whitwham
Copy link
Member

@whitwham whitwham commented Mar 4, 2021

There are now three ways to force the URL style in the S3 plugin. By setting addressing_style in the aws credentials file, by setting host_bucket in .s3cfg to anything not containing %(bucket).s (this is how s3cmd works) or by setting the HTS_S3_ADDRESS_STYLE env variable to path.

Should fix samtools/samtools#1377.

hfile_s3.c Outdated
"aws_session_token", &ad->token,
"region", &ad->region, NULL);
"region", &ad->region,
"addressing_style", &url_style, NULL);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You or I should have put this NULL on the next line — so extra entries can be added more clearly — ages ago 😄 (and similarly in the other parse_ini call below).

Suggested change
"addressing_style", &url_style, NULL);
"addressing_style", &url_style,
NULL);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted (will squash on merge).

@daviesrob
Copy link
Member

Thanks, I've tested the various configuration options and they all behave as expected.

Will merge shortly...

@daviesrob daviesrob merged commit 1949a15 into samtools:develop Mar 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3 file access fails with long paths

3 participants

Comments