Custom STS endpoint configuration to allow connecting to S3 from isolated EC2 subnet

[mkfmnn]

Describe the bug
I am setting up an environment where users use Windows EC2 instances that are configured without open-ended internet access, as described here: https://repost.aws/knowledge-center/ec2-systems-manager-vpc-endpoints

I want to use Cyberduck on those instances to enable users to easily transfer files to and from S3 (via VPC endpoint) through a graphical interface; without internet access there's no AWS Console.

However, Cyberduck complains that it can't reach sts.amazonaws.com, which is hard-coded here. There's no way for me to expose that endpoint to a private VPC; only the regional STS endpoints can have a VPC endpoint configured.

It seems like one of two changes (or both) would need to be made to unblock my use of Cyberduck for this use case:

1. Make Cyberduck use the appropriate regional sts endpoint, based on the detected or configured S3 region.
2. Make the STS dependency optional (by disabling it in configuration, or by not throwing an exception if it fails) or remove it entirely. It's not entirely clear to me from my initial glance of the source code why it's there, and I can save $20/month/subnet for each VPC endpoint I can get rid of, so personally I'd prefer not to have to depend on STS.

cyberduck.log

To Reproduce
Steps to reproduce the behavior:

1. Set up a Windows EC2 instance in a VPC & subnet in us-east-1 that does not have an internet gateway but that has EC2 instance endpoints for S3, SSM, and STS, and an EC2 instance profile that supports accessing S3 and SSM.
2. Configure Cyberduck with an "S3 (Credentials from Instance Metadata)" profile/bookmark.
3. Attempt to connect to s3.us-east-1.amazonaws.com
4. Observe error: Unable to execute HTTP request: Connect to sts.amazonaws.com:443 [sts.amazonaws.com/67.220.251.181] failed: Connection timed out: connect

Obviously these reproduction steps are non-trivial. If it is helpful to you, I can try to throw together a CloudFormation template and/or AWS CLI shell script that does the environment setup.

Alternately, if you have some sort of firewall that can block outbound connections to sts.amazonaws.com, you can probably reproduce it that way.

Expected behavior
When using the bookmark, Cyberduck should show a list of buckets in my account instead of a connection error.

Screenshots
N/A

Desktop (please complete the following information):

• OS: Windows Server 2022 10.0.20348.0 (amd64)
• Cyberduck v9.4.1.44384

Log Files
Relevant log file section attached. Let me know if more context is needed.

Snippet:

Caused by: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to sts.amazonaws.com:443 [sts.amazonaws.com/67.220.251.181] failed: Connection timed out: connect
	...
	at com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.getCallerIdentity(AWSSecurityTokenServiceClient.java:1266)
	at ch.cyberduck.core.sts.STSAuthorizationService.getCallerIdentity(STSAuthorizationService.java:95)
	at ch.cyberduck.core.s3.S3Session.login(S3Session.java:328)
	at ch.cyberduck.core.KeychainLoginService.authenticate(KeychainLoginService.java:182)
	at ch.cyberduck.core.LoginConnectionService.authenticate(LoginConnectionService.java:169)
	at ch.cyberduck.core.LoginConnectionService.connect(LoginConnectionService.java:160)
	at ch.cyberduck.core.LoginConnectionService.check(LoginConnectionService.java:105)
	at ch.cyberduck.core.pool.StatelessSessionPool.borrow(StatelessSessionPool.java:57)
	...

Additional context
N/A

[dkocher]

You should be able to workaround this issue with a custom connection profile that sets the STS Endpoint key to the regional endpoint.

1. Take the existing profile as a template and change the Vendor key.
2. Add the STS Endpoint configuration

        <key>STS Endpoint</key>
        <string>https://sts.us-west-2.amazonaws.com</string>