Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow connection profiles to use custom STS and OIDC endpoints #14972

Merged
merged 84 commits into from Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
bc01679
Connect to Minio with temporary credentials from STS obtained with Op…
dkocher Aug 6, 2023
e3b46c3
Review tests.
dkocher Aug 6, 2023
fb90c10
Fix null pointer.
dkocher Aug 6, 2023
7ae0a45
Set "Accept" header to request JSON response for interoperability wit…
dkocher Aug 6, 2023
c209bcd
Set default STS endpoint.
dkocher Aug 6, 2023
221bad0
Set token in request header from configured credentials depending on …
dkocher Aug 7, 2023
da81dff
Delete unused accessor.
dkocher Aug 7, 2023
99537e1
Refactor OAUth and STS flow to allow reuse of interceptors.
dkocher Aug 7, 2023
049e6b0
Delete bundled key.
dkocher Aug 7, 2023
7490c15
Must only handle specific 400 error codes.
dkocher Aug 7, 2023
d81acca
Logging.
dkocher Aug 7, 2023
918b2a5
Fix retrieving Id (OIDC) token.
dkocher Aug 7, 2023
0330165
Make sure to fetch tokens from instance metadata when expired when us…
dkocher Aug 7, 2023
8dfa552
Extract authentication handlers classes.
dkocher Aug 7, 2023
220916b
Skip x-amz-security-token header set in general handler.
dkocher Aug 8, 2023
e3c5167
Review registration of handlers depending on setup from AWS CLI.
dkocher Aug 8, 2023
4f2c2c8
Allow failure when reloading credentials configuration.
dkocher Aug 8, 2023
ef567c1
Reuse interceptor to provide login credentials from AWS CLI config.
dkocher Aug 8, 2023
6c9a54e
Use configured directory.
dkocher Aug 8, 2023
23d02f3
Fix #10917.
dkocher Aug 8, 2023
2b1669f
Set subject from JWT claims as role session name by default.
dkocher Aug 8, 2023
0b78be9
Move class.
dkocher Aug 8, 2023
b55ec15
Move and rename class.
dkocher Aug 8, 2023
85891c6
Review.
dkocher Aug 8, 2023
1cd8631
Add support for AssumeRoleWithSAML.
dkocher Aug 8, 2023
3ca574e
Fix sts:GetSessionToken to include MFA token.
dkocher Aug 8, 2023
fb59e7a
Delete unused profile.
dkocher Aug 9, 2023
4249129
Add and review tests.
dkocher Aug 9, 2023
50e3e1d
Logging.
dkocher Aug 9, 2023
1621350
Extract variable.
dkocher Aug 9, 2023
174a632
Rename field.
dkocher Aug 9, 2023
c452ca3
Fix deprecated usage.
dkocher Aug 9, 2023
03b18a7
Use "Try Again" as default button title on alert to retry after OAuth…
dkocher Aug 10, 2023
0051e17
Add mapping for `TokenRefreshRequired`.
dkocher Aug 10, 2023
16e6cdc
Review.
dkocher Aug 10, 2023
d4bc80b
Extract method to allow refresh from subclass on different error code.
dkocher Aug 10, 2023
b35f633
Only reset provider credentials after refresh.
dkocher Aug 10, 2023
a315db1
Review implementation.
dkocher Aug 10, 2023
ef14ea1
Slimmed keycloak-realm.json.
chenkins Aug 11, 2023
d3bd6e1
Throw failure on missing or invalid JWT in Id token.
dkocher Aug 11, 2023
0db1815
Fix typo.
dkocher Aug 11, 2023
62b0c22
Remove Role ARN configuration not required.
dkocher Aug 11, 2023
f6c68a2
Delete obsolete environment passed as parameter below.
dkocher Aug 11, 2023
99fa10e
Add missing login prompt parameter.
dkocher Aug 11, 2023
cb0d603
Pull out case for anonymous credentials.
dkocher Aug 11, 2023
79aba79
Log credentials format.
dkocher Aug 11, 2023
0d5bdbd
Logging.
dkocher Aug 11, 2023
d93d5a1
Logging.
dkocher Aug 11, 2023
5f38795
Prompt for Role ARN when defined as empty value in connection profile.
dkocher Aug 11, 2023
f093297
Review tests.
dkocher Aug 11, 2023
0794fab
Allow explicit empty definition of client secret in profile to disabl…
dkocher Aug 11, 2023
74c170d
Rename profile.
dkocher Aug 11, 2023
165de98
Rename base test class.
dkocher Aug 11, 2023
1cd797d
Add handling for response headers from Minio for HEAD requests with n…
dkocher Aug 11, 2023
8df1814
Missing int in preferences defaults to -1.
dkocher Aug 11, 2023
f4655a7
Do not set empty values in defaults.
dkocher Aug 11, 2023
5441361
Mark as token configurable to ensure AWS Session Token is saved and r…
dkocher Aug 12, 2023
c5dccba
Add mapper for STS API failures. Map to `ExpiredTokenException` when …
dkocher Aug 12, 2023
9d9cb54
Explicitly skip validation for OAuth.
dkocher Aug 12, 2023
de95cc5
Logging.
dkocher Aug 12, 2023
fee11df
Explict field for STS tokens in credentials used by configurator and …
dkocher Aug 12, 2023
57ea077
Simplify retry handling. Refresh OAuth token when STS request fails b…
dkocher Aug 12, 2023
293741e
Set subject from JWT token as username to make sure OAuth tokens can …
dkocher Aug 12, 2023
11b79b0
Inject OAuth TTL into TestContainer tests.
chenkins Aug 12, 2023
1479ace
Implement equals.
dkocher Aug 12, 2023
6516800
Review tests.
dkocher Aug 12, 2023
eb35e00
Temporary credentials are not saved.
dkocher Aug 12, 2023
e2baf39
Add abstraction for authentication strategy.
dkocher Aug 12, 2023
c157b42
Review.
dkocher Aug 13, 2023
81a8f3b
Review.
dkocher Aug 13, 2023
25a7576
Fix test.
dkocher Aug 13, 2023
a2eec98
Merge implementations retrieving latest credentials on error response.
dkocher Aug 13, 2023
3f7f77d
Fix test.
dkocher Aug 14, 2023
1913156
Logging.
dkocher Aug 14, 2023
03d9bca
Review error mapping to handle error codes.
dkocher Aug 14, 2023
44f7d7f
Add tests.
dkocher Aug 14, 2023
5ccac05
Refactor flows.
dkocher Aug 15, 2023
2bfb964
Delete long running tests.
dkocher Aug 15, 2023
72f2829
Bugfix STS testcontainer tests (BeforeClass annotation in abstract te…
chenkins Aug 15, 2023
595c12e
Review test setup.
dkocher Aug 18, 2023
7e572ca
Formatting.
dkocher Aug 21, 2023
8150cf5
Review hash code.
dkocher Aug 21, 2023
bdfdcf0
Remove obsolete keycloak.json manipulation before running AssumeRoleW…
chenkins Aug 22, 2023
a999e86
Merge token ivar.
dkocher Aug 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 8 additions & 4 deletions .github/workflows/build.yml
Expand Up @@ -19,12 +19,16 @@ jobs:
with:
distribution: 'temurin'
java-version: 17
- name: Disable Testcontainers for Windows and MacOS
run: echo "args=-P=no-testcontainers" >> "$GITHUB_ENV"
shell: bash
if: runner.os == 'windows' || runner.os == 'macos'
- run: choco install bonjour
if: runner.os == 'Windows'
if: runner.os == 'windows'
- uses: ilammy/msvc-dev-cmd@v1
if: runner.os == 'Windows'
if: runner.os == 'windows'
- uses: microsoft/setup-msbuild@v1.3
if: runner.os == 'Windows'
if: runner.os == 'windows'
with:
msbuild-architecture: x64
- name: Cache local Maven repository
Expand All @@ -35,4 +39,4 @@ jobs:
restore-keys: |
${{ runner.os }}-maven-
- name: Build with Maven
run: mvn --no-transfer-progress verify -DskipITs -DskipSign --batch-mode -Drevision=0
run: mvn --no-transfer-progress verify -DskipITs -DskipSign ${{ env.args }} --batch-mode -Drevision=0
5 changes: 2 additions & 3 deletions box/src/main/java/ch/cyberduck/core/box/BoxSession.java
Expand Up @@ -71,15 +71,14 @@ public CloseableHttpClient connect(final Proxy proxy, final HostKeyCallback key,
authorizationService = new OAuth2RequestInterceptor(configuration.build(), host, prompt)
.withRedirectUri(host.getProtocol().getOAuthRedirectUrl());
configuration.addInterceptorLast(authorizationService);
configuration.setServiceUnavailableRetryStrategy(new OAuth2ErrorResponseInterceptor(host, authorizationService, prompt));
configuration.setServiceUnavailableRetryStrategy(new OAuth2ErrorResponseInterceptor(host, authorizationService));
return configuration.build();
}

@Override
public void login(final Proxy proxy, final LoginCallback prompt, final CancelCallback cancel) throws BackgroundException {
authorizationService.authorize(host, prompt, cancel);
try {
final Credentials credentials = host.getCredentials();
final Credentials credentials = authorizationService.validate();
credentials.setUsername(new UsersApi(new BoxApiClient(client)).getUsersMe(Collections.emptyList()).getLogin());
}
catch(ApiException e) {
Expand Down
Expand Up @@ -19,6 +19,7 @@
import ch.cyberduck.core.Credentials;
import ch.cyberduck.core.CredentialsConfigurator;
import ch.cyberduck.core.Host;
import ch.cyberduck.core.exception.LoginCanceledException;

import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.LogManager;
Expand All @@ -41,7 +42,7 @@ public Credentials configure(final Host host) {
}

@Override
public CredentialsConfigurator reload() {
public CredentialsConfigurator reload() throws LoginCanceledException {
return this;
}
}
9 changes: 9 additions & 0 deletions core/src/main/java/ch/cyberduck/core/AbstractProtocol.java
Expand Up @@ -211,6 +211,11 @@ public boolean isOAuthPKCE() {
return true;
}

@Override
public String getSTSEndpoint() {
return null;
}

@Override
public String getDefaultHostname() {
// Blank by default
Expand Down Expand Up @@ -316,6 +321,10 @@ public boolean validate(final Credentials credentials, final LoginOptions option
return StringUtils.isNotBlank(credentials.getPassword());
}
}
if(options.oauth) {
// Always refresh tokens in login
return true;
}
if(options.token) {
return StringUtils.isNotBlank(credentials.getToken());
}
Expand Down
51 changes: 32 additions & 19 deletions core/src/main/java/ch/cyberduck/core/Credentials.java
Expand Up @@ -38,7 +38,7 @@ public class Credentials implements Comparable<Credentials> {
* The login password
*/
private String password = StringUtils.EMPTY;
private String token = StringUtils.EMPTY;
private TemporaryAccessTokens tokens = TemporaryAccessTokens.EMPTY;
private OAuthTokens oauth = OAuthTokens.EMPTY;

/**
Expand Down Expand Up @@ -72,7 +72,7 @@ public Credentials() {
public Credentials(final Credentials copy) {
this.user = copy.user;
this.password = copy.password;
this.token = copy.token;
this.tokens = copy.tokens;
dkocher marked this conversation as resolved.
Show resolved Hide resolved
this.oauth = copy.oauth;
this.identity = copy.identity;
this.identityPassphrase = copy.identityPassphrase;
Expand All @@ -97,7 +97,7 @@ public Credentials(final String user, final String password) {
public Credentials(final String user, final String password, final String token) {
this.user = user;
this.password = password;
this.token = token;
this.tokens = new TemporaryAccessTokens(token);
}

/**
Expand All @@ -113,8 +113,7 @@ public void setUsername(final String user) {
}

public Credentials withUsername(final String user) {
this.user = user;
this.passed = false;
this.setUsername((user));
return this;
}

Expand All @@ -136,23 +135,35 @@ public void setPassword(final String password) {
}

public Credentials withPassword(final String password) {
this.password = password;
this.passed = false;
this.setPassword(password);
return this;
}

public String getToken() {
return token;
return tokens.getSessionToken();
}

public void setToken(final String token) {
this.token = token;
this.tokens = new TemporaryAccessTokens(token);
this.passed = false;
}

public Credentials withToken(final String token) {
this.token = token;
this.setToken(token);
return this;
}

public TemporaryAccessTokens getTokens() {
return tokens;
}

public void setTokens(final TemporaryAccessTokens tokens) {
this.tokens = tokens;
this.passed = false;
}

public Credentials withTokens(final TemporaryAccessTokens tokens) {
this.setTokens(tokens);
return this;
}

Expand All @@ -165,7 +176,7 @@ public void setOauth(final OAuthTokens oauth) {
}

public Credentials withOauth(final OAuthTokens oauth) {
this.oauth = oauth;
this.setOauth(oauth);
return this;
}

Expand All @@ -186,7 +197,7 @@ public void setSaved(final boolean saved) {
}

public Credentials withSaved(final boolean saved) {
this.saved = saved;
this.setSaved(saved);
return this;
}

Expand All @@ -213,7 +224,7 @@ public boolean isPasswordAuthentication() {
}

public boolean isTokenAuthentication() {
return StringUtils.isNotBlank(token);
return StringUtils.isNotBlank(tokens.getSessionToken());
}

public boolean isOAuthAuthentication() {
Expand Down Expand Up @@ -300,6 +311,7 @@ public boolean validate(final Protocol protocol, final LoginOptions options) {
public void reset() {
this.setPassword(StringUtils.EMPTY);
this.setToken(StringUtils.EMPTY);
this.setTokens(TemporaryAccessTokens.EMPTY);
this.setOauth(OAuthTokens.EMPTY);
this.setIdentityPassphrase(StringUtils.EMPTY);
}
Expand Down Expand Up @@ -328,24 +340,25 @@ public boolean equals(final Object o) {
}
final Credentials that = (Credentials) o;
return Objects.equals(user, that.user) &&
Objects.equals(password, that.password) &&
Objects.equals(token, that.token) &&
Objects.equals(identity, that.identity) &&
Objects.equals(certificate, that.certificate);
Objects.equals(password, that.password) &&
Objects.equals(tokens, that.tokens) &&
Objects.equals(oauth, that.oauth) &&
Objects.equals(identity, that.identity) &&
Objects.equals(certificate, that.certificate);
}

@Override
public int hashCode() {
return Objects.hash(user, password, token, identity, certificate);
return Objects.hash(user, password, tokens, oauth, identity, certificate);
}

@Override
public String toString() {
final StringBuilder sb = new StringBuilder("Credentials{");
sb.append("user='").append(user).append('\'');
sb.append(", password='").append(StringUtils.repeat("*", Integer.min(8, StringUtils.length(password)))).append('\'');
sb.append(", tokens='").append(tokens).append('\'');
sb.append(", oauth='").append(oauth).append('\'');
sb.append(", token='").append(StringUtils.repeat("*", Integer.min(8, StringUtils.length(token)))).append('\'');
sb.append(", identity=").append(identity);
sb.append('}');
return sb.toString();
Expand Down
Expand Up @@ -18,6 +18,8 @@
* dkocher@cyberduck.ch
*/

import ch.cyberduck.core.exception.LoginCanceledException;

public interface CredentialsConfigurator {

/**
Expand All @@ -27,7 +29,7 @@ public interface CredentialsConfigurator {
*/
Credentials configure(Host host);

CredentialsConfigurator reload();
CredentialsConfigurator reload() throws LoginCanceledException;

CredentialsConfigurator DISABLED = new CredentialsConfigurator() {
@Override
Expand Down
Expand Up @@ -18,6 +18,8 @@
* dkocher@cyberduck.ch
*/

import ch.cyberduck.core.exception.LoginCanceledException;

public final class CredentialsConfiguratorFactory {

private CredentialsConfiguratorFactory() {
Expand All @@ -29,6 +31,11 @@ private CredentialsConfiguratorFactory() {
* @return Configurator for default settings
*/
public static CredentialsConfigurator get(final Protocol protocol) {
return protocol.getCredentialsFinder().reload();
try {
return protocol.getCredentialsFinder().reload();
}
catch(LoginCanceledException e) {
return CredentialsConfigurator.DISABLED;
}
}
}
Expand Up @@ -18,8 +18,6 @@
*/

import ch.cyberduck.core.exception.LocalAccessDeniedException;
import ch.cyberduck.core.preferences.Preferences;
import ch.cyberduck.core.preferences.PreferencesFactory;

import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.LogManager;
Expand All @@ -30,8 +28,6 @@
public abstract class DefaultHostPasswordStore implements HostPasswordStore {
private static final Logger log = LogManager.getLogger(DefaultHostPasswordStore.class);

private final Preferences preferences = PreferencesFactory.get();

/**
* Find password for login
*
Expand Down Expand Up @@ -161,7 +157,9 @@ public OAuthTokens findOAuthTokens(final Host bookmark) {
String.format("%s OAuth2 Access Token", prefix)),
this.getPassword(bookmark.getProtocol().getScheme(), bookmark.getPort(), hostname,
String.format("%s OAuth2 Refresh Token", prefix)),
expiry != null ? Long.parseLong(expiry) : -1L));
expiry != null ? Long.parseLong(expiry) : -1L,
this.getPassword(bookmark.getProtocol().getScheme(), bookmark.getPort(), hostname,
String.format("%s OIDC Id Token", prefix))));
}
catch(LocalAccessDeniedException e) {
log.warn(String.format("Failure %s searching in keychain", e));
Expand Down Expand Up @@ -232,6 +230,11 @@ public void save(final Host bookmark) throws LocalAccessDeniedException {
this.addPassword(this.getOAuthHostname(bookmark), String.format("%s OAuth2 Token Expiry", prefix),
String.valueOf(credentials.getOauth().getExpiryInMilliseconds()));
}
if(StringUtils.isNotBlank(credentials.getOauth().getIdToken())) {
this.addPassword(bookmark.getProtocol().getScheme(),
bookmark.getPort(), this.getOAuthHostname(bookmark),
String.format("%s OIDC Id Token", prefix), credentials.getOauth().getIdToken());
}
}
}

Expand Down Expand Up @@ -272,6 +275,10 @@ public void delete(final Host bookmark) throws LocalAccessDeniedException {
if(credentials.getOauth().getExpiryInMilliseconds() != null) {
this.deletePassword(this.getOAuthHostname(bookmark), String.format("%s OAuth2 Token Expiry", prefix));
}
if(StringUtils.isNotBlank(credentials.getOauth().getIdToken())) {
this.deletePassword(protocol.getScheme(), bookmark.getPort(), this.getOAuthHostname(bookmark),
String.format("%s OIDC Id Token", prefix));
}
}
}
}
Expand Up @@ -18,6 +18,8 @@
* dkocher@cyberduck.ch
*/

import ch.cyberduck.core.exception.LoginCanceledException;

public final class JumpHostConfiguratorFactory {

private JumpHostConfiguratorFactory() {
Expand All @@ -29,6 +31,11 @@ private JumpHostConfiguratorFactory() {
* @return Configurator for default settings
*/
public static JumphostConfigurator get(final Protocol protocol) {
return protocol.getJumpHostFinder().reload();
try {
return protocol.getJumpHostFinder().reload();
}
catch(LoginCanceledException e) {
return JumphostConfigurator.DISABLED;
}
}
}
Expand Up @@ -18,11 +18,13 @@
* dkocher@cyberduck.ch
*/

import ch.cyberduck.core.exception.LoginCanceledException;

public interface JumphostConfigurator {

Host getJumphost(String alias);

JumphostConfigurator reload();
JumphostConfigurator reload() throws LoginCanceledException;

JumphostConfigurator DISABLED = new JumphostConfigurator() {
@Override
Expand Down
Expand Up @@ -160,7 +160,7 @@ public boolean prompt(final Host bookmark, final String message, final LoginCall
}
if(options.oauth) {
prompt.warn(bookmark, LocaleFactory.localizedString("Login failed", "Credentials"), message,
LocaleFactory.localizedString("Continue", "Credentials"),
LocaleFactory.localizedString("Try Again", "Alert"),
LocaleFactory.localizedString("Cancel", "Localizable"), null);
log.warn(String.format("Reset OAuth tokens for %s", bookmark));
credentials.setOauth(OAuthTokens.EMPTY);
Expand Down