Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: also check DOCKER_AUTH_CONFIG for registry auth config as an alternative to config.json #6238

Merged
merged 9 commits into from
Feb 18, 2023
Merged

feat: also check DOCKER_AUTH_CONFIG for registry auth config as an alternative to config.json #6238

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a91320b
feat: also check DOCKER_AUTH_CONFIG for registry auth config as an al…
Nov 25, 2022
de9fec6
Merge branch 'main' into docker_registry_auth
roseo1 Nov 29, 2022
9540fcc
Merge branch 'main' into docker_registry_auth
roseo1 Nov 30, 2022
b128785
fix formatting
Nov 30, 2022
d2e8927
Polish
eddumelendez Dec 14, 2022
94b0f30
Merge branch 'main' into docker_registry_auth
roseo1 Dec 15, 2022
9d7a70f
Merge branch 'main' into docker_registry_auth
eddumelendez Feb 18, 2023
1d6949b
Fix order: read env var first then config file
eddumelendez Feb 18, 2023
becfe9a
Fix docs
eddumelendez Feb 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
47 changes: 38 additions & 9 deletions core/src/main/java/org/testcontainers/utility/RegistryAuthLocator.java
View file
Open in desktop
Expand Up @@ -2,6 +2,7 @@

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.dockerjava.api.exception.NotFoundException;
import com.github.dockerjava.api.model.AuthConfig;
import com.google.common.annotations.VisibleForTesting;
import org.apache.commons.lang3.StringUtils;
Expand Down Expand Up @@ -33,6 +34,8 @@ public class RegistryAuthLocator {

private static final String DEFAULT_REGISTRY_NAME = "https://index.docker.io/v1/";

private static final String DOCKER_AUTH_ENV_VAR = "DOCKER_AUTH_CONFIG";

private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();

private static RegistryAuthLocator instance;
Expand All @@ -43,6 +46,8 @@ public class RegistryAuthLocator {

private final File configFile;

private final String configEnv;

private final Map<String, Optional<AuthConfig>> cache = new ConcurrentHashMap<>();

/**
Expand All @@ -54,11 +59,13 @@ public class RegistryAuthLocator {
@VisibleForTesting
RegistryAuthLocator(
File configFile,
String configEnv,
String commandPathPrefix,
String commandExtension,
Map<String, String> notFoundMessageHolderReference
) {
this.configFile = configFile;
this.configEnv = configEnv;
this.commandPathPrefix = commandPathPrefix;
this.commandExtension = commandExtension;

Expand All @@ -72,6 +79,7 @@ protected RegistryAuthLocator() {
.getenv()
.getOrDefault("DOCKER_CONFIG", System.getProperty("user.home") + "/.docker");
this.configFile = new File(dockerConfigLocation + "/config.json");
this.configEnv = System.getenv(DOCKER_AUTH_ENV_VAR);
this.commandPathPrefix = "";
this.commandExtension = "";

Expand Down Expand Up @@ -131,15 +139,8 @@ public AuthConfig lookupAuthConfig(DockerImageName dockerImageName, AuthConfig d
}

private Optional<AuthConfig> lookupUncachedAuthConfig(String registryName, DockerImageName dockerImageName) {
log.debug(
"RegistryAuthLocator has configFile: {} ({}) and commandPathPrefix: {}",
configFile,
configFile.exists() ? "exists" : "does not exist",
commandPathPrefix
);

try {
final JsonNode config = OBJECT_MAPPER.readTree(configFile);
final JsonNode config = getDockerAuthConfig();
log.debug("registryName [{}] for dockerImageName [{}]", registryName, dockerImageName);

// use helper preferentially (per https://docs.docker.com/engine/reference/commandline/cli/)
Expand All @@ -162,15 +163,43 @@ private Optional<AuthConfig> lookupUncachedAuthConfig(String registryName, Docke
}
} catch (Exception e) {
log.info(
"Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: {}, configFile: {}. Falling back to docker-java default behaviour. Exception message: {}",
"Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: {}, configFile: {}, configEnv: {}). Falling back to docker-java default behaviour. Exception message: {}",
dockerImageName,
configFile,
DOCKER_AUTH_ENV_VAR,
e.getMessage()
);
}
return Optional.empty();
}

private JsonNode getDockerAuthConfig() throws Exception {
log.debug(
"RegistryAuthLocator has configFile: {} ({}) configEnv: {} ({}) and commandPathPrefix: {}",
configFile,
configFile.exists() ? "exists" : "does not exist",
DOCKER_AUTH_ENV_VAR,
configEnv != null ? "exists" : "does not exist",
commandPathPrefix
);

if (configFile.exists()) {
log.debug("RegistryAuthLocator reading from configFile: {}", configFile);
return OBJECT_MAPPER.readTree(configFile);
} else if (configEnv != null) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the env var be evaluated first? Usually, you can use env vars to override config files but this would not be the case here (if you have both the conf file and the env var). Maybe I'm missing something here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! yes, you're right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roseo1 can you take care of this, please? If not, LMK so I can do it before merge it.

log.debug("RegistryAuthLocator reading from environment variable: {}", DOCKER_AUTH_ENV_VAR);
return OBJECT_MAPPER.readTree(configEnv);
}

throw new NotFoundException(
"No config supplied. Checked in order: " +
configFile +
" (file not found), " +
DOCKER_AUTH_ENV_VAR +
" (not set)"
);
}

private AuthConfig findExistingAuthConfig(final JsonNode config, final String reposName) throws Exception {
final Map.Entry<String, JsonNode> entry = findAuthNode(config, reposName);

Expand Down
130 changes: 114 additions & 16 deletions core/src/test/java/org/testcontainers/utility/RegistryAuthLocatorTest.java
View file
Open in desktop
Expand Up @@ -2,12 +2,16 @@

import com.github.dockerjava.api.model.AuthConfig;
import com.google.common.io.Resources;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.SystemUtils;
import org.jetbrains.annotations.NotNull;
import org.junit.Test;

import java.io.File;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;

Expand All @@ -16,7 +20,7 @@
public class RegistryAuthLocatorTest {

@Test
public void lookupAuthConfigWithoutCredentials() throws URISyntaxException {
public void lookupAuthConfigWithoutCredentials() throws URISyntaxException, IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was IOException added to all the throws clauses of the tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because of this https://github.com/testcontainers/testcontainers-java/pull/6238/files#diff-d2e41605733b1a39c4ec3e5839fbebd71851fd71a6f3b42eab97d61f50989dd4R280

final RegistryAuthLocator authLocator = createTestAuthLocator("config-empty.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -32,7 +36,7 @@ public void lookupAuthConfigWithoutCredentials() throws URISyntaxException {
}

@Test
public void lookupAuthConfigWithBasicAuthCredentials() throws URISyntaxException {
public void lookupAuthConfigWithBasicAuthCredentials() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-basic-auth.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -48,7 +52,7 @@ public void lookupAuthConfigWithBasicAuthCredentials() throws URISyntaxException
}

@Test
public void lookupAuthConfigWithJsonKeyCredentials() throws URISyntaxException {
public void lookupAuthConfigWithJsonKeyCredentials() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-with-json-key.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -64,7 +68,7 @@ public void lookupAuthConfigWithJsonKeyCredentials() throws URISyntaxException {
}

@Test
public void lookupAuthConfigUsingStore() throws URISyntaxException {
public void lookupAuthConfigUsingStore() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-with-store.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -84,7 +88,7 @@ public void lookupAuthConfigUsingStore() throws URISyntaxException {
}

@Test
public void lookupAuthConfigUsingHelper() throws URISyntaxException {
public void lookupAuthConfigUsingHelper() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-with-helper.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -104,7 +108,7 @@ public void lookupAuthConfigUsingHelper() throws URISyntaxException {
}

@Test
public void lookupAuthConfigUsingHelperWithToken() throws URISyntaxException {
public void lookupAuthConfigUsingHelperWithToken() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-with-helper-using-token.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -118,7 +122,7 @@ public void lookupAuthConfigUsingHelperWithToken() throws URISyntaxException {
}

@Test
public void lookupUsingHelperEmptyAuth() throws URISyntaxException {
public void lookupUsingHelperEmptyAuth() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-empty-auth-with-helper.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -138,7 +142,7 @@ public void lookupUsingHelperEmptyAuth() throws URISyntaxException {
}

@Test
public void lookupNonEmptyAuthWithHelper() throws URISyntaxException {
public void lookupNonEmptyAuthWithHelper() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-existing-auth-with-helper.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
Expand All @@ -158,7 +162,7 @@ public void lookupNonEmptyAuthWithHelper() throws URISyntaxException {
}

@Test
public void lookupAuthConfigWithCredentialsNotFound() throws URISyntaxException {
public void lookupAuthConfigWithCredentialsNotFound() throws URISyntaxException, IOException {
Map<String, String> notFoundMessagesReference = new HashMap<>();
final RegistryAuthLocator authLocator = createTestAuthLocator(
"config-with-store.json",
Expand All @@ -184,7 +188,7 @@ public void lookupAuthConfigWithCredentialsNotFound() throws URISyntaxException
}

@Test
public void lookupAuthConfigWithCredStoreEmpty() throws URISyntaxException {
public void lookupAuthConfigWithCredStoreEmpty() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-with-store-empty.json");

DockerImageName dockerImageName = DockerImageName.parse("registry2.example.com/org/repo");
Expand All @@ -193,18 +197,106 @@ public void lookupAuthConfigWithCredStoreEmpty() throws URISyntaxException {
assertThat(authConfig.getAuth()).as("CredStore field will be ignored, because value is blank").isNull();
}

@Test
public void lookupAuthConfigFromEnvVarWithCredStoreEmpty() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator(null, "config-with-store-empty.json");

DockerImageName dockerImageName = DockerImageName.parse("registry2.example.com/org/repo");
final AuthConfig authConfig = authLocator.lookupAuthConfig(dockerImageName, new AuthConfig());

assertThat(authConfig.getAuth()).as("CredStore field will be ignored, because value is blank").isNull();
}

@Test
public void lookupAuthConfigWithoutConfigFile() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator(null);

final AuthConfig authConfig = authLocator.lookupAuthConfig(
DockerImageName.parse("unauthenticated.registry.org/org/repo"),
new AuthConfig()
);

assertThat(authConfig.getRegistryAddress())
.as("Default docker registry URL is set on auth config")
.isEqualTo("https://index.docker.io/v1/");
assertThat(authConfig.getUsername()).as("No username is set").isNull();
assertThat(authConfig.getPassword()).as("No password is set").isNull();
}

@Test
public void lookupAuthConfigRespectsCheckOrderPreference() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator("config-basic-auth.json", "config-empty.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
DockerImageName.parse("registry.example.com/org/repo"),
new AuthConfig()
);

assertThat(authConfig.getRegistryAddress())
.as("Default docker registry URL is set on auth config")
.isEqualTo("https://registry.example.com");
assertThat(authConfig.getUsername()).as("Username is set").isEqualTo("user");
assertThat(authConfig.getPassword()).as("Password is set").isEqualTo("pass");
}

@Test
public void lookupAuthConfigFromEnvironmentVariable() throws URISyntaxException, IOException {
final RegistryAuthLocator authLocator = createTestAuthLocator(null, "config-basic-auth.json");

final AuthConfig authConfig = authLocator.lookupAuthConfig(
DockerImageName.parse("registry.example.com/org/repo"),
new AuthConfig()
);

assertThat(authConfig.getRegistryAddress())
.as("Default docker registry URL is set on auth config")
.isEqualTo("https://registry.example.com");
assertThat(authConfig.getUsername()).as("Username is set").isEqualTo("user");
assertThat(authConfig.getPassword()).as("Password is set").isEqualTo("pass");
}

@NotNull
private RegistryAuthLocator createTestAuthLocator(String configName, String envConfigName)
throws URISyntaxException, IOException {
return createTestAuthLocator(configName, envConfigName, new HashMap<>());
}

@NotNull
private RegistryAuthLocator createTestAuthLocator(String configName) throws URISyntaxException {
return createTestAuthLocator(configName, new HashMap<>());
private RegistryAuthLocator createTestAuthLocator(String configName) throws URISyntaxException, IOException {
return createTestAuthLocator(configName, null, new HashMap<>());
}

@NotNull
private RegistryAuthLocator createTestAuthLocator(String configName, Map<String, String> notFoundMessagesReference)
throws URISyntaxException {
final File configFile = new File(Resources.getResource("auth-config/" + configName).toURI());
throws URISyntaxException, IOException {
return createTestAuthLocator(configName, null, notFoundMessagesReference);
}

String commandPathPrefix = configFile.getParentFile().getAbsolutePath() + "/";
@NotNull
private RegistryAuthLocator createTestAuthLocator(
String configName,
String envConfigName,
Map<String, String> notFoundMessagesReference
) throws URISyntaxException, IOException {
File configFile = null;
String commandPathPrefix = "";
String commandExtension = "";
String configEnv = null;

if (configName != null) {
configFile = new File(Resources.getResource("auth-config/" + configName).toURI());

commandPathPrefix = configFile.getParentFile().getAbsolutePath() + "/";
} else {
configFile = new File(new URI("file:///not-exists.json"));
}

if (envConfigName != null) {
final File envConfigFile = new File(Resources.getResource("auth-config/" + envConfigName).toURI());
configEnv = FileUtils.readFileToString(envConfigFile, StandardCharsets.UTF_8);

commandPathPrefix = envConfigFile.getParentFile().getAbsolutePath() + "/";
}

if (SystemUtils.IS_OS_WINDOWS) {
commandPathPrefix += "win/";
Expand All @@ -214,6 +306,12 @@ private RegistryAuthLocator createTestAuthLocator(String configName, Map<String,
commandExtension = ".bat";
}

return new RegistryAuthLocator(configFile, commandPathPrefix, commandExtension, notFoundMessagesReference);
return new RegistryAuthLocator(
configFile,
configEnv,
commandPathPrefix,
commandExtension,
notFoundMessagesReference
);
}
}
9 changes: 9 additions & 0 deletions docs/supported_docker_environment/index.md
View file
Open in desktop
Expand Up @@ -45,3 +45,12 @@ Testcontainers will try to connect to a Docker daemon using the following strate
* `DOCKER_CERT_PATH=~/.docker`
* If Docker Machine is installed, the docker machine environment for the *first* machine found. Docker Machine needs to be on the PATH for this to succeed.
* If you're going to run your tests inside a container, please read [Patterns for running tests inside a docker container](continuous_integration/dind_patterns.md) first.

## Docker registry authentication

Testcontainers will try to authenticate to registries with supplied config using the following strategies in order:

* Docker config
* At location specified in `DOCKER_CONFIG` or at `{HOME}/.docker/config.json`
* Environment variables:
* `DOCKER_AUTH_CONFIG`