Skip to content
This repository has been archived by the owner on Mar 21, 2022. It is now read-only.

support for credsStore or credential-helper in new docker versions #657

Closed
johnflavin opened this issue Mar 2, 2017 · 16 comments
Closed

Comments

@johnflavin
Copy link
Contributor

The current implementation of RegistryAuth has several static builder methods that parse the credentials out of the supplied or default config.json file. That strategy does not work in the newest docker version (v17.03.0-ce), as the structure of that file has changed and the credentials are no longer be stored there. They are stored in a more secure system-native way (i.e. the keychain on macOS) using the docker credential helpers.

These changes have already rolled out on the beta channel of Docker for Mac. When you use docker login to store your credentials, they are put into the keychain. The config.json file looks like this:

$ cat ~/.docker/config.json
{
	"auths": {
		"https://index.docker.io/v1/": {}
	},
	"credsStore": "osxkeychain"
}

So that presents a problem for Remote API users like us. We can no longer read the credentials from a file. I don't know exactly how to interact with the credential helpers, whether they have their own API or if we need to use system calls to execute the binary. It is possible that there is already some easy solution and I just don't know what it is.

I want to open this ticket to discuss the coming changes and brainstorm solutions.

@mattnworb
Copy link
Member

Good catch, and thanks for opening the issue up.

@mattnworb
Copy link
Member

mattnworb commented May 16, 2017

seems like the API for the credential helpers is defined at the bottom of https://github.com/docker/docker-credential-helpers#development - a client is support to execute the process passing certain values in the first argument

@NicoTexas
Copy link

NicoTexas commented Jun 1, 2017

This is true also on Windows as Docker login is storing in wincred.

{ "auths": { "docker-dev.domain.com": {} }, "credsStore": "wincred" }

@mattnworb
Copy link
Member

@NicoTexas yea there is no support in docker-client for credsStore or helpers yet.

I think it would be possible to add support for the credential helpers to this library, since the API seems pretty well established, although it would feel weird that this library might end up execing random processes on the user's host.

@beihaifeiwu
Copy link

I got the fllow exception on windows after docker upgrade:

[INFO] --- dockerfile-maven-plugin:1.3.0:build (default) @ nagrand-ws ---
[INFO] Building Docker context F:\Source\nagrand\nagrand-ws
[INFO]
[INFO] Image will be built as 10.0.25.21:5000/nagrand-ws:2.1-SNAPSHOT
[INFO]
[WARNING] An attempt failed, will retry 1 more times
org.apache.maven.plugin.MojoExecutionException: Could not build image
        at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:164)
        at com.spotify.plugin.dockerfile.BuildMojo.execute(BuildMojo.java:95)
        at com.spotify.plugin.dockerfile.AbstractDockerMojo.tryExecute(AbstractDockerMojo.java:219)
        at com.spotify.plugin.dockerfile.AbstractDockerMojo.execute(AbstractDockerMojo.java:208)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: com.spotify.docker.client.exceptions.DockerException: com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.docker.client.messages.RegistryAuth: no String-argument constructor/factory method to deserialize from String value ('wincred')
 at [Source: N/A; line: -1, column: -1] (through reference chain: java.util.LinkedHashMap["credsStore"])
        at com.spotify.docker.client.auth.ConfigFileRegistryAuthSupplier.authForBuild(ConfigFileRegistryAuthSupplier.java:96)
        at com.spotify.docker.client.auth.MultiRegistryAuthSupplier.authForBuild(MultiRegistryAuthSupplier.java:77)
        at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1371)
        at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1348)
        at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:157)
        ... 25 more
Caused by: com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.docker.client.messages.RegistryAuth: no String-argument constructor/factory method to deserialize from String value ('wincred')
 at [Source: N/A; line: -1, column: -1] (through reference chain: java.util.LinkedHashMap["credsStore"])
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1456)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1012)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.ValueInstantiator._createFromStringFallbacks(ValueInstantiator.java:370)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromString(StdValueInstantiator.java:315)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromString(BeanDeserializerBase.java:1283)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:159)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:150)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:517)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:362)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:27)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1187)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3770)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2099)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:2596)
        at com.spotify.docker.client.DockerConfigReader.parseDockerConfig(DockerConfigReader.java:94)
        at com.spotify.docker.client.DockerConfigReader.fromConfig(DockerConfigReader.java:48)
        at com.spotify.docker.client.auth.ConfigFileRegistryAuthSupplier.authForBuild(ConfigFileRegistryAuthSupplier.java:94)
        ... 29 more
[INFO] Building Docker context F:\Source\nagrand\nagrand-ws

Is there any workaround about this issue ?

@jordijansen
Copy link

+1
I'm getting the same error as @beihaifeiwu

After ugrapding to 17.06.0-ce. I'm now unable to build any image using the spotify-dockerfile plugin.

@jordijansen
Copy link

jordijansen commented Jul 6, 2017

@beihaifeiwu if you don't use the credStore you can remove it from your config.json (C:\Users<user>.docker\config.json). Remove the credStore key from the JSON object. You then at least can build images again. This is just a temporary workaround.

@ranef
Copy link

ranef commented Jul 25, 2017

I faced the same problem and tried the workaround suggested of removing the credStore key from the config.json. In my environment this was the only entry in the json file and when I removed it I got an exception about the file being empty.
The only thing that solved the problem in my case was removing the entire file

@ljnelson
Copy link

It seems that if you remove the ~/.docker/config.json file, then the plugin will fall back to looking in your Maven's settings information. That's good. But then unless you jump through other hoops your docker command line won't work, right? Isn't this a bug, not an enhancement?

@mattnworb
Copy link
Member

@ljnelson I am not sure if the semantics makes a difference, this is something that was introduced in docker relatively recently (~6 months) which docker-client does not yet have support for. We would welcome any PRs to add support for credsStore, but have not had a chance to do it ourselves yet.

@ljnelson
Copy link

I see; thanks for the information!

@ranef
Copy link

ranef commented Aug 16, 2017

The docker command line works fine without the config.json file
One thing I noticed which is pretty annoying is that config.json file is recreated each time the docker service is started (e.g. every windows restart) so I have to keep deleting it each time.
Is there any dummy entries I can put in this file that will eliminate the credStore error but will prevent docker from recreating it?

@jschneid-nmdp
Copy link

When using the useMavenSettingsForAuth plugin configuration option, it appears dockerfile-maven-plugin version 1.3.6 requires the Docker server password in the Maven settings file to be stored as cleartext, not "encrypted" via mvn --encrypt-password.

@stromnet
Copy link

stromnet commented Feb 12, 2018

FYI: As described earlier, OS X installation adds credsStore to config file causing some operations to fall, for example .pull.

Our temporary workaround (we don't need auth since it's internal repos) is to create the client with .registryAuthSupplier(new MultiRegistryAuthSupplier(Collections.emptyList()))

@smythie86
Copy link
Contributor

I believe since my PR was merged above this issue can be closed.

@alibbbian
Copy link

alibbbian commented Aug 5, 2021

I got the fllow exception on windows after docker upgrade:

[INFO] --- dockerfile-maven-plugin:1.3.0:build (default) @ nagrand-ws ---
[INFO] Building Docker context F:\Source\nagrand\nagrand-ws
[INFO]
[INFO] Image will be built as 10.0.25.21:5000/nagrand-ws:2.1-SNAPSHOT
[INFO]
[WARNING] An attempt failed, will retry 1 more times
org.apache.maven.plugin.MojoExecutionException: Could not build image
        at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:164)
        at com.spotify.plugin.dockerfile.BuildMojo.execute(BuildMojo.java:95)
        at com.spotify.plugin.dockerfile.AbstractDockerMojo.tryExecute(AbstractDockerMojo.java:219)
        at com.spotify.plugin.dockerfile.AbstractDockerMojo.execute(AbstractDockerMojo.java:208)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: com.spotify.docker.client.exceptions.DockerException: com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.docker.client.messages.RegistryAuth: no String-argument constructor/factory method to deserialize from String value ('wincred')
 at [Source: N/A; line: -1, column: -1] (through reference chain: java.util.LinkedHashMap["credsStore"])
        at com.spotify.docker.client.auth.ConfigFileRegistryAuthSupplier.authForBuild(ConfigFileRegistryAuthSupplier.java:96)
        at com.spotify.docker.client.auth.MultiRegistryAuthSupplier.authForBuild(MultiRegistryAuthSupplier.java:77)
        at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1371)
        at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1348)
        at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:157)
        ... 25 more
Caused by: com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.docker.client.messages.RegistryAuth: no String-argument constructor/factory method to deserialize from String value ('wincred')
 at [Source: N/A; line: -1, column: -1] (through reference chain: java.util.LinkedHashMap["credsStore"])
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1456)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1012)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.ValueInstantiator._createFromStringFallbacks(ValueInstantiator.java:370)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromString(StdValueInstantiator.java:315)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromString(BeanDeserializerBase.java:1283)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:159)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:150)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:517)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:362)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:27)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1187)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3770)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2099)
        at com.spotify.docker.client.shaded.com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:2596)
        at com.spotify.docker.client.DockerConfigReader.parseDockerConfig(DockerConfigReader.java:94)
        at com.spotify.docker.client.DockerConfigReader.fromConfig(DockerConfigReader.java:48)
        at com.spotify.docker.client.auth.ConfigFileRegistryAuthSupplier.authForBuild(ConfigFileRegistryAuthSupplier.java:94)
        ... 29 more
[INFO] Building Docker context F:\Source\nagrand\nagrand-ws

Is there any workaround about this issue ?

delete the file ./docker/config.json,but The file appears again after restart docker.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests