ARG not support in FROM

[aaronjwhiteside]

Description

Trying to build a Dockerfile like the following fails when using the maven plugin, but is supported and succeeds when building from the command line.

https://docs.docker.com/engine/reference/builder/#from

ARG VERSION:latest
FROM something:$VERSION

[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.22.1:build (build-docker-image) on project docker: Execution build-docker-image of goal io.fabric8:docker-maven-plugin:0.22.1:build failed: Given Docker name 'something:$VERSION' is invalid:
[ERROR] * tag part '$VERSION' doesn't match allowed pattern '^[\w][\w.-]{0,127}$'
[ERROR] See http://bit.ly/docker_image_fmt for more details
[ERROR] -> [Help 1]

VS when run from the command line

Sending build context to Docker daemon  482.7MB
Step 1/14 : ARG VERSION=latest
 ---> 
Step 2/14 : FROM something:$VERSION
 ---> bd7be3f0d8c6
...........
...........

Info

• Maven version (mvn -v) :

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /opt/apache-maven-3.3.9
Java version: 1.8.0_112, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"

• Docker version (docker version):

Client:
 Version:      17.06.2-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 20:12:06 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.06.2-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 19:59:19 2017
 OS/Arch:      linux/amd64
 Experimental: true

[rhuss]

It's only recently that Docker allows parameterization of the image name, and the validation in d-m-p is not updated.

As a workaround you could try to set the VERSION as a Maven property in the properties section like in

<properties>
   <VERSION>latest</VERSION>
</properties>

This will trigger d-m-p to replace all ${VERSION} with this value before sending it to Docker.

Is this a valid solution for you ? If not, please open an issue for extending the image name validation which is currently solely based on this regexp

[abhsc]

I am unable to use the above as a workaround in the latest version of the plugin

[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.25.0:build (default-cli) on project ppd-sandbox: Unable to check image [code.com:5000/test-db:$VERSION] : no such image: code.com:5000/test-db:$VERSION: invalid reference format (Bad Request: 400) -> [Help 1]

Do you know if this issue will suffice for addressing the problem or do you have a suggestion of what the regex change issue would look like? I would be happy to create that issue if necessary.

[rhuss]

Do you mean that is a regression for you (so, that it had worked previously for you), or does it never had worked ?

Could you please post or link a reference project where you see this issue ?

[kucera-jan-cz]

@rhuss I can confirm that your workaround works.

[kevin-canadian]

This workaround may work but it forces one to mix maven and docker configuration in a rather unfortunate way.

We typically perform all configuration of the docker plugin in a common parent pom.xml file. This is possible since the configuration only contains information common to all projects. Specifally, we configure the plugin using maven meta-data such as ${project.version}, and common things like our target registry. This way, maven child projects simply need to use the parent pom.xml file; they don't need to include any configuration related to the docker plugin at all.

If we need to include information about the FROM image via maven, then this information needs to be included in each child project pom.xml, since they are based on different images. If ARG was supported in FROM, we could have this information only in the Dockerfiles (which are different anyway), rather than having to modify each child pom.xml file as well.

Updating the regex patterns in the ImageName class (e.g. to allow the pattern \$\{[^}]+\}) will not be enough, since an ARG can be used for any or all parts of the image name. If you only update the regex you will no longer be able to properly determine the source registry (if specified). Ultimately you will need to expand ARG values in the FROM command before processing it, as docker does, in order to determine where you should be pulling from.

[PieterJanVanAeken]

I am experiencing the same issue and while the above workaround does the trick, it's not particularly clean for our set-up. I'm looking into creating a pull request for this but I have some questions about the approach.

1. The idea I currently have (and it seems to work) is to interpolate the image name before storing it in the variable imageName on the first line of the snippet. I was wondering if this is the best place to do this though.

2. Can I just use the buildArgs parameter of the function or should I move up the mergedBuildMap variable defined at the end of the snippet and use this instead?

3. I am also trying to wrap my head around the plexus interpolation library. From what I understand the syntax is either ${BUILD_ARGUMENT} or $BUILD_ARGUMENT. I can put the build arguments in a MapBasedValueSource, and then use that source in the FixedStringSearchInterpolator for the first syntax, and use the PrefixedValueSourceWrapper (prefix = $) for the other syntax but I didn't find a way to combine them. Any tips here would be welcome as I am not at all familiar with the library.

docker-maven-plugin/src/main/java/io/fabric8/maven/docker/service/BuildService.java

Lines 93 to 116 in d97ea9e

	String imageName = imageConfig.getName();
	ImageName.validate(imageName);
	BuildImageConfiguration buildConfig = imageConfig.getBuildConfiguration();
	String oldImageId = null;
	CleanupMode cleanupMode = buildConfig.cleanupMode();
	if (cleanupMode.isRemove()) {
	oldImageId = queryService.getImageId(imageName);
	}
	long time = System.currentTimeMillis();
	if (buildConfig.getDockerArchive() != null) {
	docker.loadImage(imageName, buildConfig.getAbsoluteDockerTarPath(params));
	log.info("%s: Loaded tarball in %s", buildConfig.getDockerArchive(), EnvUtil.formatDurationTill(time));
	return;
	}
	File dockerArchive = archiveService.createArchive(imageName, buildConfig, params, log);
	log.info("%s: Created %s in %s", imageConfig.getDescription(), dockerArchive.getName(), EnvUtil.formatDurationTill(time));
	Map<String, String> mergedBuildMap = prepareBuildArgs(buildArgs, buildConfig);

[rhuss]

@PieterJanVanAeken thanks a lot for your support-offer and sorry for the delayed answer. Indeed the tricky part would be to do the interpolation on our own.

1. The idea I currently have (and it seems to work) is to interpolate the image name before storing it in the variable imageName on the first line of the snippet. I was wondering if this is the best place to do this though.

Isn't it the base image's name which need to be substitute not the image name itself, right ? So, when the Dockerfile mode is used then the FROM line should be processed for the base image being able to be pulled.

2. Can I just use the buildArgs parameter of the function or should I move up the mergedBuildMap variable defined at the end of the snippet and use this instead?

Yes, the fully mergedBuildMap should be used and could be moved upwards.

3. I am also trying to wrap my head around the plexus interpolation library. From what I understand the syntax is either ${BUILD_ARGUMENT} or $BUILD_ARGUMENT. I can put the build arguments in a MapBasedValueSource, and then use that source in the FixedStringSearchInterpolator for the first syntax, and use the PrefixedValueSourceWrapper (prefix = $) for the other syntax but I didn't find a way to combine them. Any tips here would be welcome as I am not at all familiar with the library.

Tbh, I would do this replacement with some simple regexp instead of using an interpolation library. The default values need to be respected as well, but that should not be a big issue.

[frawa]

The issue is related to the pull policy feature, the mojo will query for FROM images in all cases.
And docker engine will refuse image names with unexpanded ARGs.

For users choosing ARGs in FROMs, a workaround could be to have pull policy Never avoid querying the docker engine.
Or, introduce a dedicated pull policy, that allows to avoid the query with such image names.

See

...
Caused by: io.fabric8.maven.docker.access.hc.http.HttpRequestException: {"message":"no such image: myimage:mytag-${arch}: invalid reference format"} (Bad Request: 400)
    at io.fabric8.maven.docker.access.hc.ApacheHttpClientDelegate$StatusCodeCheckerResponseHandler.handleResponse (ApacheHttpClientDelegate.java:177)
    at org.apache.http.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:223)
    at org.apache.http.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:165)
    at org.apache.http.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:140)
    at io.fabric8.maven.docker.access.hc.ApacheHttpClientDelegate.get (ApacheHttpClientDelegate.java:74)
    at io.fabric8.maven.docker.access.hc.DockerAccessWithHcClient.hasImage (DockerAccessWithHcClient.java:372)
    at io.fabric8.maven.docker.service.QueryService.hasImage (QueryService.java:169)
    at io.fabric8.maven.docker.service.BuildService.autoPullBaseImage (BuildService.java:341)
    at io.fabric8.maven.docker.service.BuildService.buildImage (BuildService.java:67)
    at io.fabric8.maven.docker.BuildMojo.buildAndTag (BuildMojo.java:74)
    at io.fabric8.maven.docker.BuildMojo.processImageConfig (BuildMojo.java:105)
    at io.fabric8.maven.docker.BuildMojo.executeInternal (BuildMojo.java:61)
    at io.fabric8.maven.docker.AbstractDockerMojo.execute (AbstractDockerMojo.java:230)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
...

[vinsgithub]

Tested with latest 0.31.0 release and had same issue.
Tried with @frawa suggestion but I had same result.
The only workaroud that worked was putting args as canonical maven properties as stated by @rhuss
Until this unexpected bug gets resolved, I would suggest to update documentation (which is well written in general) with an alert linking this issue.

[KowKiller]

Hi, I found a use case on which I'm not able to use the provided workaround.

Basically, I'm trying build two images in the same pom file.
Both images are built with the same Dockerfile applied to different base images.

Now, even if I put something like FROM some_image:${VERSION}, I can't find a way to specify different versions for the two images (maven properties are shared between the images).

Does someone have any idea?

[rhuss]

@KowKiller sorry, there is probably no good easy solution. You could go to use multi-modules (which then has different property names) or use profiles and two runs, but that's tedious.

Maybe we should fix this issue ? Unfortunately, I don't have time to code on it, but maybe someone could pick up the ideas laid out in this issue ?

[KowKiller]

Just for reference, at the end the "easiest" solution I could find was to use antrun plugin for filtering the files. (Refactoring the whole project as a multi-module pom would have been quite complex)

I added this plugin here:

<plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <executions>
        <execution>
            <id>filter-dockerfiles-for-different-base-versions</id>
            <phase>package</phase>
            <goals>
                <goal>run</goal>
            </goals>
            <configuration>
                <tasks>
                    <copy file="docker/Dockerfile" toFile="target/antrun/Dockerfile.v1">
                        <filterset>
                            <filter token="version" value="v1"/>
                        </filterset>
                    </copy>
                    <copy file="docker/Dockerfile" toFile="target/antrun/Dockerfile.v2">
                        <filterset>
                            <filter token="version" value="v2"/>
                        </filterset>
                    </copy>
                </tasks>
            </configuration>
        </execution>
    </executions>
</plugin>

I used the ant delimiter (@) in the Dockerfile:

FROM base_image:@version@

And changed the image builds to point to the generated Dockerfiles:

<image>
    <name>image:v1</name>
    <build>
        <contextDir>${basedir}/docker</contextDir>
        <dockerFile>${project.build.directory}/antrun/Dockerfile.v1</dockerFile>
    </build>
</image>
<image>
    <name>image:v2</name>
    <build>
        <contextDir>${basedir}/docker</contextDir>
        <dockerFile>${project.build.directory}/antrun/Dockerfile.v2</dockerFile>
    </build>
</image>

[JollyGoodHolly]

@rohanKanojia @rhuss Any updates on this PR? I would really like to use this feature in my project.

[rohanKanojia]

@JollyGoodHolly : hmm, let me revisit this whenever I can.

[JollyGoodHolly]

Ping @rhuss in case he hasn't seen the reworked PR.

[ThatDockerUser]

Thanks for merging this. Is a new version release planned soon?

[rhuss]

Yes. I just want to integrate some of the remaining PRs, and then do a release right after this.

[gurpiarbassi]

When will this fix be available?

[rohanKanojia]

@gurpiarbassi : Is this issue still reproducible in v0.38.0?