failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running - runc did not terminate sucessfully

[SeanFacer]

Mac Catilina
Docker Version: 2.4.0.0

Docker file

ARG ALPINE_VERSION
ARG PHP_VERSION
ARG SECRETS_PARAM_NAME

FROM alpine:${ALPINE_VERSION} as python
RUN apk --no-cache --update add \
  py-pip && \
  pip install --upgrade awscli

FROM madnight/docker-alpine-wkhtmltopdf as wkhtmltox

FROM php:${PHP_VERSION}-fpm-alpine${ALPINE_VERSION}

VOLUME ["/app/webroot"]

RUN apk --no-cache --purge --update add \
    --virtual .build-deps \
    autoconf \
    build-base \
    libtool && \
  apk --no-cache --purge --update add \
    --virtual .persistent-deps \
    py-pip  \
    freetype-dev \
    gnumeric \
    icu-dev \
    imagemagick \
    imagemagick-dev \
    jq \
    glib libcrypto1.0 libgcc libintl libjpeg-turbo-dev libssl1.0 libstdc++ libx11 libxext libxrender \
    libmcrypt-dev \
    libpng-dev \
    postgresql-dev \
    postgresql-client \
    python2 \
    ttf-dejavu ttf-droid ttf-freefont ttf-liberation ttf-ubuntu-font-family && \
  docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ && \
  docker-php-ext-install gd intl opcache pdo_pgsql mcrypt zip && \
  pecl install apcu imagick xdebug && \
  docker-php-ext-enable apcu imagick && \
  pip install --upgrade awscli es2csv && \
  apk del .build-deps && \
  rm -rf /root/.cache && \
  rm -rf /tmp/*

COPY --from=python /root/.cache /root/.cache
COPY --from=wkhtmltox /bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
RUN ln -s /usr/local/bin/wkhtmltopdf wkhtmltopdf

COPY php/ini/*.ini  /usr/local/etc/php/conf.d/
COPY php/fpm/*.conf /usr/local/etc/php-fpm.d/

ENV SECRETS_PARAM_NAME ${SECRETS_PARAM_NAME}
COPY dms-entrypoint /usr/local/bin/
ENTRYPOINT ["dms-entrypoint"]
CMD ["docker-php-entrypoint", "php-fpm"]

[riteshgurung]

@SeanFacer I was facing a similar issue. What really worked for me was

• Stop the containers using the docker image
• Remove the volumes used by the containers

Rebuild the image.

Alternatively, tag the build image with suffix abc:1.x

[favourkiiza]

same issue....any fix??

[kushthedude]

We are facing the same issue recently with our Docker build and push, https://github.com/layer5io/meshery/pull/1959/checks?check_run_id=1435279720. Any fixes?

[janpauldahlke]

I can confirm the behavior on both MacOS and Win10. Is there any fix, yet? googling me into the ground.

Please consider my Dockerfile

FROM node:lts as builder

COPY . /denta-manual-site

WORKDIR /denta-manual-site

RUN npm install
RUN npm install --arch=x64 --platform=linux sharp
RUN $(npm bin)/gatsby build

FROM nginx

COPY --from=builder /denta-manual-site/public/* /usr/share/nginx/html/

EXPOSE 80

my build command is
docker build -t denta-manual-site .

Is someone out there able to fix this?

[kushthedude]

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

This worked for me @janpauldahlke

[EmreErinc]

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

This worked for me @janpauldahlke

How did you fix? I am stuck in build "Push Docker Image to Github Packages". I am newbie at Github Actions

[janpauldahlke]

@kushthedude Hi ty for replying. You want this export to be added in line 2? Iam just reading here https://docs.docker.com/engine/reference/commandline/export/

[kushthedude]

You need to add into your terminal where you are building your docker image.

[janpauldahlke]

@kushthedude Great man, it instantly worked now. I will add those lines to my README.md. ty bro. BTW: is there any explanation why this occurs?

[kushthedude]

@kushthedude Great man, it instantly worked now. I will add those lines to my README.md. ty bro. BTW: is there any explanation why this occurs?

Looks like a bug in the latest BUILDKIT, am still not sure. @adrubesh may have more info on it

[kushthedude]

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

This worked for me @janpauldahlke

How did you fix? I am stuck in build "Push Docker Image to Github Packages". I am newbie at Github Actions

@EmreErinc In the run part of the action add the above export lines like done here https://github.com/layer5io/meshery/blob/0049d3eef50666352e40ac6706ac08d8138f0da3/.github/workflows/build-and-release.yml#L53

[thaJeztah]

Is this only happening with github's registry, or also when using docker hub?

[kushthedude]

Is this only happening with github's registry, or also when using docker hub?

We are using docker-hub and it occurred their too, plus I think registry won't matter here as this is happening in the build step rather than push to registry step 😅

[thaJeztah]

plus I think registry won't matter here as this is happening in the build step rather than push to registry step 😅

thx! mostly checking, because the "old" github image registry is known to be largely broken (as it only implements parts of the distribution (registry) specification; moby/moby#41687)

[adrubesh]

It happens during build, my best guess is it's a bug in buildkit, which is fine considering buildkit still isn't stable yet. The two exports I mentioned above disable buildkit in your shell environment. If you are using Docker desktop for Mac / Windows you might have to also disable it in your 'Docker Engine' json config.
Docker desktop -> Settings -> Docker Engine -> Change the "features": { buildkit: true} to "features": { buildkit: false}.

Note, this isn't a FIX, this is a workaround until someone on the docker teams implements a proper fix. Please give buildkit another shot when it's a little more stable :)

[thaJeztah]

@adrubesh what's the exact error you're getting? Do you have a repository to reproduce the issue? (the Dockerfile in your previous comment COPY's files from your source-code, so without the source, it won't be possible to try the dockerfile.

[EmreErinc]

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

This worked for me @janpauldahlke

How did you fix? I am stuck in build "Push Docker Image to Github Packages". I am newbie at Github Actions

@EmreErinc In the run part of the action add the above export lines like done here https://github.com/layer5io/meshery/blob/0049d3eef50666352e40ac6706ac08d8138f0da3/.github/workflows/build-and-release.yml#L53

thanks for reply @kushthedude , i am viewing your meshery yml files. learn too much things

[kushthedude]

@adrubesh what's the exact error you're getting? Do you have a repository to reproduce the issue? (the Dockerfile in your previous comment COPY's files from your source-code, so without the source, it won't be possible to try the dockerfile.

@thaJeztah you can see the error over here https://github.com/layer5io/meshery/pull/1959/checks?check_run_id=1435279720 where we are building image in GitHub action runner and pushing it to dockerhub.

[adrubesh]

@thaJeztah The exact error I was getting is the title of this issue, which is why I decided to comment here instead of create my own issue. I don't have a repository with a sample. However, for me, it happens when doing npm install and npm run build commands to build assets via npm.

[thaJeztah]

@kushthedude so looking at https://pipelines.actions.githubusercontent.com/yzMpWGqvkojJTIqTDBcNkCpnajSBnB0u0MB0Phbp1op4nmEvFe/_apis/pipelines/1/runs/5257/signedlogcontent/8?urlExpires=2020-11-24T13%3A56%3A20.3986902Z&urlSigningMethod=HMACV1&urlSignature=9TuQSpebPinjLnqwn8Ja6OLYqgpA2EKongO78RXV7SY%3D

The actual error is in this part;

2020-11-21T09:44:55.6953094Z #13 [meshery-server 5/5] RUN cd cmd; GOPROXY=direct GOSUMDB=off go build -ld...
2020-11-21T09:44:55.6953810Z #13 68.41 verifying github.com/prologic/bitcask@v0.3.6/go.mod: checksum mismatch
2020-11-21T09:44:55.6954724Z #13 68.41 	downloaded: h1:WQuqL23CGZcC83DhKuXH6KMHe1m25+Eb43s8yM3MnF0=
2020-11-21T09:44:55.6955673Z #13 68.41 	go.sum:     h1:lGs61R8cREh1Wq/O+eErqTdGmsaen7cDxu3Zdbs9xBA=
2020-11-21T09:44:55.6956221Z #13 68.41
2020-11-21T09:44:55.6956538Z #13 68.41 SECURITY ERROR
2020-11-21T09:44:55.6957043Z #13 68.41 This download does NOT match an earlier download recorded in go.sum.
2020-11-21T09:44:55.6957712Z #13 68.41 The bits may have been replaced on the origin server, or an attacker may
2020-11-21T09:44:55.6958317Z #13 68.41 have intercepted the download attempt.
2020-11-21T09:44:55.6958911Z #13 68.41
2020-11-21T09:44:55.6959479Z #13 68.41 For more information, see 'go help module-auth'.
2020-11-21T09:45:01.2476077Z #13 ...

The executor failed running errors are definitely horrible to read, but looks like the failures is something that happens during that step of the docker build

2020-11-21T09:45:02.2729148Z #13 [meshery-server 5/5] RUN cd cmd; GOPROXY=direct GOSUMDB=off go build -ld...
2020-11-21T09:45:02.2731145Z #13 ERROR: executor failed running [/bin/sh -c cd cmd; GOPROXY=direct GOSUMDB=off go build -ldflags="-w -s -X main.globalTokenForAnonymousResults=$TOKEN -X main.version=$GIT_VERSION -X main.commitsha=$GIT_COMMITSHA" -tags draft -a -o /meshery .]: runc did not terminate sucessfully
2020-11-21T09:45:02.4197529Z

full logs from that run;
8.txt

@adrubesh yes, so (see above) the error is effectively (but admittedly, horribly formatted) saying; something failed in a RUN step, and the executors (runc) existed while executing the container for that step.

[kushthedude]

@kushthedude so looking at https://pipelines.actions.githubusercontent.com/yzMpWGqvkojJTIqTDBcNkCpnajSBnB0u0MB0Phbp1op4nmEvFe/_apis/pipelines/1/runs/5257/signedlogcontent/8?urlExpires=2020-11-24T13%3A56%3A20.3986902Z&urlSigningMethod=HMACV1&urlSignature=9TuQSpebPinjLnqwn8Ja6OLYqgpA2EKongO78RXV7SY%3D

The actual error is in this part;

2020-11-21T09:44:55.6953094Z #13 [meshery-server 5/5] RUN cd cmd; GOPROXY=direct GOSUMDB=off go build -ld...
2020-11-21T09:44:55.6953810Z #13 68.41 verifying github.com/prologic/bitcask@v0.3.6/go.mod: checksum mismatch
2020-11-21T09:44:55.6954724Z #13 68.41 	downloaded: h1:WQuqL23CGZcC83DhKuXH6KMHe1m25+Eb43s8yM3MnF0=
2020-11-21T09:44:55.6955673Z #13 68.41 	go.sum:     h1:lGs61R8cREh1Wq/O+eErqTdGmsaen7cDxu3Zdbs9xBA=
2020-11-21T09:44:55.6956221Z #13 68.41
2020-11-21T09:44:55.6956538Z #13 68.41 SECURITY ERROR
2020-11-21T09:44:55.6957043Z #13 68.41 This download does NOT match an earlier download recorded in go.sum.
2020-11-21T09:44:55.6957712Z #13 68.41 The bits may have been replaced on the origin server, or an attacker may
2020-11-21T09:44:55.6958317Z #13 68.41 have intercepted the download attempt.
2020-11-21T09:44:55.6958911Z #13 68.41
2020-11-21T09:44:55.6959479Z #13 68.41 For more information, see 'go help module-auth'.
2020-11-21T09:45:01.2476077Z #13 ...

This was due to the modcache which we cleared later in the action runner as well as in docker.

The executor failed running errors are definitely horrible to read, but looks like the failures is something that happens during that step of the docker build

2020-11-21T09:45:02.2729148Z #13 [meshery-server 5/5] RUN cd cmd; GOPROXY=direct GOSUMDB=off go build -ld...
2020-11-21T09:45:02.2731145Z #13 ERROR: executor failed running [/bin/sh -c cd cmd; GOPROXY=direct GOSUMDB=off go build -ldflags="-w -s -X main.globalTokenForAnonymousResults=$TOKEN -X main.version=$GIT_VERSION -X main.commitsha=$GIT_COMMITSHA" -tags draft -a -o /meshery .]: runc did not terminate sucessfully
2020-11-21T09:45:02.4197529Z

Yes, This is the error that got fixed when we didn't use buildkit, strange is that we are using buildkit for more than 6 months but it occurred a few weeks ago maybe some recent commit may have broken the functionality.

[thaJeztah]

Looking at that repository, I see that multiple tags were pushed on the same day; Looking at the corresponding versions on https://pkg.go.dev/github.com/prologic/bitcask?tab=versions, those indicate the v3.6.0 tag should've been before that date. I suspect that someone actually (accidentally) force-pushed/overwrite an existing tag

[thaJeztah]

Ah, sorry, my comment crossed your previous comment

[adrubesh]

@thaJeztah Yes, I understand the error occurring during the RUN step, however, that run step error only occurred during a docker build with buildkit. It did not occur when not using buildkit. It did not occur on the machine locally when not in docker. That's why I suggested to just disable buildkit. I know, it doesn't help buildkit devs much saying that but I was in a rush to get things working and didn't capture the entire log. If I run into it again and this issue is still open I can hopefully drop some more meaningful output here :)

[tonistiigi]

I think we should close this issue to not confuse new people. Only report that provided full error output is #426 (comment) and that one clearly is a temporary application error.

2020-11-21T09:44:55.6953810Z #13 68.41 verifying github.com/prologic/bitcask@v0.3.6/go.mod: checksum mismatch
2020-11-21T09:44:55.6954724Z #13 68.41 	downloaded: h1:WQuqL23CGZcC83DhKuXH6KMHe1m25+Eb43s8yM3MnF0=
2020-11-21T09:44:55.6955673Z #13 68.41 	go.sum:     h1:lGs61R8cREh1Wq/O+eErqTdGmsaen7cDxu3Zdbs9xBA=
2020-11-21T09:44:55.6956221Z #13 68.41

This is just a prefix that appears is any error that happens in your application logic inside the container. Eg. a bug, external service misbehaving or network flakiness. If the failing process produced logs for the error they are shown above it.

If you do see it and can't figure out the cause from logs, open an issue with a reproducer and full logs output.

Other than making the error message more clear there is nothing to work on here atm.

[padresmurfa]

I had a very similar issue, for which I believe I have a better workaround / fix. In my case I was experiencing:

failed to solve with frontend dockerfile.v0: failed to build LLB: failed to compute cache key: " ... " not found: not found

I noticed this issue, mentioned above, where @kushthedude said We had error in modcache which caused docker failures but due to poor logging in buildkit. Thus it seemed to me that if I could clear the modcache, I would perhaps be able to continue my work.

I did not find an official way to do so, but the issue disappeared after I did the following:

• delete the offending statements in my dockerfile
• rebuild the docker image
• reintroduce the statements.
• rebuild the docker image

Thus I could imagine the same type of solution might work for others with similar issues, such as the one OP described.

Later the issue popped up again when I did the following two things, but disappeared when I resolved them:

• COPY from a symlink to a directory
• COPY from a fullpath to that same directory

So my hypothesis is that perhaps modcache was thrown into a bad state by a symlink, but I do not have a reference to the bug in question, so I can't really say for sure. Perhaps computing cache keys is confused in some way by the fact that a symlinked target effectively has multiple filenames / fullpaths.

For the record, I had tried a few other things first, without success. I'm including them here in case my success was actually dependent on them in addition to the steps above:

• disabling buildkit, as per the instructions further up in this thread
• docker image prune -a
• docker container prune -a
• docker volume prune
• docker builder prune -a
• reinstalling Docker Desktop for Mac
• uninstalling the stable version Docker Desktop for Mac, and installing the edge version

[brownchow]

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

I'm using windows10, Hyper-V, this works for me!

[tonistiigi]

#426 (comment)

[GreyIce12]

Make sure you saved the docker file ctrl s its not finding it because you didn't save the file

[bixu]

I also had to set

DOCKER_BUILDKIT=0

when using macOS Docker 3.1.0

[ghusta]

It happens during build, my best guess is it's a bug in buildkit, which is fine considering buildkit still isn't stable yet. The two exports I mentioned above disable buildkit in your shell environment. If you are using Docker desktop for Mac / Windows you might have to also disable it in your 'Docker Engine' json config.
Docker desktop -> Settings -> Docker Engine -> Change the "features": { buildkit: true} to "features": { buildkit: false}.

Note, this isn't a FIX, this is a workaround until someone on the docker teams implements a proper fix. Please give buildkit another shot when it's a little more stable :)

Great ! Nothing new, just adding the Docker documentation explaining the same :

• https://docs.docker.com/develop/develop-images/build_enhancements/#to-enable-buildkit-builds

[kannasuresh99]

FROM mcr.microsoft.com/windows:1903 AS build
WORKDIR C:\\odtsetup
ADD https://download.microsoft.com/download/2/7/A/27AF1BE6-DD20-4CB4-B154-EBAB8A7D4A7E/officedeploymenttool_11617-33601.exe odtsetup.exe
RUN odtsetup.exe /quiet /norestart /extract:C:\\odtsetup
FROM mcr.microsoft.com/windows:1903 AS download
WORKDIR C:\\odtsetup
COPY --from=build C:\\odtsetup\\setup.exe .
ADD config.xml .
RUN setup.exe /download C:\\odtsetup\\config.xml
FROM mcr.microsoft.com/windows:1903
MAINTAINER rkttu
WORKDIR C:\\odtsetup
COPY --from=build C:\\odtsetup\\setup.exe .
COPY --from=download C:\\odtsetup\\Office .
ADD config.xml .
RUN setup.exe /configure C:\\odtsetup\\config.xml
WORKDIR /
RUN rmdir /s /q C:\\odtsetup
# https://stackoverflow.com/questions/10837437/interop-word-documents-open-is-null
RUN powershell -Command new-object -comobject word.application
RUN mkdir C:\\Windows\\SysWOW64\\config\\systemprofile\\Desktop
VOLUME C:\\data

This is my Dockerfile, I'm getting this error.
Can anybody else, help me?

[thaJeztah]

This is my Dockerfile, I'm getting this error.
Can anybody else, help me?

You need to switch Docker Desktop to Windows containers if you want to run native windows images, otherwise the Linux daemon will attempt to run the Windows image, which will fail; https://docs.docker.com/docker-for-windows/#switch-between-windows-and-linux-containers

[jinuhwang]

Just in case anyone else is making the same mistake I did,
I was putting MAINTAINER before FROM.

Found it on this stack overflow answer.

[tudor-pop]

I disabled Big Sur virtualization framework and it worked

[jacob418]

It happens during build, my best guess is it's a bug in buildkit, which is fine considering buildkit still isn't stable yet. The two exports I mentioned above disable buildkit in your shell environment. If you are using Docker desktop for Mac / Windows you might have to also disable it in your 'Docker Engine' json config.
Docker desktop -> Settings -> Docker Engine -> Change the "features": { buildkit: true} to "features": { buildkit: false}.

Note, this isn't a FIX, this is a workaround until someone on the docker teams implements a proper fix. Please give buildkit another shot when it's a little more stable :)

Solved it for me!

[Diluka]

I disabled Big Sur virtualization framework and it worked

it works. but hyperkit is very power-consuming, high CPU usage, and heat generation.

[rodrigoacastro]

For some reason, this error only happens if I try to run docker using bash. Using WSL it does NOT happen. You guys should try it.