Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yarn not found in node:alpine #649

Closed
beldur opened this issue Mar 16, 2018 · 71 comments
Closed

yarn not found in node:alpine #649

beldur opened this issue Mar 16, 2018 · 71 comments
Assignees

Comments

@beldur
Copy link

@beldur beldur commented Mar 16, 2018

I'm using the node:alpine image in a gitlab-ci build.
Since a few minutes ago I get the following error in gitlab-ci:

$ yarn config set cache-folder .yarn
/bin/sh: eval: line 50: yarn: not found
ERROR: Job failed: exit code 127

My .gitlab-ci.yaml:

build:
  image: node:alpine
  stage: build
  script:
    - yarn config set cache-folder .yarn
    - yarn
    - yarn build
@mmcgarr
Copy link

@mmcgarr mmcgarr commented Mar 16, 2018

I'm seeing the same in circleci

Step 6/9 : RUN yarn
 ---> Running in a8731d6619d5
/bin/sh: yarn: not found
The command '/bin/sh -c yarn' returned a non-zero code: 127

@rtorino
Copy link

@rtorino rtorino commented Mar 16, 2018

I'm experiencing the same issue.

@Scukerman
Copy link

@Scukerman Scukerman commented Mar 16, 2018

Same here.
Need to revert ae26d21.

@thrawny
Copy link

@thrawny thrawny commented Mar 16, 2018

Same for the normal node:9.8.0

~ » docker run -it --rm node:9.8.0 sh  
# yarn
sh: 1: yarn: not found

@fabriciocolombo
Copy link

@fabriciocolombo fabriciocolombo commented Mar 16, 2018

Same here with node:carbon

@nicolas-duvauchel
Copy link

@nicolas-duvauchel nicolas-duvauchel commented Mar 16, 2018

This was caused by this commit: ae26d21

Given the way the symlink is done mkdir -p /opt/yarn should have been deleted

@mmcgarr
Copy link

@mmcgarr mmcgarr commented Mar 16, 2018

node:9.7-alpine image seems unaffected

@SimenB
Copy link
Member

@SimenB SimenB commented Mar 16, 2018

@tredston
Copy link

@tredston tredston commented Mar 16, 2018

Seems to affect all supported tags except chakracore.

@michalpiekarski
Copy link

@michalpiekarski michalpiekarski commented Mar 16, 2018

The problem is with mkdir -p /opt/yarn (related to missing /opt directory see #648).
The correct command should be mkdir -p /opt.

@cecton
Copy link
Contributor

@cecton cecton commented Mar 16, 2018

latest is affected I think:

The link /usr/local/bin/yarn is broken, it targets /opt/yarn/bin/yarn which doesn't exist. But /opt/yarn/yarn-v1.5.1 exists.

root@5706d42db0c6:/# ls /usr/local/
CHANGELOG.md  LICENSE  README.md  bin  etc  games  include  lib  man  sbin  share  src
root@5706d42db0c6:/# ls /usr/local/bin
node  nodejs  npm  npx  yarn  yarnpkg
root@5706d42db0c6:/# /usr/local/bin/yarn
bash: /usr/local/bin/yarn: No such file or directory
root@5706d42db0c6:/# ls /usr/local/bin
node  nodejs  npm  npx  yarn  yarnpkg
root@5706d42db0c6:/# stat /usr/local/bin/yarn
  File: '/usr/local/bin/yarn' -> '/opt/yarn/bin/yarn'
  Size: 18              Blocks: 0          IO Block: 4096   symbolic link
Device: 31h/49d Inode: 62          Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-03-16 10:19:06.000000000 +0000
Modify: 2018-03-16 10:19:06.000000000 +0000
Change: 2018-03-16 11:13:53.243600600 +0000
 Birth: -
root@5706d42db0c6:/# /usr/local/bin/yarn
bash: /usr/local/bin/yarn: No such file or directory
root@5706d42db0c6:/# /usr/local/bin/n
node    nodejs  npm     npx
root@5706d42db0c6:/# /usr/local/bin/yarn
bash: /usr/local/bin/yarn: No such file or directory
root@5706d42db0c6:/# /opt/yarn/bin/yarn
bash: /opt/yarn/bin/yarn: No such file or directory
root@5706d42db0c6:/# ls /opt/yarn
yarn-v1.5.1
root@5706d42db0c6:/# ls /opt/yarn/yarn-v1.5.1
LICENSE  README.md  bin  lib  package.json
root@5706d42db0c6:/# ls /opt/yarn/yarn-v1.5.1

@lingz
Copy link

@lingz lingz commented Mar 16, 2018

Broke our CI as well this morning!

@SimenB
Copy link
Member

@SimenB SimenB commented Mar 16, 2018

Should #639, #647 and #648 all be reverted, or can someone send a new PR fixing it?

@elboletaire
Copy link

@elboletaire elboletaire commented Mar 16, 2018

Same here, with every version of node. We've already tried carbon and latest. Both fail with the yarn command.

@SimenB
Copy link
Member

@SimenB SimenB commented Mar 16, 2018

This affects 9.8.0, 9.8, 9, latest, 8.10.0, 8.10, 8, carbon, 6.13.1, 6.13, 6, boron, 4.8.7, 4.8, 4, argon and all variants (alpine, slim, onbuild, wheezy, stretch)

@cecton
Copy link
Contributor

@cecton cecton commented Mar 16, 2018

@SimenB I volunteer to make a PR. I will remove the mkdir -p and replace by a symlink where the yarn has been decompressed.

@SimenB
Copy link
Member

@SimenB SimenB commented Mar 16, 2018

@cecton thanks!

@pesho
Copy link
Member

@pesho pesho commented Mar 16, 2018

I will be able to fix that in several hours. If anyone can take care of it earlier, please do.

@SimenB
Copy link
Member

@SimenB SimenB commented Mar 16, 2018

I asked for a revert in docker-library/official-images#4131, but I think they are all on the west coast of the US, so probably asleep.

@ghindle
Copy link

@ghindle ghindle commented Mar 16, 2018

Adding the following to my Dockerfile as a temporary fix helped. Remember YMMV!
RUN mkdir -p /opt/yarn/bin && ln -s /opt/yarn/yarn-v1.5.1/bin/yarn /opt/yarn/bin/yarn

st3v3nhunt added a commit to nhsuk/pharmacy-data-etl that referenced this issue Mar 16, 2018
Reason is a change to `8.10.0` broke the image. See
[issue](nodejs/docker-node#649) for more
details.
st3v3nhunt added a commit to nhsuk/nearby-services-api that referenced this issue Mar 16, 2018
Reason is a change to `8.10.0` broke the image. See
[issue](nodejs/docker-node#649) for more
details.
chorrell added a commit that referenced this issue Mar 16, 2018
@cecton
Copy link
Contributor

@cecton cecton commented Mar 16, 2018

Close to finish....

st3v3nhunt added a commit to nhsuk/connecting-to-services that referenced this issue Mar 16, 2018
Reason is a change to `8.10.0` broke the image. See
[issue](nodejs/docker-node#649) for more
details.
@chorrell
Copy link
Contributor

@chorrell chorrell commented Mar 16, 2018

It's ok, I have a branch with the required changes. Will PR it in a minute. Will test locally as well.

@cecton
Copy link
Contributor

@cecton cecton commented Mar 16, 2018

@chorrell @SimenB please have a look ^_^ I put so much effort into it, I would be glad to contribute to the project.

@florentchauveau
Copy link

@florentchauveau florentchauveau commented Mar 16, 2018

The image node:9.7-alpine can be used as a workaround until this is fixed.

@oyvindwe
Copy link

@oyvindwe oyvindwe commented Mar 16, 2018

A docker image is an artifact of its own, and thus needs it own version. The image version may not need to be a full semver though, only a build increment. node:alpine-8.6.0_16 for the 17th release of node-alpine-8.6.0 might be sufficient. node:alpine-8-6 and node:alpine-8 might still be mutable "tags".

@ojab
Copy link

@ojab ojab commented Mar 16, 2018

As @SimenB already written, image id is immutable identifier of the docker image, i. e. instead of

$ docker pull node:9.8.0-alpine
9.8.0-alpine: Pulling from library/node
Digest: sha256:1fbcd77d0eb2af765d24ae4758b5a94b0d55c6002a15a4431d55795a449ebd3d
Status: Image is up to date for node:9.8.0-alpine

you should use

$ docker pull node@sha256:1fbcd77d0eb2af765d24ae4758b5a94b0d55c6002a15a4431d55795a449ebd3d

@LaurentGoderre
Copy link
Member

@LaurentGoderre LaurentGoderre commented Mar 16, 2018

There's a good reason for tags not being immutable. One of the main benefit of Docker is that you inherit security fixes to underlying images. Sure there's a chance that a change to the base image might break a build but it's a trade-off that I personally gladly take to get the security benefits.

@robinclaes
Copy link

@robinclaes robinclaes commented Mar 16, 2018

Happening to our builds too...!

@oyvindwe
Copy link

@oyvindwe oyvindwe commented Mar 16, 2018

@ojab as I wrote in my first comment, I am perfectly aware how to lock down a version. However, I find the use of “tag” misleading. In version control systems, a tag usually points to a given revision or commit (possibly on a specific branch). When a “tag” is updated, it is more like a VCS branch. The main problem with using the digest notation is that you don’t immediately see which tag it originated from.

Other repositories as Maven, NPM, Yum, etc., contain versioned artifacts. I do not see why Docker should not.

@oyvindwe
Copy link

@oyvindwe oyvindwe commented Mar 16, 2018

@LaurentGoderre There exist syntaxes for specifying that you want the latest version (at some level) for semver. But providing people the possibility to easily lock down the version would be nice. You never know when even a security patch breaks your production system.

@LaurentGoderre
Copy link
Member

@LaurentGoderre LaurentGoderre commented Mar 16, 2018

We are staying consistent with the other docker official images. Maybe there will be a solution to this issue in the future (in the form of a lockfile or something)

@Shahor
Copy link

@Shahor Shahor commented Mar 16, 2018

Hey guys,

thanks for the quick fix <3

That being said we continue having the problem when building our images :(
Is there something peculiar that should be done to get back to work properly?

@SimenB
Copy link
Member

@SimenB SimenB commented Mar 16, 2018

You need to wait for the image to be published. It's currently being built

@oyvindwe
Copy link

@oyvindwe oyvindwe commented Mar 16, 2018

@LaurentGoderre consistency is a good thing. :) Repeatable builds provides even better consistency. My comment is not limited to the node image. Where is the right place to raise this issue?

@CalebKing3
Copy link

@CalebKing3 CalebKing3 commented Mar 16, 2018

I'm happy we were not the only one experiencing this issue 😅 , reverting to a earlier tag was our solution but kudos to the community on this form for providing suggestions.

@cecton
Copy link
Contributor

@cecton cecton commented Mar 16, 2018

I'm happy we were not the only one experiencing this issue 😅 , reverting to a earlier tag was our solution but kudos to the community on this form for providing suggestions.

I think the whole world ended up here when they realized their builds were not passing lol!

@djbingham
Copy link

@djbingham djbingham commented Mar 16, 2018

The fix seems to have made its way through the system. I've just had a GitLab CI build pass using the node:alpine image.

@danielwhatmuff
Copy link

@danielwhatmuff danielwhatmuff commented Mar 19, 2018

@SimenB @LaurentGoderre - PR with changes to prevent this sort of thing happening again - #658

@rarkins
Copy link

@rarkins rarkins commented Mar 19, 2018

I wrote up a blog post to explain how/why this happened, why it's not completely unexpected and can happen again, and what you can do to protect against it in future: https://renovateapp.com/blog/docker-mutable-tags

In summary:

  1. Docker tags can change at any time. They are mutable and that is by design, not fault
  2. Even a Docker tag that looks like semver (e.g. node:8.10.0) still has the properties of a tag and can be changed/updated at any time, like happened here
  3. There are valid reasons for the image a tag points to to be changed (such as refactoring the Dockerfile or a security patch to the base), so it's not a good idea to insist that a tag that looks like a semver should never be changed
  4. The only way to ensure your build is reproducible/consistent is to use Docker digests as the identifier instead of the tag
  5. You can conveniently still include a Docker tag in the digest for human readability, e.g. node:8.10.0-alpine@sha256:a55d3e87802b2a8464b3bfc1f8c3c409f89e9b70a31f1dccce70bd146501f1a0
  6. If you were using digests for your node base images instead of tags, you not only wouldn't have "accidentally" upgraded to the new broken image in the first place, and even if you had updated on purpose then you could have immediately rolled back the digest in your Dockerfile once you discovered the break, instead of waiting hours
  7. Beyond that, these digests are a little "developer-unfriendly" to update by hand, but open source tools to automate it exist

@cecton
Copy link
Contributor

@cecton cecton commented Mar 20, 2018

I believe there is a much simpler solution: if you use Docker for your work, host your own registry and update the images when you need to.

@oyvindwe
Copy link

@oyvindwe oyvindwe commented Mar 20, 2018

@cecton no build time Internet is good for private projects, but not open source development. Also, it does not solve the problems with not having a repeatable build when debugging.

@oyvindwe
Copy link

@oyvindwe oyvindwe commented Mar 20, 2018

That should read: “no build time Internet dependencies”

@cecton
Copy link
Contributor

@cecton cecton commented Mar 20, 2018

Yes actually, you made a good point... I was more thinking about professional projects.

@jwineman
Copy link

@jwineman jwineman commented Jul 24, 2019

Fixed by adding

RUN apk add --no-cache --virtual .build-deps \
    ca-certificates \
    wget \
    tar && \
    cd /usr/local/bin && \
    wget https://yarnpkg.com/latest.tar.gz && \
    tar zvxf latest.tar.gz && \
    ln -s /usr/local/bin/dist/bin/yarn.js /usr/local/bin/yarn.js && \
    apk del .build-deps

from https://hub.docker.com/r/jfyne/node-alpine-yarn/dockerfile to my Dockerfile.

@jstorm31
Copy link

@jstorm31 jstorm31 commented Apr 13, 2021

For me the issue was in using singlequotes in Dockerfile CMD command

CMD ['yarn', 'start'] // Wrong
CMD ["yarn", "start"] // Correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet