Please add yarn(awesome npm alternative)

[nodkz]

Will be cool if yarn was integrated to latest node@6 images.
https://yarnpkg.com/

Yarn: A new package manager for JavaScript

[pesho]

It may be a bit premature to decide that yet, but I was thinking about the same yesterday. Given the overwhelmingly positive response to the Yarn release (~9300 GitHub stars in just one day, ~1600 upvotes on HN) this might become a very common request.

I've done a quick test, and adding Yarn to the -slim image (via npm install -g yarn) adds 21.1 MB to the size.

[nodkz]

Current official node image sizes:

node    slim    4d8411ec1363    13 days ago 210.3 MB
node    onbuild 275dae106538    13 days ago 653.9 MB

Yep, Yarp adds 10% to size for node:slim but:

• disk space is a most cheaper thing
• time till run is a most expensive 👟🚀

Agreed that needs to wait some time (a couple of hours 😂) till it became a new standart.

List of most fat packages in yarn. May be some dependencies can be removed. Or bundled with tree-shaking.

6,6M    ./core-js
4,8M    ./lodash
1,7M    ./es5-ext
1,7M    ./node-gyp
928K    ./babel-runtime
548K    ./github

208K    ./bl
244K    ./diff
216K    ./hawk
220K    ./json-schema
244K    ./request
256K    ./sshpk
244K    ./tough-cookie

156K    ./agent-base
168K    ./babel-types
104K    ./dashdash
116K    ./fstream
124K    ./gauge
104K    ./har-validator
112K    ./hoek
100K    ./jodid25519
172K    ./mime-db
168K    ./node-emoji
176K    ./readable-stream
104K    ./tar
184K    ./tweetnacl

[chorrell]

It's probably something we could add in the future if there was enough demand for it, but I'm pretty ambivalent about it at this point since it's so new. I'd prefer to keep the images fairly simple, especially the slim variant, since the main point is to use it as a base image and install dependencies etc on top of it.

For installation, I would be more inclined to use the Yarn Debian package repository (under Linux):

https://yarnpkg.com/en/docs/install

That more closely follows the best practices for official images (pgp signed packages):

https://github.com/docker-library/official-images#image-build

[pesho]

For installation, I would be more inclined to use the Yarn Debian package repository (under Linux):

https://yarnpkg.com/en/docs/install

That more closely follows the best practices for official images (pgp signed packages):

https://github.com/docker-library/official-images#image-build

Tried that yesterday. Currently doesn't work on our images, because their packages depend on an installed recent nodejs DEB package (which should probably be relaxed to Recommends or Suggests).

[chorrell]

Ugh, ok. Maybe at some point they'll pgp sign the tarball.

[pesho]

It looks like there may be soon a ~ 2 MB bundled release which is CLI-compatible: yarnpkg/yarn#888. If that works, I'm definitely +1 on including it.

[Daniel15]

Currently doesn't work on our images, because their packages depend on an installed recent nodejs DEB package (which should probably be relaxed to Recommends or Suggests).

This would break the out-of-the-box experience for people that are just getting started. Without Node.js being installed, apt-get install yarn then yarn will show an error about nodejs not being found.

Most people are on Ubuntu 16.04 which has Node.js 4.2.6 in the repo, and the NodeSource repo can be added to get Node 6.

[pesho]

Without Node.js being installed

The absence of a nodejs package does not indicate that Node.js is not installed. There are many popular alternative installation methods, such as nvm, nave, n, make install, binary tarballs, and so on.

[pesho]

@Daniel15 a Recommends relation would also not break the out-of-the-box experience, as recommended packages are installed by default on both Ubuntu and Debian. But it would allow those who have installed Node in a different way to opt out of the DEB package.

[Fishrock123]

Are we talking about shipping with Yarn? Because that's not a thing in core yet and I'm not sure that is a good idea at this time.

[Starefossen]

@Fishrock123 I don't think any of Docker WG members are suggesting we should add Yarn at this point, but that might change if Yarn becomes a popular request by the masses.

[chorrell]

Yes, that's my understanding too: this is just about adding yarn to the docker images.

[andrewmclagan]

Any timeline for this?

[Starefossen]

@andrewmclagan at them moment no, there is a similar discussion over at nodejs/node#9161.

[jeremyzahner]

If you guys would tag your releases on here we could fork the repo and use our custom images (including yarn). But since you don't, it's pretty hard to establish a workflow based on your official image (which is sad).

[Starefossen]

@jeremyzahner This is how you can use the official image with yarn – today:

FROM node:6.9.1
RUN npm install -g yarn

...

I agree that it would be more convenient to have it pre installed, but it is not a blocker from basing on the official image.

[chorrell]

@jeremyzahner I don't understand your "tag your releases" comment. Can you elaborate? That is, I don't understand how this fits with your desired workflow.

[Starefossen]

The issue would be to run one-off commands like this since yarn is not installed by default:

$ docker run --rm node:argon yarn install

[Daniel15]

Currently doesn't work on our images, because their packages depend on an installed recent nodejs DEB package (which should probably be relaxed to Recommends or Suggests).

This has been fixed, by the way. The Debian package should be suitable for you to use now 😄 The repo is GPG signed.

[kachkaev]

Hey guys, would it be possible to prioritize this issue somehow? It is the most 👍-d in the whole project at the moment :–)

I would love to use yarn in GitLab CI to test and build react apps. Today's lack of an official node-yarn image keeps me and others on NPM. I know it is possible to use yarn locally and then NPM in CI, but that feels a bit dangerous. A couple of unofficial images exist and can help, but they a probably not the best choice for anything serious.

[Starefossen]

There is nothing preventing you from making your own image and publishing it to Docker Hub if you want to use yarn today:

FROM node:6-alpine

ENV YARN_VERSION 0.20.0

ADD https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v${YARN_VERSION}.tar.gz /opt/yarn.tar.gz

RUN yarnDirectory=/opt/yarn && \
    mkdir -p "$yarnDirectory" && \
    tar -xzf /opt/yarn.tar.gz -C "$yarnDirectory" && \
    ln -s "$yarnDirectory/dist/bin/yarn" /usr/local/bin/ && \
    rm /opt/yarn.tar.gz

As it stands today yarn is not an official part of Node.js, we do not have tests etc. to make sure that we maintain compatibility with yarn from version to version. Yarn is not yet reached a v1.0 stability level which makes breaking changes a problem.

[jeremyzahner]

@kachkaev We already did so: https://hub.docker.com/r/jshmrtn/node-yarn/

[kachkaev]

Good news! There is now an image by yarn developers:
https://hub.docker.com/r/yarnpkg/node-yarn/

There are not that many tags at the moment (e.g. alpine version is missing), but this is still a great step forward! Related PR: yarnpkg/yarn#2721

Only after @Starefossen replied to my comment I realised that this issue is in nodejs/docker-node repo instead of docker-library/official-images or similar. Now I also think that this is probably not the best place to ask for yarn support in docker, as that package manager comes from a completely different source than node+npm. @nodkz how about closing the issue here and continuing the conversation in another repo, e.g. yarnpkg/node-yarn?

[Daniel15]

There are not that many tags at the moment (e.g. alpine version is missing)

The current node-yarn image is based off Debian Jessie, not Alpine. I can add an Alpine image if there's demand for it, but generally I haven't seen a lot of demand as most people are already using other Debian/Ubuntu packages as part of their build system anyways.

[nodkz]

My current solution for building docker image for production (good hint for starting your own). This solution is used a half year and rewritten several times.

Run yarn build-docker, which calculates hash of package.json and yarn.lock and, if image does not exists, creates node-modules image app-kz/node-modules:production-[HASH] (if image exists, skip this build step for speed). After that used this node-modules image for creation app image with name app-kz/production:170214-1357-30c37cc ([YYMMDD]-[hhmm]-[commitHash])

package.json

{
  ...
  "scripts": {
    "build-docker": "BABEL_ENV=build ./node_modules/.bin/babel-node ./bin/run deploy/docker/buildAppImage",
  },
  ...

run.js

Script which runs other js files with some logging info

import chalk from 'chalk';

function format(time) {
  return time.toTimeString().replace(/.*(\d{2}:\d{2}:\d{2}).*/, '$1');
}

async function run(fn, options) {
  const start = new Date();
  console.log(chalk.bold.yellow(`[${format(start)}] Starting '${fn.name}'...`));

  let result;
  let hasError = false;
  try {
    result = await fn(options);
  } catch (err) {
    console.log(chalk.bgRed.bold.white(`Failed '${fn.name}': ${err.message}`));
    console.error(err.stack);
    hasError = true;
  }

  const finish = new Date();
  const time = finish.getTime() - start.getTime();
  if (hasError) {
    console.log(chalk.bgRed.bold.white(
      `[${format(finish)}] Finished '${fn.name}' after ${time} ms`
    ));
  } else {
    console.log(chalk.bold.green(
      `[${format(finish)}] Finished '${fn.name}' after ${time} ms`
    ));
  }
  return result;
}

if (process.mainModule.children.length === 0 && process.argv.length > 2) {
  delete require.cache[__filename];
  const module = require(`./${process.argv[2]}.js`).default;
  run(module);
}

module.exports = run;
module.exports.default = run;

config.js

import path from 'path';
import cp from 'child_process';

export const regionECR = 'eu-west-1';
export const remoteRepo = `000000000000.dkr.ecr.${regionECR}.amazonaws.com`;
export const repositoryName = 'repo-kz';
export const rootDir = path.resolve(__dirname, '../../');
export const buildEnv = 'production';
export const webpackBuildDir = path.resolve(rootDir, `./build/${buildEnv}`);

export function isDockerImageExists(imageNameWithTag) {
  const imageId = cp.execSync(`docker images -q ${imageNameWithTag}`).toString();
  return imageId && imageId.length > 0;
}

export function getContainerInfo(imageNameWithTag) {
  return cp.execSync(`docker images ${imageNameWithTag}`).toString();
}

buildNodeImage.js

/* eslint-disable no-use-before-define, camelcase, no-multi-str */

import cp from 'child_process';
import fsExtra from 'fs-extra';
import hashFiles from 'hash-files';
import chalk from 'chalk';
import {
  repositoryName,
  rootDir,
  buildEnv,
  isDockerImageExists,
  getContainerInfo,
} from '../config';

const nodeVersion = '6.9.4';
const nodeSlimImage = `node:${nodeVersion}-slim`;
const nodeBuildImage = `node:${nodeVersion}-onbuild`;
const imageName = `${repositoryName}/node-modules`;
const ramDiskName = `docker_${new Date().toJSON().slice(0, 19).replace(/[^0-9]+/g, '')}`;
const dockerContextFolder = `/Volumes/${ramDiskName}`;
console.log(imageName);

export default function build_DockerContainer_With_NodeModules() {
  const imageTag = calcPackageHash(buildEnv);
  const imageNameWithTag = `${imageName}:${imageTag}`;

  if (!isDockerImageExists(imageNameWithTag)) {
    try {
      console.log(`Mounting RAM disk ${ramDiskName}`);
      mountRamDisk();

      console.log(
        chalk.magenta('Installing node_modules for debian '
                    + `via Docker container (${nodeBuildImage} 650MB) to the RAM disk`)
      );
      npmInstallViaFatContainer(buildEnv);

      console.log(chalk.magenta(`Creating slim docker image (from ${nodeSlimImage}): ${imageNameWithTag}...`));
      createDockerContainer(imageNameWithTag);
    } finally {
      console.log(`Unmounting RAM disk ${ramDiskName}`);
      unmountRamDisk();
    }

    console.log(chalk.magenta('Docker container with NODE_MODULES successfully builded.'));
  } else {
    console.log(chalk.magenta('Docker container with NODE_MODULES already exists.'));
  }
  console.log(chalk.magenta(getContainerInfo(imageNameWithTag)));
  console.log(chalk.magenta('You can run container via command:'));
  console.log(chalk.bold.magenta(`docker run -i -t --rm ${imageNameWithTag} /bin/bash`));

  return {
    name: imageName,
    tag: imageTag,
    nameWithTag: imageNameWithTag,
  };
}

function createDockerContainer(imageNameWithTag) {
  const dockerfile = [
    `FROM ${nodeSlimImage}`, // 210MB container only with node
    'WORKDIR /src',
    'COPY . /src',
  ];
  fsExtra.outputFileSync(`${dockerContextFolder}/Dockerfile`, dockerfile.join('\n'));

  cp.execSync(`docker build \
    -t ${imageNameWithTag} \
    ${dockerContextFolder}`,
    {
      cwd: dockerContextFolder,
      stdio: [0, 1, 2],
    }
  );
}

function npmInstallViaFatContainer() {
  fsExtra.emptyDirSync(dockerContextFolder);
  fsExtra.copySync(`${rootDir}/package.json`, `${dockerContextFolder}/package.json`);
  fsExtra.copySync(`${rootDir}/yarn.lock`, `${dockerContextFolder}/yarn.lock`);
  const npmInstallCmd = (
    'wget https://yarnpkg.com/install.sh && chmod +x install.sh && ./install.sh --nightly && rm -f install.sh '
      + '&& PATH=$PATH:~/.yarn/bin/ ' // add yarn to paths
      + '&& yarn add string-width --production ' // needed for build bcrypt module and runtime
      + '&& yarn install --production ' // install app modules
      + '&& rm -rf ~/.yarn ' // remove cached modules and yarn itself for reducing container size
  );

  // IMPORTANT: `node:onbuild` is 650MB container with build scripts
  // needed for build `fsevent` and `crypto` bin-files for debian
  cp.execSync(`docker run \
             -t --rm -v ${dockerContextFolder}:/src \
             ${nodeBuildImage} \
             /bin/bash -c "cd /src && ${npmInstallCmd}"`,
    {
      cwd: dockerContextFolder,
      stdio: [0, 1, 2],
    }
  );
}

function calcPackageHash(env) {
  const hash = hashFiles.sync({
    files: [
      `${rootDir}/package.json`,
      `${rootDir}/yarn.lock`,
    ],
  });

  return `${env}-${hash}`;
}

function mountRamDisk() {
  cp.execSync(`diskutil erasevolume hfsx ${ramDiskName} \`hdiutil attach -nomount ram://1200000\``,
    { stdio: [0, 1, 2] }
  );
}

function unmountRamDisk() {
  cp.execSync(`diskutil unmountDisk force ${ramDiskName}`,
    { stdio: [0, 1, 2] }
  );
}

buildAppImage.js

/* eslint-disable no-use-before-define, camelcase */

import cp from 'child_process';
import del from 'del';
import fs from 'fs';
import fsExtra from 'fs-extra';
import chalk from 'chalk';
import run from '../../run';
import buildNodeImage from './buildNodeImage';
import {
  repositoryName,
  buildEnv,
  webpackBuildDir,
  isDockerImageExists,
  getContainerInfo,
} from '../config';

const imageName = `${repositoryName}/${buildEnv}`;

export default async function build_DockerContainer_With_APP() {
  let buildDirStat = fs.statSync(webpackBuildDir);
  if (!buildDirStat) {
    cp.execSync('yarn build', { cwd: webpackBuildDir, stdio: [0, 1, 2] });
  }

  buildDirStat = fs.statSync(webpackBuildDir);
  if (!buildDirStat) {
    console.log(chalk.magenta.bold('Firstly bundle app via command: `yarn build`'));
    throw new Error(`Webpack folder with bundled code does not exist: ${webpackBuildDir}`);
  }

  const nodeImage = await run(buildNodeImage);
  if (!nodeImage) {
    throw new Error('Cannot build app container. NodeModules container not provided.');
  }

  const imageTagFallback = buildDirStat
                    .mtime
                    .toJSON()
                    .slice(0, 19)
                    .replace('T', '-')
                    .replace(/[^-0-9]+/g, '');
  const revision = fs.readFileSync(`${webpackBuildDir}/revision.txt`, 'utf8').toString().trim();
  const imageTag = revision || imageTagFallback;
  const imageNameWithTag = `${imageName}:${imageTag}`;

  console.log(`Creating app docker image: ${imageNameWithTag}...`);

  if (!isDockerImageExists(imageNameWithTag)) {
    console.log('Remove source map files for avoiding vulnerability of code');
    del.sync([`${webpackBuildDir}/public/**/*.map`]);

    createDockerfile(nodeImage.nameWithTag);
    cp.execSync(`docker build \
      -t ${imageNameWithTag} \
      ${webpackBuildDir}`,
      { cwd: webpackBuildDir, stdio: [0, 1, 2] });

    console.log(chalk.magenta('Docker container successfully builded.'));
  } else {
    console.log(chalk.magenta('Docker container already exists.'));
  }
  console.log(chalk.magenta(getContainerInfo(imageNameWithTag)));
  console.log(chalk.magenta('Command for enter to container:'));
  console.log(chalk.bold.magenta(`docker run -i -t --rm ${imageNameWithTag} /bin/bash`));
  console.log(chalk.magenta('Command to start server on port 5000:'));
  console.log(chalk.bold.magenta(`docker run -i --rm -p 5000:5000 ${imageNameWithTag}`));

  return {
    name: imageName,
    tag: imageTag,
    nameWithTag: imageNameWithTag,
  };
}

function createDockerfile(modulesImage) {
  const dockerfile = [
    `FROM ${modulesImage}`,
    'WORKDIR /src',
    'COPY . /src',
    'EXPOSE 5000',
    'CMD ["node", "/src/server.js"]',
  ];
  const filename = `${webpackBuildDir}/Dockerfile`;
  fsExtra.outputFileSync(filename, dockerfile.join('\n'));

  return filename;
}

[SimenB]

Someone recently built an Alpine package for Yarn; see yarnpkg/yarn#1326

That package doesn't really help, as that package will install its own version of Node, ignoring the already installed one.

Nice to see some movement here, though! 😄

[Daniel15]

@pesho - .js release files are now signed, I've signed all previous releases too. 👍

[pesho]

New proposal PR created: #337

[chorrell]

The latest version of yarn is now being added via #337

Here's the Docker Hub PR: docker-library/official-images#2703

[kachkaev]

Looks like yarn was added to all official node images – perfect! README on Docker Hub does not mention yarn so it may be confusing, but the recipes themselves do include it! 🎉 🎉 🎉

[Starefossen]

Yes, there is a pending PR to update documentation.

[styfle]

@Starefossen This still seems to be missing from the docs.

I was surprised when I looked at the environment variables for a recent node app to see "YARN_VERSION":"0.21.3". I checked the docs on Docker Hub and there was nothing mentioning Yarn. Eventually I found this issue.

Which PR is supposed to add the docs?

[Starefossen]

@styfle #345 added this to the README of this repository, I'll look into why that has not been propagated onto Docker Hub. Thank you 🙂

[romandrahan]

Why yarn version is so much outdated? It's still 0.24.6 while latest published is already 0.27.5, which has many minor bug fixes.

[SimenB]

Because of a miscommunication in when to update it. Next release will have it updated

[styfle]

@Starefossen @SimenB Any idea why the readme on Docker Hub is still outdated*?

*Notice no mention of yarn

[SimenB]

I don't know how that file is updated...

[chorrell]

It requires a separate PR to https://github.com/docker-library/docs

They don't automatically pull in our README.md or docs unfortunately.

[styfle]

@chorrell Thanks, I submitted a PR in docker-library/docs#964

[SimenB]

@romandragan FWIW 8.2.0 will have a updated Yarn installation. We'll update the others when a new version of node is released for them