Install private npm modules while using cloud-builders/npm

[jrbaudin]

Hello 👋

I'm having issues installing private npm modules while using Cloud Build. After lots of googling I haven't found any resonable solutions.

I wonder if anyone has any insights into what might be causing this and how to solve it

cloudbuild.yaml

steps:
- name: 'gcr.io/cloud-builders/npm'
  id: 'install'
  env: ['NPM_TOKEN=${_NPM_TOKEN}']
  args: ['install']
- name: 'gcr.io/cloud-builders/npm'
  id: 'test'
  env: ['NPM_TOKEN=${_NPM_TOKEN}']
  args: ['run', 'test']
- name: 'gcr.io/cloud-builders/docker'
  id: 'build'
  args: [
    'build',
    '--build-arg',
    'npmtoken=${_NPM_TOKEN}',
    '-t',
    'eu.gcr.io/$PROJECT_ID/$REPO_NAME:$SHORT_SHA',
    '.']
substitutions:
    _NPM_TOKEN: _this_is_actually_sent_in_via_the_build_trigger_config_
images: ['eu.gcr.io/$PROJECT_ID/$REPO_NAME:$SHORT_SHA']

(project specific) .npmrc

//registry.npmjs.org/:_authToken=${NPM_TOKEN}

Logs from Cloud Build

Starting build "b2972c49-bfa9-4036-b9cc-sdgdsfh64631"

FETCHSOURCE
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/{censored}-207616/r/github-{censored}
* branch bb8aa7aa806eed6d1340548b378a31c7664c52d5 -> FETCH_HEAD
HEAD is now at bb8aa7a Reverted to npm cloud builder
BUILD
Starting Step #0 - "install"
Step #0 - "install": Already have image (with digest): gcr.io/cloud-builders/npm
Step #0 - "install": npm WARN deprecated istanbul-lib-hook@1.2.1: 1.2.0 should have been a major version bump
Step #0 - "install": npm WARN deprecated @types/winston@2.4.4: This is a stub types definition. winston provides its own type definitions, so you do not need this installed.
Step #0 - "install": npm WARN notice [SECURITY] lodash has the following vulnerability: 1 low. Go here for more details: https://nodesecurity.io/advisories?search=lodash&version=4.13.1 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info.
Step #0 - "install": npm WARN deprecated npmconf@2.1.1: this package has been reintegrated into npm and is now out of date with respect to npm
Step #0 - "install": npm WARN notice [SECURITY] request has the following vulnerability: 1 moderate. Go here for more details: https://nodesecurity.io/advisories?search=request&version=2.42.0 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info.
Step #0 - "install": npm WARN deprecated npmconf@2.0.9: this package has been reintegrated into npm and is now out of date with respect to npm
Step #0 - "install": npm WARN deprecated graceful-fs@3.0.11: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
Step #0 - "install": npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
Step #0 - "install": npm WARN deprecated hoek@0.9.1: The major version is no longer supported. Please update to 4.x or newer
Step #0 - "install": npm ERR! code E404
Step #0 - "install": npm ERR! 404 Not Found: {{the private package names is supposed to be here, i've removed it}}
Step #0 - "install": 
Step #0 - "install": npm ERR! A complete log of this run can be found in:
Step #0 - "install": npm ERR! /builder/home/.npm/_logs/2018-08-31T14_45_30_705Z-debug.log
Finished Step #0 - "install"
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/npm" failed: exit status 1

This is not an issue while building the Docker container and I'm to unfamiliar with the Cloud Build structure to see any solution.

Dockerfile

FROM node:alpine

ARG port=4000
ARG npmtoken

RUN mkdir -p /code
WORKDIR /code
ADD . /code

RUN echo "//registry.npmjs.org/:_authToken=$npmtoken" >> ~/.npmrc

RUN yarn && \
  yarn run build && \
  yarn cache clean

EXPOSE $port
CMD [ "yarn", "start" ]

I've tried experimenting with using node:alpine instead of gcr.io/cloud-builders/npm but no change.

To me It seems the project specific .npmrc doesn't work in the cloud build infrastructure (...or something).

Thankful for any guidance!

[Philmod]

Hey Joel,

I've never played with npm private modules unfortunately.
One thing I've done in the past is using private GitHub repositories in a package.json, using this technique

On the other side, I also wrote this article about publishing npm modules from Cloud Build, but I don't think that's the right technique for your use case.

[jrbaudin]

@Philmod Thanks for the reply! Unfortunately, as you say, doesn't seem to fit with my current issue :(

[jrbaudin]

@Philmod When doing some more thinking I do believe your approach for publishing npm modules is the correct also for my use case (seeing as installing private modules is also an authed npm action).

So I've followed your guide and updated my cloudbuild.yaml to:

steps:
- name: 'gcr.io/cloud-builders/gcloud'
  id: 'decrypt npmrc'
  args:
  - kms
  - decrypt
  - --ciphertext-file=npmrc.enc
  - --plaintext-file=/root/.npmrc
  - --location=global
  - --keyring=my-keyring
  - --key=npm-key
  volumes:
    - name: 'home'
      path: /root/
- name: 'gcr.io/cloud-builders/npm'
  id: 'install'
  args: ['install']
  env:
  - HOME=/root/
  volumes:
  - name: 'home'
    path: /root/
- name: 'gcr.io/cloud-builders/npm'
  id: 'test'
  args: ['run', 'test']
- name: 'gcr.io/cloud-builders/docker'
  id: 'build'
  args: [
    'build',
    '--build-arg',
    'npmtoken=${_NPM_TOKEN}',
    '-t',
    'eu.gcr.io/$PROJECT_ID/$REPO_NAME:$SHORT_SHA',
    '.']
#- name: 'gcr.io/cloud-builders/gcloud'
  #args: ['app', 'deploy']
substitutions:
    _NPM_TOKEN: ___actual_npm_token_here___
images: ['eu.gcr.io/$PROJECT_ID/$REPO_NAME:$SHORT_SHA']

However I'm getting issues from Cloud Build (please note: I've added the Cloud KMS CryptoKey Decrypter role to the Cloud Build service account)

starting build "c92c68a3-bc27-4a63-9b75-c8223b53658c"

FETCHSOURCE
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/{censored}-207616/r/github-{censored}
* branch 1ae02f97d395e7acd5fa5077982c7f58b6ef2a6f -> FETCH_HEAD
HEAD is now at 1ae02f9 Reverted to using npm
BUILD
Starting Step #0 - "decrypt npmrc"
Step #0 - "decrypt npmrc": Already have image (with digest): gcr.io/cloud-builders/gcloud
Step #0 - "decrypt npmrc": ERROR: (gcloud.kms.decrypt) Failed to read ciphertext file [npmrc.enc]: Unable to read file [npmrc.enc]: [Errno 2] No such file or directory: 'npmrc.enc'
Finished Step #0 - "decrypt npmrc"
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/gcloud" failed: exit status 1

[jrbaudin]

Oh! Nevermind 🙈 I didn't get that a .enc file was created and I needed to add it to the repo. It works now!