Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using Bit with Zeit Now #1756

Closed
williamluke4 opened this issue Jun 24, 2019 · 35 comments
Closed

Using Bit with Zeit Now #1756

williamluke4 opened this issue Jun 24, 2019 · 35 comments

Comments

@williamluke4
Copy link

Does anyone know how to install public/private components when deploying to Zeit Now.
This is the Error I'm getting as I don't know how to add the bit npm registry


ERR! 404 Not Found: @bit/atto-byte.components.select@0.0.6
--
BUILD | Jun 24 2019 12:47:34:473 | ...next.config.js | npm ERR! A complete log of this run can be found in: npm ERR!     /tmp/.npm/_logs/2019-06-24T09_47_34_456Z-debug.log
BUILD | Jun 24 2019 12:47:34:487 | ...next.config.js | Error: Exited with 1     at ChildProcess.child.on (/tmp/5d9ffe0a/.build-utils/node_modules/@now/build-utils/dist/fs/run-user-scripts.js:30:24)     at emitTwo (events.js:126:13)     at ChildProcess.emit (events.js:214:7)     at maybeClose (internal/child_process.js:925:16)     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)


@itaymendel
Copy link
Contributor

itaymendel commented Jun 24, 2019

Hi,
I have found several issues in the Zeit community that refers to the same problem, and they provide some interesting workarounds.

I have also found this blog post about Zeit's build env feature. From reading this, I assume that you can configure the dockerfile to create an .npmrc file with the configuration to consume components from bit.dev.

This is how we configure the .npmrc file for npm to know how to fetch components from Bit.

echo "@bit:registry=https://node.bit.dev" >> .npmrc
echo "//node.bit.dev/:_authToken={$BIT_NODE_TOKEN}" >> .npmrc

You should be able to configure it through the dockerfile, as shown in the blog post.

@williamluke4
Copy link
Author

Unfortunately, I'm not using a dockerfile I'm using Zeit's @now/next builder.

@paulogdm
Copy link

Kindly take a look at the documentation here.

@williamluke4
Copy link
Author

Hi @paulogdm Yes this is something that I already use as I have a private npm repo! But Bit requires a private npm registry to be added not a private repo

@williamluke4
Copy link
Author

@paulogdm
Copy link

My apologies for the confusion.
Not possible at the moment but it is certainly in our roadmap.

@williamluke4
Copy link
Author

Do you have any workarounds as this has just stopped all progress

@paulogdm
Copy link

A very hacky solution would be to use a "preinstall" script and then "preinstall": "chmod +x run.sh && ./run.sh".
Can't recommend this, since the proper solution will be released in the future.

@williamluke4
Copy link
Author

Thanks, @paulogdm Any chance of an ETA?

@paulogdm
Copy link

Not a priority at the moment, since we are focused on other features. So I would say it is at least a couple of months away.

@williamluke4
Copy link
Author

Ok I'm am using the following shell script

#! /bin/bash
echo "Adding Bit Registery"
echo "@bit:registry=https://node.bit.dev" >> .npmrc
echo "//node.bit.dev/:_authToken={$BIT_NODE_TOKEN}" >> .npmrc

Which runs and the BIT_NODE_TOKEN was taken from my dev pc's .npmrc file. The preinstall runs and picks up the BIT_NODE_TOKEN but I still recive the following error

ERR! 404 Not Found: @bit/atto-byte.bnb.ui.tabs@0.0.2

@itaymendel
Copy link
Contributor

Do you have any idea or a way to validate where the .npmrc file is created?

You can try and create it in the home dir of the account (~/.npmrc). I saw that in Zeit's blog post they use that location.

echo "@bit:registry=https://node.bit.dev" >> ~/.npmrc
echo "//node.bit.dev/:_authToken={$BIT_NODE_TOKEN}" >> ~/.npmrc

Additionally, do you use yarn or npm to fetch dependencies?
If you are using yarn, add this config as well:

echo "always-auth=true" >> ~/.npmrc

@williamluke4
Copy link
Author

npm config ls -l | grep config

Returns this

; cli configs
; environment configs
globalconfig = "/node8/etc/npmrc"
userconfig = "/tmp/.npmrc"
; userconfig /tmp/.npmrc
globalconfig = "/node8/etc/npmrc"
userconfig = "/tmp/.npmrc"

@williamluke4
Copy link
Author

I have tried both ~/.npmrc and /tmp/.npmrc but still no luck

echo "Adding Bit Registery"
echo "@bit:registry=https://node.bit.dev" >> /tmp/.npmrc
echo "//node.bit.dev/:_authToken={$BIT_NODE_TOKEN}" >> /tmp/.npmrc
echo "always-auth=true" >> /tmp/.npmrc
cat /tmp/.npmrc
echo "NPM Config"
npm config ls -l

Output

2019-06-25T10:47:11.142Z  > qg2@0.9.6 preinstall /tmp/31691200
                          > chmod +x bit.sh && ./bit.sh
2019-06-25T10:47:11.152Z  Adding Bit Registery
2019-06-25T10:47:11.153Z  //registry.npmjs.org/:_authToken=MY_NPM_TOKEN@bit:registry=https://node.bit.dev
                          //node.bit.dev/:_authToken={MY_BIT_TOKEN}
                          always-auth=true
2019-06-25T10:47:11.153Z  NPM Config
2019-06-25T10:47:11.448Z  ; cli configs
                          long = true
                          scope = ""
                          user-agent = "npm/5.6.0 node/v8.10.0 linux x64"

                          ; environment configs
                          ; argv = "{\"remain\":[],\"cooked\":[\"install\",\"--unsafe-perm\"],\"original\":[\"install\",\"--unsafe-perm\"]}" (overridden)
                          auth-type = "legacy"
                          bin-links = true
                          cache = "/tmp/.npm"
                          cache-lock-retries = 10
                          cache-lock-stale = 60000
                          cache-lock-wait = 10000
                          cache-max = null
                          cache-min = 10
                          color = true
                          commit-hooks = true
                          depth = null
                          description = true
                          editor = "vi"
                          fetch-retries = 2
                          fetch-retry-factor = 10
                          fetch-retry-maxtimeout = 60000
                          fetch-retry-mintimeout = 10000
                          git = "git"
                          git-tag-version = true
                          globalconfig = "/node8/etc/npmrc"                          globalignorefile = "/node8/etc/npmignore"
                          heading = "npm"
                          init-license = "ISC"
                          init-module = "/tmp/.npm-init.js"                          init-version = "1.0.0"
                          loglevel = "notice"
                          logs-max = 10
                          maxsockets = 50
                          message = "%s"
                          metrics-registry = "https://registry.npmjs.org/"
                          node-gyp = "/node8/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js"
                          node-version = "8.10.0"
                          optional = true
                          package-lock = true
                          prefix = "/node8"
                          progress = true
                          rebuild-bundle = true
                          registry = "https://registry.npmjs.org/"
                          rollback = true
                          save = true
                          save-prefix = "^"
                          scripts-prepend-node-path = "warn-only"
                          searchlimit = 20
                          searchstaleness = 900
                          shell = "bash"
                          shrinkwrap = true
                          sso-poll-frequency = 500
                          sso-type = "oauth"
                          strict-ssl = true
                          tag = "latest"
                          tag-version-prefix = "v"
                          tmp = "/tmp"
                          umask = 18
                          unicode = true
                          unsafe-perm = true
                          user-agent = "npm/5.6.0 node/v8.10.0 linux x64"
                          userconfig = "/tmp/.npmrc"
                          viewer = "man"

                          ; userconfig /tmp/.npmrc
                          always-auth = true

                          ; default values
                          access = null
                          allow-same-version = false
                          also = null
                          ; always-auth = false (overridden)
                          auth-type = "legacy"
                          bin-links = true
                          browser = null
                          ca = null
                          cache = "/tmp/.npm"
                          cache-lock-retries = 10
                          cache-lock-stale = 60000
                          cache-lock-wait = 10000
                          cache-max = null
                          cache-min = 10
                          cafile = undefined
                          cert = null
                          cidr = null
                          color = true
                          commit-hooks = true
                          depth = null
                          description = true
                          dev = false
                          dry-run = false
                          editor = "vi"
                          engine-strict = false
                          fetch-retries = 2
                          fetch-retry-factor = 10
                          fetch-retry-maxtimeout = 60000
                          fetch-retry-mintimeout = 10000
                          force = false
                          git = "git"
                          git-tag-version = true
                          global = false
                          global-style = false
                          globalconfig = "/node8/etc/npmrc"                          globalignorefile = "/node8/etc/npmignore"
                          group = 0
                          ham-it-up = false
                          heading = "npm"
                          https-proxy = null
                          if-present = false
                          ignore-prepublish = false
                          ignore-scripts = false
                          init-author-email = ""
                          init-author-name = ""
                          init-author-url = ""
                          init-license = "ISC"
                          init-module = "/tmp/.npm-init.js"                          init-version = "1.0.0"
                          json = false
                          key = null
                          legacy-bundling = false
                          link = false
                          local-address = undefined
                          loglevel = "notice"
                          logs-max = 10
                          ; long = false (overridden)
                          maxsockets = 50
                          message = "%s"
                          ; metrics-registry = null (overridden)
                          node-options = null
                          node-version = "8.10.0"
                          offline = false
                          onload-script = null
                          only = null
                          optional = true
                          otp = 0
                          package-lock = true
                          package-lock-only = false
                          parseable = false
                          prefer-offline = false
                          prefer-online = false
                          prefix = "/node8"
                          production = false
                          progress = true
                          proxy = null
                          read-only = false
                          rebuild-bundle = true
                          registry = "https://registry.npmjs.org/"
                          rollback = true
                          save = true
                          save-bundle = false
                          save-dev = false
                          save-exact = false
                          save-optional = false
                          save-prefix = "^"
                          save-prod = false
                          scope = ""
                          script-shell = null
                          scripts-prepend-node-path = "warn-only"
                          searchexclude = null
                          searchlimit = 20
                          searchopts = ""
                          searchstaleness = 900
                          send-metrics = false
                          shell = "bash"
                          shrinkwrap = true
                          sign-git-tag = false
                          sso-poll-frequency = 500
                          sso-type = "oauth"
                          strict-ssl = true
                          tag = "latest"
                          tag-version-prefix = "v"
                          timing = false
                          tmp = "/tmp"
                          umask = 18
                          unicode = true
                          ; unsafe-perm = false (overridden)
                          usage = false
                          user = 0
                          ; user-agent = "npm/{npm-version} node/{node-version} {platform} {arch}" (overridden)
                          userconfig = "/tmp/.npmrc"
                          version = false
                          versions = false
                          viewer = "man"

@williamluke4
Copy link
Author

This is the /tmp/.npmrc which pretty much mirrors my local .npmrc file but I'm still getting ERR! 404 Not Found: @bit/atto-byte.bnb.ui.tabs@0.0.2

//registry.npmjs.org/:_authToken=NPM_TOKEN_REMOVED
@bit:registry=https://node.bit.dev
//node.bitsrc.io/:_authToken=BIT_TOKEN_REMOVED
//node.bit.dev/:_authToken=BIT_TOKEN_REMOVED
always-auth=true

@paulogdm Do you have any thoughts?

@paulogdm
Copy link

Let me relay this one.

@styfle
Copy link

styfle commented Jun 26, 2019

The .npmrc file is not observed with ZEIT Now.

Today, there is an environment variable, NPM_TOKEN, that can can be used to install private npm packages published to npmjs.org but this does not change the registry URL.

@williamluke4 If there was support for a new environment variable, something like NPM_REGISTRY_URL, would this solve your problem? Or do we need something more complex to mix bit packages with npmjs.com packages?

@williamluke4
Copy link
Author

@styfle I don't think this would solve the issue as it is a scoped registry (@itaymendel Correct me if I'm wrong). I think the best option, for now, would be to expose the .npmrc to ZEIT Now

@itaymendel
Copy link
Contributor

itaymendel commented Jun 26, 2019

@styfle @williamluke4 specifically for Bit's case it should work, as Bit's registry is also a proxy for npmjs.org... so in case a package is missing in BIt, it falls back to npmjs.org.

I'm not sure how many teams that want to use Zeit use both a private package registry (using jfrog or verdaccio for example), or even use services like fort-awesome... and need an ability to fetch components from several repositories. So perhaps this is something to consider.

@williamluke4
Copy link
Author

williamluke4 commented Jun 27, 2019

If you were to expose the .npmrc file via preinstall you would have to make sure that it only got exposed to the root package.json as this could be an attack vector for malicious packages changing the registry

@williamluke4
Copy link
Author

@styfle Any chance you have a workaround as this has halted development

@styfle
Copy link

styfle commented Jun 28, 2019

It seems like the best solution would be to provide a way for users to define the contents of .npmrc in a secret which could be assigned to the NPM_RC environment variable. That would cover all the bases for mixing and matching registries from npm, Bit, and even GitHub Package Registry.

How does that sound?

@styfle
Copy link

styfle commented Jun 28, 2019

I implemented this feature.

In order to add the secret, you can do the following:

now secrets add my-app-npmrc "$(cat ~/.npmrc)"

Then you'll need to link the secret to the environment variable NPM_RC:

{
  "version": 2,
  "builds": [
    { "src": "index.js", "use": "@now/node" }
  ],
  "build": {
    "env": {
      "NPM_RC": "@my-app-npmrc"
    }
  }
}

I prefix my secrets with the app name but you can change my-app-npmrc to whatever you wish.

However, the env var name,NPM_RC, is a requirement.

@itaymendel
Copy link
Contributor

From the looks of it Zeit now support installing packages from Bit and all other non-nmpjs registries!
I’m closing the issue and will add a reference to this discussion from the docs.

@williamluke4
Copy link
Author

@styfle @itaymendel Is this the correct way to use this or am I missing somthing?

now secrets add bit-npmrc "$(cat ~/.npmrc)\n@bit:registry=https://node.bit.dev\n//node.bitsrc.io/:_authToken=REMOVED\n//node.bit.dev/:_authToken=REMOVED\nalways-auth=true"

@styfle
Copy link

styfle commented Jun 29, 2019

It’s probably best to put the contents in a file instead of dealing with escape sequences in bash.

My example above assumes that your .npmrc file works locally with Bit and you want to add to ZEIT Now secrets.

@williamluke4
Copy link
Author

@styfle @itaymendel Thanks for all the help guys! I got it working, it's not pretty but it works :)

@itaymendel
Copy link
Contributor

awesome!
Can to share the solution you ended up doing? so others that encounter the same issue will have something to start with.

@williamluke4
Copy link
Author

Steps

  1. Ensure that you are using Now V2
  2. Find you bit token in your local .npmrc file
  3. Add NPM_RC to your 'now.json'
{
  "name": "bit",
  "version": 2,
  "builds": [{
    "src": "next.config.js",
    "use": "@now/next"
  }],
  "build": {
    "env": {
      "NPM_RC": "//registry.npmjs.org/:_authToken=YOUR_NPM_TOKEN\n@bit:registry=https://node.bit.dev\n//node.bitsrc.io/:_authToken=YOUR_BIT_TOKEN\n//node.bit.dev/:_authToken=YOUR_BIT_TOKEN"
    }
  }
}
  1. Deploy

Notes

  • Remove //registry.npmjs.org/:_authToken=YOUR_NPM_TOKEN\n if you are not using a private npm repo
  • Remove //node.bitsrc.io/:_authToken=YOUR_BIT_TOKEN\n//node.bit.dev/:_authToken=YOUR_BIT_TOKEN if you are not using a private bit collection
  • You can't have a env variable that begins with a "@" as NOW thinks that you are referencing a secret
    I.e You can't have "NPM_RC": "@bit:registry=https://node.bit.dev"
  • The better way to do this would be to add your NPM_RC string to a Now Secret then reference the secret in your now.json build envs (I did not use this method as it was giving me problems).
    Example
now secrets add npmrc "//registry.npmjs.org/:_authToken=YOUR_NPM_TOKEN\n@bit:registry=https://node.bit.dev\n//node.bitsrc.io/:_authToken=YOUR_BIT_TOKEN\n//node.bit.dev/:_authToken=YOUR_BIT_TOKEN"

now.json

{
  "name": "bit",
  "version": 2,
  "builds": [{
    "src": "next.config.js",
    "use": "@now/next"
  }],
  "build": {
    "env": {
      "NPM_RC": "@npmrc"
    }
  }
}

@konsumer
Copy link

konsumer commented Oct 11, 2019

I am using verdaccio, and can't seem to get this to work with now dev.

I have this in now.json:

{
  "version": 2,
  "builds": [
    {
      "src": "package.json",
      "use": "@now/static-build"
    }
  ],
  "routes": [],
  "env": {},
  "build": {
    "NPM_RC": "@npmrc"
  }
}

I have set the secret with this:

now secrets add npmrc "$(cat ~/.npmrc)"

I also added it to .env.build (so it will work with dev):

NPM_RC="@NAMESPACE:registry=MYVERDACCIOSERVER/\n//MYVERDACCIOSERVER:_authToken=MYLONGTOKEN"

Where NAMESPACE, MYVERDACCIOSERVER, and MYLONGTOKEN are replaced with what's in my .npmrc.

Just to test, I tried adding the same to my now.json, which also couldn't find the module.

With my regular .npmrc, the project works fine, outside of now.

Am I missing something?

@konsumer
Copy link

konsumer commented Oct 11, 2019

Hmm, maybe I screwed something up. just rebuilt the private-npm module in question, double-checked to make make sure it worked outside of now, and set the project up from scratch, and it worked ok with now dev

@GiladShoham
Copy link
Member

There are docs about Zeit in bit's docs here - https://docs.bit.dev/docs/ci#zeit

@konsumer
Copy link

konsumer commented Oct 15, 2019

With verdaccio, I have a huge token that causes this error:

Error! Environment variables must must not be greater than 4kb when JSON-stringified (got 9.81KB)

We are using gitlab auth integration, and it looks like my token is a giant JWT from there. I will look into getting a smaller verdaccio auth token.

I guess this is all very unrelated to bit, though.

@GiladShoham
Copy link
Member

@konsumer
you may want to take a look at this:
https://github.com/teambit/bit/blob/master/e2e/npm-ci-registry.ts
During the e2e tests of bit we are setting a vardaccio server.

@konsumer
Copy link

Thanks! I ended up just putting a .npmrcfile in the dir of the project that needed private npm repo. worked great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants