Docker environment to run licensed from command line: "pip.rb:48: syntax error, unexpected '.' (SyntaxError)"

[cnaslain]

Hi,

I've been asked to evaluate this tool in a POC for an internal application. Licensed; looks nice and interesting. I'm not a ruby developer/user, more someone from the CI/CD team, and our app is not written in ruby but uses npm as package manager. I plan to integrate a licensed check step into our Jenkins pipeline and instead of installing licensed in our CI nodes, I choose to build a docker image so I can mount our code and run licensed into a docker container.

I first start with creating a Docker image from the official ruby image (as licensed was written in ruby); here is my Dockerfile:

FROM ruby:2.3.7-jessie
RUN echo deb http://ftp.debian.org/debian jessie-backports main >> /etc/apt/sources.list
RUN apt-get update && \
    apt-get install -y cmake pkg-config npm nodejs
RUN gem install licensed
RUN useradd --no-log-init --create-home -r -u 1000 -g users test
RUN mkdir /app && \
    chown test:users /app
USER test
WORKDIR /app

Build the image:

docker build --rm -t licensed:ruby-2.3.7 .

Here is my licensed app config: .licensed.yaml

name: 'placement-back'
cache_path: '/app/test1/.licenses'
source_path: '/app/test1'
sources:
  npm: true
allowed:
  - mit

... and my package.json:

{
  "name": "placement-back",
  "version": "0.0.0",
  "dependencies": {
    "async": "^2.6.0",
    "atomic": "0.0.2"
  }
}

Then I can mount the app and run licensed; here is a simple example:

docker run -it --rm --network="host" -v /home/chris/workspace/app:/app licensed:ruby-2.3.7 bash
$ cd /app/test1
$ npm update
atomic@0.0.2 node_modules/atomic
async@2.6.1 node_modules/async
└── lodash@4.17.10
$ licensed cache
Caching licenses for placement-back:
  npm dependencies:
    Caching async (2.6.1)
    Caching lodash (4.17.10)
    Caching atomic (0.0.2)
License caching complete!
* placement-back npm dependencies: 3
$ licensed status
Checking licenses for placement-back: 3 dependencies
...
3 dependencies checked, 0 warnings found.

$ uname -a
Linux localhost.localdomain 4.15.14-1.el7.elrepo.x86_64 #1 SMP Wed Mar 28 14:16:59 EDT 2018 x86_64 GNU/Linux
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian

This worked fine with this simple test1 example. Great!

My problems comes when I tried to use the real package.json from my app, having something 40+ main dependencies (approx 500 dependencies in total after running npm). When I run npm update with this json file; npm returns errors. I asked the developers for help and they told me that the npm engine that I used was too old; they requires a npm 8.9.3 version. This first docker image have a npm 1.4.21 version.

Instead of creating the docker image from ruby; I thought that it should be easier to start with a nodejs image to avoid having the npm issue.

Here is my second Docker image from node:

FROM node:8.11.1
RUN echo deb http://ftp.debian.org/debian jessie-backports main >> /etc/apt/sources.list
RUN apt-get update && \
    apt-get install -y cmake pkg-config ruby ruby-dev python-pip
RUN gem install licensed
RUN mkdir /app && \
    chown node:users /app
USER node
WORKDIR /app

Build the image:

docker build -f Dockerfile.node --rm -t licensed:node-8.11 .

Then I run the same test with this new image:

docker run -it --rm --network="host" -v /home/chris/workspace/licensed:/app licensed:node-8.11 bash
$ cd /app/test1
$ npm update
npm WARN placement-back@0.0.0 No description
npm WARN placement-back@0.0.0 No repository field.
npm WARN placement-back@0.0.0 No license field.

+ atomic@0.0.2
+ async@2.6.1
added 3 packages in 4.474s
$ licensed cache
/usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': /var/lib/gems/2.1.0/gems/licensed-1.1.0/lib/licensed/source/pip.rb:48: syntax error, unexpected '.' (SyntaxError)
          a[k.strip] = v&.strip
                          ^
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /var/lib/gems/2.1.0/gems/licensed-1.1.0/lib/licensed.rb:14:in `<top (required)>'
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /var/lib/gems/2.1.0/gems/licensed-1.1.0/lib/licensed/cli.rb:2:in `<top (required)>'
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /var/lib/gems/2.1.0/gems/licensed-1.1.0/exe/licensed:4:in `<top (required)>'
	from /usr/local/bin/licensed:23:in `load'
	from /usr/local/bin/licensed:23:in `<main>'

Licensed failed (even a licensed help command fails).

Here are some additional information regarding config & installed gems:

$ uname -a
Linux localhost.localdomain 4.15.14-1.el7.elrepo.x86_64 #1 SMP Wed Mar 28 14:16:59 EDT 2018 x86_64 GNU/Linux
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
$ gem list --local

*** LOCAL GEMS ***

addressable (2.5.2)
bigdecimal (1.2.4)
faraday (0.15.2)
io-console (0.4.2)
json (1.8.1)
licensed (1.1.0)
licensee (9.6.0)
minitest (4.7.5)
multipart-post (2.0.0)
octokit (4.9.0)
pathname-common_prefix (0.0.1)
psych (2.0.5)
public_suffix (3.0.2)
rake (10.1.0)
rdoc (4.1.0)
rugged (0.27.1)
sawyer (0.8.1)
test-unit (2.1.5.0)
thor (0.20.0)
tomlrb (1.2.7)

I don't know what exactly is missing in this docker image to run licensed. I've tried install python-pip and pip gem; and start building licensed from source but it does not fix this and it was not part of the 1st ruby image... still the same error.

Help/idea appreciated ^^

Regards,

Chris

[jonabc]

@cnaslain 👋 sorry for the difficulties.

The issue with your second docker image is that the version of ruby is a bit old and doesn't support the & operator, causing the syntax error.

There are two ways to fix this.

If you want to use a ruby base docker image, you can install different versions of nodejs in your docker image by following https://github.com/nodesource/distributions#installation-instructions. It looks like you'd want the 8.x installer.

If you want to use a node base docker image, you can install more recent versions of ruby installing and using ruby-install. You'll need Ruby >= 2.3.0 I think.

Let me know if you have any other questions 🙇

[mlinksva]

Should licensed.gemspec include https://guides.rubygems.org/specification-reference/#required_ruby_version so that gem install licensed would fail and say why?

[cnaslain]

@jonabc I've tried the official ruby image + custom setup for nodejs; works like a charm. Thanks!
PS: I also tried the nodejs + manually install ruby; it works too but the docker image is much bigger and more complicated to build; so starting by ruby is the best choice.

For those who are interested in using docker:

FROM ruby:2.4.4-jessie
RUN echo deb http://ftp.debian.org/debian jessie-backports main >> /etc/apt/sources.list
RUN apt-get update &&
apt-get install -y cmake pkg-config
RUN gem install licensed
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - &&
apt-get install -y nodejs
RUN useradd --no-log-init --create-home -r -u 1000 -g users test
RUN mkdir /app &&
chown test:users /app
USER test
WORKDIR /app

[jonabc]

@cnaslain if you're still evaluating licensed, this scenario should be a bit simpler. licensed executables for darwin/mac and linux are shipped as of the 1.2.0 release.

You should be able to curl the executable directly into a docker container set up for your project - no need to use a base ruby docker image or install ruby just to use licensed.