feat: move docker images to node v12

[viceice]

Changes

• use yarn base image (includes node v12)
• merge dockerfiles and use multistage builds
• add some more docker labels

Image size (uncompressed)

• slim 586MB -> 561MB
• latest 2.67GB -> 2.64GB

[viceice]

I can try to reduce the image size some more

[viceice]

I'm also thinking of merging the build into one job to reuse build layers even more. se docker can save some build time becuse we share some build stages

[rarkins]

Any changes to base layer for openstack needed?

[viceice]

I don't think so, I've left the user and group changes as is. Only moved them a bit. Maybe that's why the image size increase.

[rarkins]

renovate/yarn probably already contains the extra user and group

[viceice]

true:
https://github.com/renovatebot/docker-base/blob/1d8e4a5e0157c8a7c1b8cb81e83fcd9e5efe145c/Dockerfile#L23-L33

[JamieMagee]

Sorry, I didn't mean to start a full blown review, but apparently commenting from GitHub on Android does this 🤷

Can you add --no-install-recommends to all the apt-get commands? Should also slim down the image a bit

[viceice]

renovatebot/docker-ubuntu#25

[viceice]

strange. shouldn't the following in base image do the no-install-recommends ?

https://github.com/renovatebot/docker-base/blob/1d8e4a5e0157c8a7c1b8cb81e83fcd9e5efe145c/Dockerfile#L8

[JamieMagee]

strange. shouldn't the following in base image do the no-install-recommends ?

Yeah, that would probably do it. I didn't realise that was in the base image 🤦

EDIT: Might also need APT::Get::Install-Suggests "false";?

[rarkins]

strange. shouldn't the following in base image do the no-install-recommends ?

https://github.com/renovatebot/docker-base/blob/1d8e4a5e0157c8a7c1b8cb81e83fcd9e5efe145c/Dockerfile#L8

Is there a way of knowing if it's actually installing recommends?

[viceice]

Yes, we have to check the build logs. apt-get shows what should /recommend /will installled

[viceice]

should we move the latest 3 commits to a new pr?

[viceice]

strange. shouldn't the following in base image do the no-install-recommends ?

Yeah, that would probably do it. I didn't realise that was in the base image 🤦

EDIT: Might also need APT::Get::Install-Suggests "false";?

renovatebot/docker-ubuntu#26

[viceice]

need to fix

[viceice]

#55 55.06 warning Error running install script for optional dependency: "/usr/src/app/node_modules/re2: Command failed.
2020-02-25T07:27:31.6653305Z #55 55.06 Exit code: 1
2020-02-25T07:27:31.6653843Z #55 55.06 Command: node-gyp rebuild
2020-02-25T07:27:31.6654283Z #55 55.06 Arguments: 
2020-02-25T07:27:31.6654612Z #55 55.06 Directory: /usr/src/app/node_modules/re2
2020-02-25T07:27:31.6654945Z #55 55.06 Output:
2020-02-25T07:27:31.6655306Z #55 55.06 gyp info it worked if it ends with ok
2020-02-25T07:27:31.6655934Z #55 55.06 gyp info using node-gyp@5.0.5
2020-02-25T07:27:31.6656219Z #55 55.06 gyp info using node@12.16.1 | linux | x64
2020-02-25T07:27:31.6656524Z #55 55.06 gyp info find Python using Python version 2.7.17 found at \"/usr/bin/python\"
2020-02-25T07:27:31.6657549Z #55 55.06 gyp http GET https://nodejs.org/download/release/v12.16.1/node-v12.16.1-headers.tar.gz
2020-02-25T07:27:31.6658364Z #55 55.06 gyp http 200 https://nodejs.org/download/release/v12.16.1/node-v12.16.1-headers.tar.gz
2020-02-25T07:27:31.6658884Z #55 55.06 gyp http GET https://nodejs.org/download/release/v12.16.1/SHASUMS256.txt
2020-02-25T07:27:31.6659239Z #55 55.06 gyp http 200 https://nodejs.org/download/release/v12.16.1/SHASUMS256.txt
2020-02-25T07:27:31.6659687Z #55 55.06 gyp info spawn /usr/bin/python
2020-02-25T07:27:31.6659969Z #55 55.06 gyp info spawn args [
2020-02-25T07:27:31.6660646Z #55 55.06 gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
2020-02-25T07:27:31.6661257Z #55 55.06 gyp info spawn args   'binding.gyp',
2020-02-25T07:27:31.6662492Z #55 55.06 gyp info spawn args   '-f',
2020-02-25T07:27:31.6663699Z #55 55.06 gyp info spawn args   'make',
2020-02-25T07:27:31.6664498Z #55 55.06 gyp info spawn args   '-I',
2020-02-25T07:27:31.6665125Z #55 55.06 gyp info spawn args   '/usr/src/app/node_modules/re2/build/config.gypi',
2020-02-25T07:27:31.6665737Z #55 55.06 gyp info spawn args   '-I',
2020-02-25T07:27:31.6666323Z #55 55.06 gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
2020-02-25T07:27:31.6832239Z #55 55.06 gyp info spawn args   '-I',
2020-02-25T07:27:31.6879413Z #55 55.06 gyp info spawn args   '/home/ubuntu/.cache/node-gyp/12.16.1/include/node/common.gypi',
2020-02-25T07:27:31.6879774Z #55 55.06 gyp info spawn args   '-Dlibrary=shared_library',
2020-02-25T07:27:31.6880087Z #55 55.06 gyp info spawn args   '-Dvisibility=default',
2020-02-25T07:27:31.6880439Z #55 55.06 gyp info spawn args   '-Dnode_root_dir=/home/ubuntu/.cache/node-gyp/12.16.1',
2020-02-25T07:27:31.6880806Z #55 55.06 gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
2020-02-25T07:27:31.6881183Z #55 55.06 gyp info spawn args   '-Dnode_lib_file=/home/ubuntu/.cache/node-gyp/12.16.1/<(target_arch)/node.lib',
2020-02-25T07:27:31.6881795Z #55 55.06 gyp info spawn args   '-Dmodule_root_dir=/usr/src/app/node_modules/re2',
2020-02-25T07:27:31.6883586Z #55 55.06 gyp info spawn args   '-Dnode_engine=v8',
2020-02-25T07:27:31.6883910Z #55 55.06 gyp info spawn args   '--depth=.',
2020-02-25T07:27:31.6884210Z #55 55.06 gyp info spawn args   '--no-parallel',
2020-02-25T07:27:31.6884516Z #55 55.06 gyp info spawn args   '--generator-output',
2020-02-25T07:27:31.6884803Z #55 55.06 gyp info spawn args   'build',
2020-02-25T07:27:31.6885102Z #55 55.06 gyp info spawn args   '-Goutput_dir=.'
2020-02-25T07:27:31.6885220Z #55 55.06 gyp info spawn args ]
2020-02-25T07:27:31.6885325Z #55 55.06 Traceback (most recent call last):
2020-02-25T07:27:31.6885690Z #55 55.06   File \"/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py\", line 47, in <module>
2020-02-25T07:27:31.6885825Z #55 55.06     import gyp
2020-02-25T07:27:31.6886203Z #55 55.06   File \"/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py\", line 10, in <module>
2020-02-25T07:27:31.6886353Z #55 55.06     import gyp.input
2020-02-25T07:27:31.6886735Z #55 55.06   File \"/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/input.py\", line 7, in <module>
2020-02-25T07:27:31.6886872Z #55 55.06     import ast
2020-02-25T07:27:31.6886992Z #55 55.06 ImportError: No module named ast
2020-02-25T07:27:31.6887097Z #55 55.06 gyp ERR! configure error 
2020-02-25T07:27:31.6887225Z #55 55.06 gyp ERR! stack Error: `gyp` failed with exit code: 1
2020-02-25T07:27:31.6887628Z #55 55.06 gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
2020-02-25T07:27:31.6887779Z #55 55.06 gyp ERR! stack     at ChildProcess.emit (events.js:311:20)
2020-02-25T07:27:31.6887926Z #55 55.06 gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
2020-02-25T07:27:31.6888246Z #55 55.06 gyp ERR! System Linux 5.0.0-1031-azure
2020-02-25T07:27:31.6888633Z #55 55.06 gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"
2020-02-25T07:27:31.6888784Z #55 55.06 gyp ERR! cwd /usr/src/app/node_modules/re2
2020-02-25T07:27:31.6889081Z #55 55.06 gyp ERR! node -v v12.16.1
2020-02-25T07:27:31.6889356Z #55 55.06 gyp ERR! node-gyp -v v5.0.5
2020-02-25T07:27:31.6889478Z #55 55.06 gyp ERR! not ok"
2020-02-25T07:27:31.6889606Z #55 55.06 info This module is OPTIONAL, you can safely ignore this error

🤔

[viceice]

fixed, node-gyp v5 tries to use python executable first, which is python2 on ubuntu. forced it to python3 now

[viceice]

we need to something to make sure re2 is installed. yarn silently drops installation. any ideas? 🤔

[rarkins]

we need to something to make sure re2 is installed. yarn silently drops installation. any ideas? 🤔

Could we add a test/re2-test.js file consisting of require('re2'); and then while in the tsbuild stage we can run node test/re2-test after the yarn install --frozen-lockfile?

[viceice]

putting it to tools, cause they already added to tsbuild image

[viceice]

mjs extension required by node v10

https://nodejs.org/docs/latest-v10.x/api/esm.html

[viceice]

ready

[viceice]

Tested it againt against https://github.com/viceice/renovate-test (dry-run).

I would prefer to merge this back to v20 and finish v20 this week

[rarkins]

[viceice]

They are merged to new job Docker Images CI / Build image (pull_request)

[rarkins]

OK so after merge I need to take them out of the required status checks?