New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to build Docker image including @pnp/generator-spfx fails #149

Closed
pkskelly opened this Issue Feb 1, 2019 · 5 comments

Comments

Projects
None yet
2 participants
@pkskelly
Copy link

pkskelly commented Feb 1, 2019

General Information

  • Usage
  • Development
  • Documentation
  • Feature Request

Specific generator

  • Addons
  • HandlebarsJS

What Version you are running?

  • NodeJS: 8.11.3 (docker base image)
  • NPM: 5.6.0 (docker base image)
  • @pnp/spfx: 1.6.1 . (@pnp/generator-spfx@1.6.1)

Also tried Dockerfile FROM node:10.10.0

  • NodeJS: 10.10.0 (docker base image)
  • NPM: 6.4.1 (docker base image)
  • @pnp/spfx: 1.6.1 . (@pnp/generator-spfx@1.6.1)

Describe your problem

When attempting to build a Docker image using the current 1.6.1 build, see Dockerfile below, the image creation fails with what looks to be issues with access issues to the file system for the user. output of build.sh below as well.

Removing the @pnp/generator-spfx@1.6.1 install completes image creation, and SPFx project can be created from running container instance.

Dockerfile:

FROM node:10.10.0
#orignally started with 8.11.3
LABEL maintainer="Pete Skelly <peteskelly@gmail.com>"
EXPOSE 5432 4321 35729

RUN npm i -g gulp@3.9.1 yo@2.0.5 @microsoft/generator-sharepoint@1.7.1 @pnp/office365-cli@next @pnp/generator-spfx@1.6.1 && \
   npm cache verify
VOLUME /usr/app/spfx
WORKDIR /usr/app/spfx
RUN useradd --create-home --shell /bin/bash spfx && \
    usermod -aG sudo spfx && \
    chown -R spfx:spfx /usr/app/spfx
USER spfx
CMD /bin/bash

build.sh

docker build --rm -f Dockerfile -t pskelly/tw-spfx:1.7.1 .

run.sh

docker run -it --rm --name hello-spfx -v $PWD:/usr/app/spfx -p 5432:5432 -p 4321:4321 -p 35729:35729 pskelly/tw-spfx:1.7.1

log.txt (from ./build.sh including generator-spfx)

Sending build context to Docker daemon  119.8kB

Step 1/9 : FROM node:10.10.0
 ---> d0d12094f6ab
Step 2/9 : LABEL maintainer="Pete Skelly <peteskelly@gmail.com>"
 ---> Using cache
 ---> 14f166bf77a5
Step 3/9 : EXPOSE 5432 4321 35729
 ---> Using cache
 ---> d5c45891f4c1
Step 4/9 : RUN npm i -g gulp@3.9.1 yo@2.0.5 @microsoft/generator-sharepoint@1.7.1 @pnp/office365-cli@next @pnp/generator-spfx@1.6.1&&    npm cache verify
 ---> Running in 7de825c1f808
�[91mnpm�[0m�[91m WARN�[0m�[91m deprecated gulp-util@3.0.8: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
�[0m�[91mnpm WARN deprecated graceful-fs@3.0.11: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
�[0m�[91mnpm WARN deprecated minimatch@2.0.10: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
�[0m�[91mnpm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
�[0m�[91mnpm WARN deprecated graceful-fs@1.2.3: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
�[0m�[91mnpm WARN deprecated cross-spawn-async@2.2.5: cross-spawn no longer requires a build toolchain, use it instead
�[0m/usr/local/bin/o365 -> /usr/local/lib/node_modules/@pnp/office365-cli/dist/index.js
/usr/local/bin/office365 -> /usr/local/lib/node_modules/@pnp/office365-cli/dist/index.js
/usr/local/bin/gulp -> /usr/local/lib/node_modules/gulp/bin/gulp.js
/usr/local/bin/yo-complete -> /usr/local/lib/node_modules/yo/lib/completion/index.js
/usr/local/bin/yo -> /usr/local/lib/node_modules/yo/lib/cli.js

> spawn-sync@1.0.15 postinstall /usr/local/lib/node_modules/@microsoft/generator-sharepoint/node_modules/spawn-sync
> node postinstall


> spawn-sync@1.0.15 postinstall /usr/local/lib/node_modules/yo/node_modules/spawn-sync
> node postinstall


> @pnp/generator-spfx@1.6.1 postinstall /usr/local/lib/node_modules/@pnp/generator-spfx
> node lib/patch.js

�[91mfs.js:115
    throw err;
    ^

Error: ENOENT: no such file or directory, mkdir '/usr/local/lib/node_modules/@pnp/generator-spfx/node_modules/@microsoft/generator-sharepoint/lib/generators/applicationCustomizer/templates/react'
    at Object.mkdirSync (fs.js:731:3)
    at Object.<anonymous> (/usr/local/lib/node_modules/@pnp/generator-spfx/lib/patch.js:123:8)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
    at startup (internal/bootstrap/node.js:279:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:696:3)
�[0m�[91mnpm ERR! code�[0m�[91m ELIFECYCLE
�[0m�[91mnpm ERR! errno 1
�[0m�[91mnpm ERR! @pnp/generator-spfx@1.6.1 postinstall: `node lib/patch.js`
npm ERR! Exit status 1
�[0m�[91mnpm ERR!�[0m�[91m 
npm ERR! Failed at the @pnp/generator-spfx@1.6.1 postinstall script.
npm �[0m�[91mERR! This is probably not a problem with npm. There is likely additional logging output above.
�[0m�[91m
npm ERR! A complete log of this run can be found in:
npm�[0m�[91m ERR!     /root/.npm/_logs/2019-02-01T13_29_16_669Z-debug.log
�[0m
@StfBauer

This comment has been minimized.

Copy link
Contributor

StfBauer commented Feb 1, 2019

Thank you for sharing - @pkskelly
There here are some issues in your docker file thing that does not make sense:

First, let's take a look at the run npm install:

RUN npm i -g 
              gulp@3.9.1 
              yo@2.0.5 
              @microsoft/generator-sharepoint@1.7.1 
              @pnp/office365-cli@next 
              @pnp/generator-spfx@1.6.1 && \
   npm cache verify

Just separate the lines for better visibility:

@microsoft/generator-sharepoint@1.7.1
This only makes sense when you want to be able to use @microsoft/generator-sharepoint exclusively without the use of @pnp/generator-spfx.
When you like to use the pnp/spfx generator you have @microsoft/generator-sharepoint included in Version 1.7.1.
The result for both will be the same you get a new project for SPFx 1.7.1.

@pnp/generator-spfx
Just automates things like including adding pnp/js, the ReactJS property controls or ReacJS reusable components. Jest testing framework, 'gulp dist' task so on and so forth. As mentioned before you get a 1.7.1 SPFx project.
ReactJS, KnockoutJS, No framework runs exactly, despite the add-ons, on 'yo @microsoft/SharePoint. VueJS, HandlebarsJS, Angular Elements has some more advanced configurations on top of @microsoft/sharepoint` and the underlying project type that used for the execution and creation of an SPFx project is in this case 'no framework'. So you get a 'no framework' + additional configuration.

Additional steps for add-ons won't be included when you run yo pnp/spfx generator again on the project. At the end. The overall result you end up is exactly the same you would get running yo @microsoft/SharePoint

@pnp/office365-cli@next
There is nothing to say against the CLI, the only thing I am not sure is why you use this in the Docker file. If you want to execute command there. Sure, in this case, a no-brainer. If you like to check the status of your projects and how to upgrade those. You do not need to install the CLI.
The CLI is also included in pnp/generator-spfx to check what version of SPFx was used to create the project. So if you only execute spfx project upgrade you can safely execute yo @pnp/spf�x.

In the background, it executed the spfx project upgrade via an API call and saves the result of the finding in a Markdown file in the root of the project directory. If your project is up to date, you also get a notification and the @microsoft/generator-sharepoint gets called.

npm install in docker file
The npm install in the docker file runs with root privileges, which means that if a 'post-install' or 'pre-install' should be executed this will fail. This is built-in NPM and mostly for security reasons because you shouldn't install anything using root privileges at all.
In the case of Docker, this is causing the problem with the post-install. Overall there are two other post-install that will fail silently because they have better error handling. I wasn't aware of this until today. So in order to run a proper install in a Docker file, especially when you have any deeper knowledge on the inside building blocks you have to run npm install with and additional switch.

You need to append to the npm install simply --unsafe-perm=true. This flag is set only for the root to 'unsafe-perm=false'. All other users have set this 'unsafe-perm' to true. In the context of the user, it is more assumable that the permissions are properly set and do not have access to the complete system.
in the end, Docker is just a tiny Linux VM for one specific purpose only. Easy to shut down and kill if it behaves badly for some reason.

So overall you should be fine with a docker file that looks like this:

FROM node:8.12.0

EXPOSE 5432 4321 35729

RUN npm i -g gulp@3 yo @pnp/generator-spfx@1.6.1 --unsafe-perm

VOLUME /usr/app/spfx
WORKDIR /usr/app/spfx

RUN useradd --create-home --shell /bin/bash spfx && \
    chown -R spfx:spfx /usr/app/spfx

USER spfx

CMD /bin/bash

In addition I removed the following line too:

usermod -aG sudo spfx

This only adds the user to a group named 'sudo', that currently do not exist on Docker images anymore. After playing with this stuff today, well I learned some things too. Thank you very much for let me go back to Docker after some time.

I still experience some issue in my docker file now but it seems to be more releated to SPFx overall but not for the installation or container.

Feel free to ask my anything you like.

@pkskelly

This comment has been minimized.

Copy link
Author

pkskelly commented Feb 1, 2019

@StfBauer THANKS so much for the detailed explanation! I thought it might be something simple. Always lots to learn and I appreciate you taking the time to provide the details.

I tried your modifications above to the Dockerfile, and the image was built successfully. However, I only get the @microsoft/sharepoint generator, and no option for the @pnp/spfx generator as I would expect. Strangely, running
npm list -g --depth=0 does list the base generator and the CLI, but not the pnp generator.

spfx@2b729e1cf27f:/usr/app/spfx$ npm list -g --depth=0
/usr/local/lib
+-- @microsoft/generator-sharepoint@1.7.1
+-- @pnp/office365-cli@1.13.0-beta.6d11277
+-- gulp@3.9.1
+-- npm@6.4.1
`-- yo@2.0.5

Is this what you are seeing as well?

@StfBauer

This comment has been minimized.

Copy link
Contributor

StfBauer commented Feb 1, 2019

@pkskelly How does your docker file look now because with the Docker file I posted I only have pnp/spfx generator installed but not the other one.

Doesn't make sense to have both. IMO.

@pkskelly

This comment has been minimized.

Copy link
Author

pkskelly commented Feb 1, 2019

@StfBauer I copied your Dockerfile content from above. Below is a copy and paste of my current docker file.

FROM node:8.12.0

EXPOSE 5432 4321 35729

RUN npm i -g gulp@3 yo @pnp/generator-spfx@1.6.1 @pnp/office365-cli@next --unsafe-perm

VOLUME /usr/app/spfx
WORKDIR /usr/app/spfx

RUN useradd --create-home --shell /bin/bash spfx && \
    chown -R spfx:spfx /usr/app/spfx

USER spfx

CMD /bin/bash

Apparently, when removing previous images, there was not a full clean. Detailed check of the image build output revealed it was still calling my old RUN command.

Step 3/9 : EXPOSE 5432 4321 35729
 ---> Running in f6a6e6076257
Removing intermediate container f6a6e6076257
 ---> decafb5ae2d7
Step 4/9 : RUN npm i -g gulp@3.9.1 yo@2.0.5 @microsoft/generator-sharepoint@1.7.1 @pnp/office365-cli@next &&    npm cache verify
 ---> Running in 6fad456e29f2

To be sure I had no remnants of old images, I ran the following to clean all images and containers.
docker system prune --all --force --volumes locally.

The build.sh now builds the image and completes fine, and I only see the pnp/spfx generator when running yo. Running the container and checking installed packages with npm list -g --depth=0 now returns expected global packages.

Seems to be all working now! And the proof is below - Thanks again!
screen shot 2019-02-01 at 2 32 27 pm

@pkskelly

This comment has been minimized.

Copy link
Author

pkskelly commented Feb 1, 2019

Solution above works. Closing issue. See conversation for details and resolution.

@pkskelly pkskelly closed this Feb 1, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment