Skip to content
This repository has been archived by the owner on Nov 29, 2023. It is now read-only.

changed: run grunt only when building Docker image #317

Merged
merged 6 commits into from Oct 27, 2021

Conversation

jnm
Copy link
Member

@jnm jnm commented Sep 22, 2021

…not at each container startup when applicable changes are detected.
Closes enketo/enketo#1135; see enketo/enketo#1133.

Also fixes the following error seen during container start up (fixes enketo/enketo#1044 and obviates #338):

Running "shell:build" (shell) task
>> internal/modules/cjs/loader.js:818
>>   throw err;
>>   ^
>> 
>> Error: Cannot find module 'esbuild'
>> Require stack:
>> - /srv/src/enketo_express/scripts/build.js
>>     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
>>     at Function.Module._load (internal/modules/cjs/loader.js:667:27)
>>     at Module.require (internal/modules/cjs/loader.js:887:19)
>>     at require (internal/modules/cjs/helpers.js:74:18)
>>     at Object.<anonymous> (/srv/src/enketo_express/scripts/build.js:3:17)
>>     at Module._compile (internal/modules/cjs/loader.js:999:30)
>>     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
>>     at Module.load (internal/modules/cjs/loader.js:863:32)
>>     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
>>     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12) {
>>   code: 'MODULE_NOT_FOUND',
>>   requireStack: [ '/srv/src/enketo_express/scripts/build.js' ]
>> }
Warning: Done, with errors: command "node ./scripts/build.js" (target "build") exited with code 1. Use --force to continue.

Dockerfile Outdated
RUN npm install -g grunt-cli pm2

COPY . ${ENKETO_SRC_DIR}

RUN npm install --production
RUN npm install
Copy link
Member

@yanokwa yanokwa Sep 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that this will also install dev dependencies, which we probably don't want in prod. This might be better?

RUN npm install
RUN grunt
RUN npm prune --production

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! I'll do this as RUN npm install && grunt && npm prune --production to shrink the resulting image a bit. With everything on one line (and one Docker layer), this step ends up adding 602 MB according to docker history, vs. 645 MB for npm install, 26.5 MB for grunt, and an additional 584 bytes (lol) for the file deletion markers added by npm prune --production.

@yanokwa
Copy link
Member

yanokwa commented Sep 24, 2021

@jnm This PR looks OK to me spiritually. I want to hold off on a final review and merge because I'm waiting for a potentially related PR to Central.

@jnm
Copy link
Member Author

jnm commented Sep 29, 2021

Hi @yanokwa, sounds fair enough. FYI, your ODK Central build will begin to fail at some point if you upgrade Enketo. I'm not sure what changed or when, but the build works okay without npm install on 2.7.3:

jnm@world$ docker run enketo/enketo-express:2.7.3 grunt
<snip>
Digest: sha256:a3419c85b2992d4f12124eabe4de2f2cc30c58a9a1fa9b76fb33448f7e9b6b5e
<snip>
Done.

Execution Time (2021-09-29 01:16:34 UTC-0)
shell:rollup  18.4s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 41%
terser:all      26s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 58%
Total 44.8s

However, on 2.8.1:

jnm@world$ docker run enketo/enketo-express:2.8.1 grunt
<snip>
Digest: sha256:218a955e98983e8cda54f90c9b8350ec3a08f5c321b78eb7f59659aaf30a9d05
<snip>
Running "shell:build" (shell) task
>> internal/modules/cjs/loader.js:818
>>   throw err;
>>   ^
>> 
>> Error: Cannot find module 'esbuild-plugin-alias'
>> Require stack:
>> - /srv/src/enketo_express/scripts/build.js
>>     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
>>     at Function.Module._load (internal/modules/cjs/loader.js:667:27)
>>     at Module.require (internal/modules/cjs/loader.js:887:19)
>>     at require (internal/modules/cjs/helpers.js:74:18)
>>     at Object.<anonymous> (/srv/src/enketo_express/scripts/build.js:3:15)
>>     at Module._compile (internal/modules/cjs/loader.js:999:30)
>>     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
>>     at Module.load (internal/modules/cjs/loader.js:863:32)
>>     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
>>     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12) {
>>   code: 'MODULE_NOT_FOUND',
>>   requireStack: [ '/srv/src/enketo_express/scripts/build.js' ]
>> }
Warning: Done, with errors: command "node ./scripts/build.js" (target "build") exited with code 1. Use --force to continue.

Aborted due to warnings.


Execution Time (2021-09-29 01:16:50 UTC-0)
loading tasks         31ms  ▇▇▇ 6%
shell:clean-js        11ms  ▇ 2%
shell:clean-css       10ms  ▇ 2%
shell:clean-locales   11ms  ▇ 2%
i18next:locales       47ms  ▇▇▇▇ 9%
widgets               31ms  ▇▇▇ 6%
sass:compile         310ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 62%
shell:build           47ms  ▇▇▇▇ 9%
Total 502ms

I've basically been getting away with this through dumb luck and didn't notice until a developer started pulling their hair out while trying to run a local instance of Enketo.

@yanokwa
Copy link
Member

yanokwa commented Sep 29, 2021

Agreed that things break starting with 2.8.x because of the changes. I ran into the problem earlier and we build the Enketo docker file differently with Central* now.

*dead link; please see https://github.com/getodk/central/blob/d5d430ae204001d23528e092d56b0586e4512394/enketo.dockerfile#L19

@jnm jnm force-pushed the 161-run-grunt-at-build-only branch from cd415a8 to c746044 Compare October 15, 2021 00:23
@jnm
Copy link
Member Author

jnm commented Oct 15, 2021

Rebased and resolved merge conflict

Copy link
Member

@yanokwa yanokwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jnm Sorry about the delay and the need to rebase. Sequencing Enketo releases is tricky business!

You open to adding the npm prune --production changes we discussed? Also, any opposition to node 14, since 12 will be EOL in April.

@jnm
Copy link
Member Author

jnm commented Oct 21, 2021

Ah! I rebased away my addition of the npm prune fix we discussed. Sorry about that, and thanks for noticing.

Node 14 sounds great. Will include both changes here momentarily.

@jnm jnm force-pushed the 161-run-grunt-at-build-only branch from c746044 to adfd7e6 Compare October 21, 2021 00:50
Copy link
Member

@yanokwa yanokwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And just to be explicit, before we merge, you've tried this build with KoBo and it works?

@yanokwa
Copy link
Member

yanokwa commented Oct 21, 2021

@eyelidlessness This looks good to me, but wanted a final sanity check from you.

@jnm
Copy link
Member Author

jnm commented Oct 21, 2021

And just to be explicit, before we merge, you've tried this build with KoBo and it works?

Yes, locally. Our big instances are still running 2.8.1.

@amks1
Copy link

amks1 commented Oct 24, 2021

I don't know whether it's because I bungled up somewhere, but I'm getting a 404 on request to /js/build/enketo-webform.js

(Never-ending blue loading circle on opening a Kobo form)

configuration. These volumes aren't necessary now that `grunt` is run
only when the Docker image is built, not at container start up, and the
mounts actually shadow the pre-built JS bundle leading to a never-ending
loading spinner and a 404 fetching `enketo-webform.js`
@jnm
Copy link
Member Author

jnm commented Oct 25, 2021

Thank you, @amks1. I needed to remove obsolete volumes from the sample docker-compose.yml. Could you please test again?

@amks1
Copy link

amks1 commented Oct 25, 2021

Thank you, @amks1. I needed to remove obsolete volumes from the sample docker-compose.yml. Could you please test again?

@jnm
Tested, working now.

@yanokwa
Copy link
Member

yanokwa commented Oct 27, 2021

Thanks for all the great work on this! I haven't had a chance to personally verify it, but LGTM. If we find issues, we can follow on with more PRs.

@yanokwa yanokwa merged commit 447cd5b into master Oct 27, 2021
@MartijnR MartijnR deleted the 161-run-grunt-at-build-only branch May 24, 2022 20:05
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants