changed: run grunt only when building Docker image by jnm · Pull Request #317 · enketo/enketo-express

jnm · 2021-09-22T00:41:57Z

…not at each container startup when applicable changes are detected.
Closes enketo/enketo#1135; see enketo/enketo#1133.

Also fixes the following error seen during container start up (fixes enketo/enketo#1044 and obviates #338):

Running "shell:build" (shell) task
>> internal/modules/cjs/loader.js:818
>>   throw err;
>>   ^
>> 
>> Error: Cannot find module 'esbuild'
>> Require stack:
>> - /srv/src/enketo_express/scripts/build.js
>>     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
>>     at Function.Module._load (internal/modules/cjs/loader.js:667:27)
>>     at Module.require (internal/modules/cjs/loader.js:887:19)
>>     at require (internal/modules/cjs/helpers.js:74:18)
>>     at Object.<anonymous> (/srv/src/enketo_express/scripts/build.js:3:17)
>>     at Module._compile (internal/modules/cjs/loader.js:999:30)
>>     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
>>     at Module.load (internal/modules/cjs/loader.js:863:32)
>>     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
>>     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12) {
>>   code: 'MODULE_NOT_FOUND',
>>   requireStack: [ '/srv/src/enketo_express/scripts/build.js' ]
>> }
Warning: Done, with errors: command "node ./scripts/build.js" (target "build") exited with code 1. Use --force to continue.

yanokwa · 2021-09-23T22:43:05Z

 COPY . ${ENKETO_SRC_DIR}

-RUN npm install --production
+RUN npm install


I believe that this will also install dev dependencies, which we probably don't want in prod. This might be better?

RUN npm install RUN grunt RUN npm prune --production

Thanks for the review! I'll do this as RUN npm install && grunt && npm prune --production to shrink the resulting image a bit. With everything on one line (and one Docker layer), this step ends up adding 602 MB according to docker history, vs. 645 MB for npm install, 26.5 MB for grunt, and an additional 584 bytes (lol) for the file deletion markers added by npm prune --production.

yanokwa · 2021-09-24T21:58:42Z

@jnm This PR looks OK to me spiritually. I want to hold off on a final review and merge because I'm waiting for a potentially related PR to Central.

jnm · 2021-09-29T01:23:37Z

Hi @yanokwa, sounds fair enough. FYI, your ODK Central build will begin to fail at some point if you upgrade Enketo. I'm not sure what changed or when, but the build works okay without npm install on 2.7.3:

jnm@world$ docker run enketo/enketo-express:2.7.3 grunt
<snip>
Digest: sha256:a3419c85b2992d4f12124eabe4de2f2cc30c58a9a1fa9b76fb33448f7e9b6b5e
<snip>
Done.

Execution Time (2021-09-29 01:16:34 UTC-0)
shell:rollup  18.4s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 41%
terser:all      26s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 58%
Total 44.8s

However, on 2.8.1:

jnm@world$ docker run enketo/enketo-express:2.8.1 grunt
<snip>
Digest: sha256:218a955e98983e8cda54f90c9b8350ec3a08f5c321b78eb7f59659aaf30a9d05
<snip>
Running "shell:build" (shell) task
>> internal/modules/cjs/loader.js:818
>>   throw err;
>>   ^
>> 
>> Error: Cannot find module 'esbuild-plugin-alias'
>> Require stack:
>> - /srv/src/enketo_express/scripts/build.js
>>     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
>>     at Function.Module._load (internal/modules/cjs/loader.js:667:27)
>>     at Module.require (internal/modules/cjs/loader.js:887:19)
>>     at require (internal/modules/cjs/helpers.js:74:18)
>>     at Object.<anonymous> (/srv/src/enketo_express/scripts/build.js:3:15)
>>     at Module._compile (internal/modules/cjs/loader.js:999:30)
>>     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
>>     at Module.load (internal/modules/cjs/loader.js:863:32)
>>     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
>>     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12) {
>>   code: 'MODULE_NOT_FOUND',
>>   requireStack: [ '/srv/src/enketo_express/scripts/build.js' ]
>> }
Warning: Done, with errors: command "node ./scripts/build.js" (target "build") exited with code 1. Use --force to continue.

Aborted due to warnings.


Execution Time (2021-09-29 01:16:50 UTC-0)
loading tasks         31ms  ▇▇▇ 6%
shell:clean-js        11ms  ▇ 2%
shell:clean-css       10ms  ▇ 2%
shell:clean-locales   11ms  ▇ 2%
i18next:locales       47ms  ▇▇▇▇ 9%
widgets               31ms  ▇▇▇ 6%
sass:compile         310ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 62%
shell:build           47ms  ▇▇▇▇ 9%
Total 502ms

I've basically been getting away with this through dumb luck and didn't notice until a developer started pulling their hair out while trying to run a local instance of Enketo.

yanokwa · 2021-09-29T13:23:45Z

Agreed that things break starting with 2.8.x because of the changes. I ran into the problem earlier and we build the Enketo docker file differently with Central* now.

*dead link; please see https://github.com/getodk/central/blob/d5d430ae204001d23528e092d56b0586e4512394/enketo.dockerfile#L19

jnm · 2021-10-15T00:24:14Z

Rebased and resolved merge conflict

yanokwa

@jnm Sorry about the delay and the need to rebase. Sequencing Enketo releases is tricky business!

You open to adding the npm prune --production changes we discussed? Also, any opposition to node 14, since 12 will be EOL in April.

jnm · 2021-10-21T00:34:47Z

Ah! I rebased away my addition of the npm prune fix we discussed. Sorry about that, and thanks for noticing.

Node 14 sounds great. Will include both changes here momentarily.

…not at each container startup when applicable changes are detected. Closes #161; see #163.

yanokwa

And just to be explicit, before we merge, you've tried this build with KoBo and it works?

yanokwa · 2021-10-21T01:02:56Z

@eyelidlessness This looks good to me, but wanted a final sanity check from you.

jnm · 2021-10-21T01:23:26Z

And just to be explicit, before we merge, you've tried this build with KoBo and it works?

Yes, locally. Our big instances are still running 2.8.1.

amks1 · 2021-10-24T00:14:13Z

I don't know whether it's because I bungled up somewhere, but I'm getting a 404 on request to /js/build/enketo-webform.js

(Never-ending blue loading circle on opening a Kobo form)

…ild-only

configuration. These volumes aren't necessary now that `grunt` is run only when the Docker image is built, not at container start up, and the mounts actually shadow the pre-built JS bundle leading to a never-ending loading spinner and a 404 fetching `enketo-webform.js`

jnm · 2021-10-25T18:50:35Z

Thank you, @amks1. I needed to remove obsolete volumes from the sample docker-compose.yml. Could you please test again?

amks1 · 2021-10-25T19:35:39Z

Thank you, @amks1. I needed to remove obsolete volumes from the sample docker-compose.yml. Could you please test again?

@jnm
Tested, working now.

yanokwa · 2021-10-27T18:56:27Z

Thanks for all the great work on this! I haven't had a chance to personally verify it, but LGTM. If we find issues, we can follow on with more PRs.

yanokwa reviewed Sep 23, 2021

View reviewed changes

jnm force-pushed the 161-run-grunt-at-build-only branch from cd415a8 to c746044 Compare October 15, 2021 00:23

jnm mentioned this pull request Oct 20, 2021

Fixes #337 (plus discussion about change log and commit messages) #338

Closed

yanokwa suggested changes Oct 20, 2021

View reviewed changes

jnm added 4 commits October 20, 2021 20:39

changed: run grunt only when building Docker image

d7f1c01

…not at each container startup when applicable changes are detected. Closes #161; see #163.

changed: prune dev dependencies in Docker build

fb6c7ef

changed: build Docker image from Node 14 base

66b67cf

Add detail to change log

adfd7e6

jnm force-pushed the 161-run-grunt-at-build-only branch from c746044 to adfd7e6 Compare October 21, 2021 00:50

yanokwa approved these changes Oct 21, 2021

View reviewed changes

yanokwa requested a review from eyelidlessness October 21, 2021 01:02

jnm added 2 commits October 25, 2021 14:40

Merge remote-tracking branch 'origin/master' into 161-run-grunt-at-bu…

eedef99

…ild-only

yanokwa merged commit 447cd5b into master Oct 27, 2021

MartijnR deleted the 161-run-grunt-at-build-only branch May 24, 2022 20:05

amks1 mentioned this pull request Oct 31, 2021

Error: Attachment not found for kobo instance media enketo/enketo#1037

Closed

Uh oh!

Conversation

jnm commented Sep 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yanokwa Sep 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jnm Sep 23, 2021

Choose a reason for hiding this comment

Uh oh!

yanokwa commented Sep 24, 2021

Uh oh!

jnm commented Sep 29, 2021

Uh oh!

yanokwa commented Sep 29, 2021 • edited by jnm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnm commented Oct 15, 2021

Uh oh!

yanokwa left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jnm commented Oct 21, 2021

Uh oh!

yanokwa left a comment

Choose a reason for hiding this comment

Uh oh!

yanokwa commented Oct 21, 2021

Uh oh!

jnm commented Oct 21, 2021

Uh oh!

amks1 commented Oct 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnm commented Oct 25, 2021

Uh oh!

amks1 commented Oct 25, 2021

Uh oh!

yanokwa commented Oct 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jnm commented Sep 22, 2021 •

edited

Loading

yanokwa Sep 23, 2021 •

edited

Loading

yanokwa commented Sep 29, 2021 •

edited by jnm

Loading

yanokwa left a comment •

edited

Loading

amks1 commented Oct 24, 2021 •

edited

Loading