Add support for Magepack

[vbuck]

Summary

This is a proposal and PoC to support Magepack for advanced JS bundling:

• https://github.com/magesuite/magepack

It works by creating an optional container from which to execute Magepack asset bundling. Editing .env to add WARDEN_MAGEPACK=1 will enable it during warden env up operations.

How to Use

If you follow the instructions for Magepack, you'll see there are a series of configuration changes to make for Magento, including the installation of a companion module. I'll post my version of those steps, which picks up where Warden ends (after environment initialization):

• https://docs.warden.dev/environments/initializing.html#initializing-an-environment

Drop into the shell: warden shell. Then run:

composer require creativestyle/magesuite-magepack
bin/magento setup:upgrade
bin/magento config:set dev/js/enable_magepack_js_bundling 1
bin/magento config:set dev/js/enable_js_bundling 0
bin/magento config:set dev/js/minify_files 1
bin/magento config:set dev/static/sign 1
bin/magento config:set dev/js/merge_files 1
bin/magento config:set dev/js/move_script_to_bottom 1
bin/magento config:set dev/css/merge_css_files 1
bin/magento config:set dev/css/minify_files 0
bin/magento config:set dev/css/use_css_critical_path 1
bin/magento config:set dev/template/minify_html 1
bin/magento cache:enable
bin/magento deploy:mode:set production

Note: Depending on your setup, you may need to use --lock-env arguments

Exit the shell, then run the following:

warden env exec magepack generate \
    --cms-url="https://app.warden-m2.test/privacy-policy-cookie-restriction-mode" \
    --category-url="https://app.warden-m2.test/women/tops-women.html" \
    --product-url="https://app.warden-m2.test/autumn-pullie.html"

Replacing the URLs with that of your own environment. In my example, I initialized an M2 project at app.warden-m2.test and installed sample data.

Magepack will generate the config needed and drop it into /var/www/html/magepack.config.js. Note that this generates on a shared mount with your php-fpm container. This is by design, as the next step for Magepack will generate and place your bundles into the static content area.

Finally, run:

warden env exec magepack bundle

When finished, you should be able to navigate to a page and observe the bundles at work; ie:

• https://app.warden-m2.test/autumn-pullie.html

And here is the corresponding Lighthouse score (before/after):

Before Magepack:

bin/magento config:set dev/js/enable_magepack_js_bundling 0

After Magepack:

bin/magento config:set dev/js/enable_magepack_js_bundling 1

Additional Notes

You may notice I'm referencing a forked version of the Magepack repo. This is necessary because there are compatibility issues with various Linux installations when trying to setup Puppeteer for headless Chrome operation. My fork disables sandboxing and ignores TLS certificate errors. I believe this is a safe change to make, as Magepack is a build tool and generally should not be subject to exploits from external forces (read: risk is low).

Update: Sandboxing has already been updated in Magepack, bound for a 2.2.1 release, and I've opened an issue for ignoring TLS errors. Once integrated we can refer to the official source.

As of my commit bef0ef7, I am now using the official latest package @^2.3, so the above notes no longer apply.

Lastly, I'll acknowledge this is a feature intended for build testing, not necessarily daily development. So I'd like to hear your thoughts on its fit as an embedded Warden feature.

[gfilice]

Hi,
using your magepack integration I have an error launching warden env up -d command. The error is:
build path /home/giuseppe/Sites/exampleproject/environments/magento2/magepack either does not exist, is not accessible, or is not a valid URL.

You can see that the path is the local project path.

[davidalger]

@gfilice Seems like there was an error in where the compose config expected the dockerfile to be on this PR. I haven't really tested any of this, as I haven't had the chance, but dropping a quick note here as I've just updated the PR her per the following:

• The work has been integrated with latest develop, i.e. Warden 0.5.1 codebase
• Updated volumes so they should work on both macOS and Linux
• Added CI to build and push magepack image to Docker Hub at docker.io/wardenenv/magepack:2.3 plus manually executed the build script to push it up for testing ahead of PR merge

I verified that warden env up -d will start the magepack image, but that's about it due to time constraints at the moment. Would love to hear feedback from others attempting to use this.

Won't be required to merge this PR, but I'd love to see a short docs page added that covers it's use similar to the docs added for MFTF.

[gfilice]

Thank you @davidalger. Just updated my Warden to 0.5.1 and set the magepack integration using your commits. Now everything works fine.
I would to thank you for your warden project. It's now my preferred working environment.

[davidalger]

Just ran through this real quick on 2.3.5-p1 and I have to say, it works flawlessly with the provided instructions. The below is example output from running the magepack commands, followed by the in-browser results on a PDP:

davidalger:09:38 PM:/sites/m2template (develop *) $ warden env exec magepack generate \
>     --cms-url="https://app.exampleproject.test/" \
>     --category-url="https://app.exampleproject.test/poppy.html" \
>     --product-url="https://app.exampleproject.test/test-p1.html"
ℹ Collecting bundle modules in the browser.                                                                                                                                                                                                                                                                                                                         02:38:16
ℹ Collecting modules for bundle "category".                                                                                                                                                                                                                                                                                                                         02:38:16
✔ Finished collecting modules for bundle "category".                                                                                                                                                                                                                                                                                                                02:38:23
ℹ Collecting modules for bundle "cms".                                                                                                                                                                                                                                                                                                                              02:38:23
✔ Finished collecting modules for bundle "cms".                                                                                                                                                                                                                                                                                                                     02:38:31
ℹ Collecting modules for bundle "product".                                                                                                                                                                                                                                                                                                                          02:38:31
✔ Finished collecting modules for bundle "product".                                                                                                                                                                                                                                                                                                                 02:38:38
ℹ Collecting modules for bundle "checkout".                                                                                                                                                                                                                                                                                                                         02:38:38
✔ Finished collecting modules for bundle "checkout".                                                                                                                                                                                                                                                                                                                02:38:59
✔ Done, outputting following modules:                                                                                                                                                                                                                                                                                                                               02:38:59
✔ common - 178 items.                                                                                                                                                                                                                                                                                                                                               02:38:59
✔ category - 7 items.                                                                                                                                                                                                                                                                                                                                               02:38:59
✔ cms - 0 items.                                                                                                                                                                                                                                                                                                                                                    02:38:59
✔ product - 31 items.                                                                                                                                                                                                                                                                                                                                               02:38:59
✔ checkout - 237 items.                                                                                                                                                                                                                                                                                                                                             02:38:59
davidalger:09:38 PM:/sites/m2template (develop *) $ warden env exec magepack bundle
ℹ Using bundling config from "/var/www/html/magepack.config.js".                                                                                                                                                                                                                                                                                                    02:39:20
ℹ Creating bundles for "pub/static/frontend/Magento/luma/en_US".                                                                                                                                                                                                                                                                                                    02:39:20
✔ Generated bundle "common"     - 778 kB (221 kB gz).                                                                                                                                                                                                                                                                                                               02:39:27
✔ Generated bundle "category"   - 9 kB (3 kB gz).                                                                                                                                                                                                                                                                                                                   02:39:27
✔ Generated bundle "cms"        - 0 kB (0 kB gz).                                                                                                                                                                                                                                                                                                                   02:39:27
✔ Generated bundle "product"    - 108 kB (34 kB gz).                                                                                                                                                                                                                                                                                                                02:39:28
✔ Generated bundle "checkout"   - 238 kB (49 kB gz).                         

[erikhansen]

@vbuck Thanks for this contribution to Warden! With it, we've been able to deploy Magepack to an M2 site, and are working on another.

We're running into one issue with Magepack that we're trying to troubleshoot, and it will require editing the /usr/bin/magepack file inside the magepack docker instance. I tried running warden env exec magepack bash (works fine for the varnish instance), but got this error:

OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "exec: \"bash\": executable file not found in $PATH": unknown

Do you have any tips to be able to do this?

[davidalger]

@erikhansen This message indicates that bash doesn't exist within the MagePack container. You could enter the container using sh instead of bash. Alternatively, you may be looking for how to simply execute the magepack command:

This will execute the magepack binary, and output help text:

warden env exec magepack magepack

The image build also contains wrappers for the two sub-commands allowing one to execute them like this:

warden env exec magepack generate

or

warden env exec magepack bundle

[erikhansen]

@davidalger Thanks. The warden env exec magepack sh command worked.

Btw, this is what we're working on implementing, thus why we need to be able to edit the magepack script.