New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Look into ways of incorporating ES6 import or CommonJS Require into the JS bundle #258

Closed
petulla opened this Issue Jan 4, 2018 · 7 comments

Comments

Projects
None yet
3 participants
@petulla
Copy link

petulla commented Jan 4, 2018

Hi there

This continues a discussion started via email with @alykat and @ghing about ways of improving JS bundling in the Rig.

As it is, a lot of unused code is uploaded with each rig template. Moving to a JS bundler that supports tree-shaking, like Webpack 2+ or Rollup, would reduce the bundle size substantially and allow for D3 imports specific to each graphic. It could be a gateway to the use of node-supported packages, which would open up creative possibilities quite a bit. One example is experimenting with server-side SVG rendering with D3-node/D3-pre.

Hoping someone's tried the Flask webpack plugin or has some ideas of where this could start. Curious if anyone at another org like @TylerFisher has looked into this.

@ghing

This comment has been minimized.

Copy link
Contributor

ghing commented Jan 4, 2018

This was @alykat's initial response:

We did some experimenting with webpack and the dailygraphics rig toward the end of 2016 and the beginning of 2017 (see the generator-dailygraphic and elections16graphics repos in the nprapps Github account) but didn't come up with anything wholly satisfying. (Some of the trouble we ran into: What we prototyped worked best when each graphic was its own repo – otherwise, the more graphics we had, the slower the whole rig got. That felt like a LOT of repos, considering how many graphics we generate, and it also meant that we couldn't quickly switch between graphics as we do now.) I don't know if we previously evaluated the Flask webpack extension you mention, though.

@ghing

This comment has been minimized.

Copy link
Contributor

ghing commented Jan 4, 2018

This is the Flask Webpack plugin Sam referenced: Flask-Webpack

@ghing

This comment has been minimized.

Copy link
Contributor

ghing commented Jan 4, 2018

This is a slightly-edited version of my response to Sam in email:

I too would love to be able to use CommonJS’s require (or ES2015’s import) in graphics code, and I believe that Flask Webpack plugin was something I peeked at when I was curious about its viability. However, my impression is that plugin tells Webpack what to build, via a manifest file, but we’d still have to have some way of running Webpack to build/watch/rebuild those assets. This is where I think we might run into the problem that Aly mentioned. You either have to have Webpack watch the entire graphics directory or start and stop Webpack processes for each individual graphic. How that would work with the existing development server and workflow wasn’t immediately obvious to me, so I didn’t go too far down the rabbit hole.

Another thing to consider, is that we have to factor in the time of both updating our tooling and updating the way we write JavaScript.

Finally, my experience with Webpack is also that it’s very opinionated and I wonder if some of the other emerging bundlers might play better with the existing Flask-based framework. In any case, I agree that tree shaking is the most desirable feature (besides just supporting modular JavaScript) as it would allow us to keep having all the boilerplate that gets copied over with the templates, but get rid of unused function and variable definitions in the output bundle.

I also wonder if it would just be easier to reimplement dailygraphics as a Node.js app that could call WebPack or another bundler’s JavaScript API directly rather than trying to integrate them with a Python/Flask app. We definitely don’t have the capacity to do that right now at NPR, but that would be the approach I would take if I could focus on one infrastructure project for a couple of months.

@ghing

This comment has been minimized.

Copy link
Contributor

ghing commented Jan 10, 2018

While investigating another issue, I noticed that one of our dependencies, cssmin reports being no longer maintained. Also, many Python projects are moving toward ending Python 2.7 support. These might be additional motivations for updating the dailygraphics architecture.

@petulla

This comment has been minimized.

Copy link

petulla commented Jan 11, 2018

@ghing Those are a really good points. I think it is worth asking whether it's sustainable to develop on what is fundamentally a front-end code bundling management application without taking advantage of the recent developments in JS bundling.

@petulla

This comment has been minimized.

Copy link

petulla commented Mar 29, 2018

@ghing I was thinking about this a bit more.

This part:

You either have to have Webpack watch the entire graphics directory or start and stop Webpack processes for each individual graphic.

What if we made a 'prod' directory.. for already pushed graphics, and a 'dev' directory, for the working codebase. There could be a command that moves projects in and out of each state.

That would limit the code that's watched.

@thomaswilburn

This comment has been minimized.

Copy link
Contributor

thomaswilburn commented Jan 3, 2019

Closed via DG-Next

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment