Add Webpack support

[browniebroke]

Description

This PR builds on top of #3535 to add a basic Webpack as an option for building front-end assets.

It does NOT introduce any FE framework like React, Vue or the likes but rather provide a basic config that should replicate most of what is currently provided by Gulp.

I want to thank @michael-yin for his work in #3106 as well as his series of blog posts which helped this massively. I adapted it to make sure the whole code is in here and fits our setup more closely.

Checklist:

• I've made sure that tests are updated accordingly (especially if adding or updating a template option)
• I've updated the documentation or confirm that my change doesn't require any updates

Rationale

Support for Webpack has been a requested feature for a long time:

• Ref Add Webpack as an Option #538
• Fix Use webpack to compile scss/sass to css and add babel support #1153
• Ref Add support for webpack #1625
• Fix React support through Webpack #1689
• Closes Replace Gulp with Webpack #3106
• Fix Replace Gulp with Webpack #3091

While Gulp works for simple cases, it's getting less attention recently in the JS community, and the recent activity on the Gulp GitHub repo indicates that the development efforts are reduced to a minimum.

[foarsitter]

Implemented this pull request in an existing project and it works like a charm.

[foarsitter]

I was lying in my previous comment, I had to change 0.0.0.0 to django in dev.config.js in order to get the proxy to work. I'm working on Linux.

[browniebroke]

Thanks for the feedback! Keep sending it in!

[steveputman]

Hi, there. Thanks for putting this together--very cool. I have one problem and a couple of questions, which may be related. I am using the implementation with Docker.

1. With @foarsitter's change in dev.config.js, I have it up and running in development with the following questions:
   -- does the implementation render the django :8000 server sort of useless? When I navigate to :8000, I get an unmodified bootstrap implementation (i.e., without any of the changes that were in place before launching the server--obviously wouldn't expect to see any that are made while it's up and reflected on the node server). That may be intentional with the idea that you'd be developing locally on the node server if the bundled css and js aren't built and written to disk.
   -- is there ever a need to load the cdn version of bootstrap in the base.html template? Is that not bundled in the webpack output?

2. I am not having any luck in production, and I can't figure out why. The rendered version of the page, webpack-stats.json, and the files in my AWS bucket all have the same hash. But I get this message in the console:
   

Thanks for putting this together and for all the work on this great project generally!

Steve

[browniebroke]

Ok, looks like the setup isn't working very well with Docker... I've mainly get it working on a project where Docker isn't used and was intending to test the Docker part in the coming days. I probably missed a few things.

does the implementation render the django:8000 server sort of useless?

You should access the dev server at localhost:3000, which goes to the node server. It serve static content, proxies requests to the Django server running at 8000 and do a hot reload when the static files change. At least that's how it's supposed to work. It's similar to how it works with Gulp.

is there ever a need to load the cdn version of bootstrap in the base.html template?

Shouldn't be needed, the webpack config should build a project.css with a custom Bootstrap CSS, project.js with your javascript and a vendors.js with Bootstrap's + other libs Javascript

[browniebroke]

Tried again the production deployment and it seems to work fine on my end.

I am not having any luck in production, and I can't figure out why

@steveputman what option have you selected for use_whitenoise and cloud_provider?

[steveputman]

@browniebroke I'm n on use_whitenoise and AWS on cloud_provider. I also probably should have fessed up that I adapted an existing project based on a diff between an empty project created with my options from your PR and my existing project, so it's entirely possible that something out of scope is screwing it up. I'll test further and set up a production environment for the empty project to see if I can duplicate my issue.

[browniebroke]

I just checked with use_whitenoise=n and I can reproduce the errors from your screenshot.

Also, I think I just understood your question earlier "is there ever a need to load the cdn version of bootstrap" - it's included in the output, but it should not (I thought you were asking whether it was missing).

[browniebroke]

I think it doesn't work without Whitenoise, because we should set the publicPath in the Webpack prod.config.js (see django-webpack/django-webpack-loader#309), for example of your bucket is called yourbucketname:

   bail: true,
+  output: {
+    publicPath: 'https://yourbucketname.s3.amazonaws.com/static/webpack_bundles/',
+  }
 })

In the Django config, this value configured using environment variables, ideally this value would come from the same ones.

[steveputman]

It does work with your change to publicPath (thanks!). I had assumed that Whitenoise was for serving files from the same host and was not needed when the static assets were on S3, so I have never used it.

And, yes, sorry I wasn't clear about the CDN version of Bootstrap: like you said, I was suggesting that it be deleted from base.html.

[browniebroke]

I had assumed that Whitenoise was for serving files from the same host and was not needed when the static assets were on S3, so I have never used it.

That's a correct statement I think, although I see it more the other way around: S3 isn't needed for static files when Whitenoise is used. Not anymore right or wrong, just a different perspective 😄

PS: Thanks for testing this out and providing feedback.

[steveputman]

Point taken! Will try out Whitenoise. Thanks for all the help!

[browniebroke]

This doesn't work currently with Docker, because the environment variables aren't set when this is config file is read (at build time).

[charleshan]

Has this been fixed?

[michael-yin]

@charleshan

If you are looking for a modern frontend solution based on Webpack for Django, I recommend you to check https://github.com/AccordBox/python-webpack-boilerplate

Thx

[charleshan]

I noticed that live reload isn't working with html changes. I fixed it by adding the following lines:

// dev.config.js
...
devServer: {
    watchFiles: [path.resolve(__dirname, '../{PROJECT_NAME}/templates/')],

[browniebroke]

I've done more testing this week-end locally and deploying to prod, and all seems to work apart from a small limitation with Docker, Webpack and no Whitenoise which I've now documented.

I feel like the only to solve this would be to move to a single .env file, but that's getting a bit out of scope of this PR.

[foarsitter]

Good work @browniebroke, thanks!

[foarsitter]

How do you cope with webpack-stats.json? My pipelines fail because webpack-stats.json is missing because it is ignored. Any clues?

[browniebroke]

Not sure what you mean... Probably best to open an issue with more details about the error:

• What pipeline? Gitlab, GitHub, something else?
• What was run in the steps before the failure?
• What step fails? pytest? deploy? something else?

Thanks