python-webpack stops rendering often

[owais]

I think I've found a bug in python-webpack or python-js-host.

Bundle's render method stops working if I save a file more than once in quick succession or and sometimes if an error occurs. It stops rendering the script tags in template.

Things work again if I restart jshost or make a change to one of the source files and refresh the page.

I guess this happens when webpack is still building and django sends a request for the bundle. Instead of blocking, the bundle returns an empty string instead of blocking the call.

Also, I'm running jshost manually like this, ./node_modules/.bin/js-host host.config.js

I've also tried to increase WATCH_DELAY setting to 1000 from 200 but the issue remains. Everything else is set to the default value mentioned in the docs.

My webpack config looks like this (but I doubt it has got anything to do with that).

var webpack = require('webpack');
var BowerWebpackPlugin = require("bower-webpack-plugin");

module.exports = {
    context: __dirname,
    entry: "./main.js",
    output: {
        path: "[bundle_dir]",
        filename: "[name]-[hash].js"
    },

    plugins: [
      new BowerWebpackPlugin(),
      new webpack.ProvidePlugin({
        $:      "jquery",
        jQuery: "jquery",
        hammerjs: "hammerjs",
        react: "React"
      })
    ],

    module: {
      loaders: [
          { test: /\.jsx$/, loader: 'jsx-loader?insertPragma=React.DOM&harmony' },
          { test: /\.css$/, loader: "style-loader!css-loader" },
          { test: /\.scss$/, loader: "style!css!sass!" },
          { test: /\.less$/, loader: "style-loader!css-loader!less-loader" },
          { test: /\.woff(\?v=\d+\.\d+\.\d+)?$/,   loader: "url?limit=10000&minetype=application/font-woff" },
          { test: /\.woff2(\?v=\d+\.\d+\.\d+)?$/,  loader: "url?limit=10000&minetype=application/font-woff" },
          { test: /\.ttf(\?v=\d+\.\d+\.\d+)?$/,    loader: "url?limit=10000&minetype=application/octet-stream" },
          { test: /\.eot(\?v=\d+\.\d+\.\d+)?$/,    loader: "file" },
          { test: /\.svg(\?v=\d+\.\d+\.\d+)?$/,    loader: "url?limit=10000&minetype=image/svg+xml" }
      ],
    },

    resolve: {
      modulesDirectories: ['web_modules', 'node_modules', 'bower_components'],
      extensions: ['', '.js', '.jsx']
    },
}

[markfinger]

Hmm, sounds strange. The code that wraps webpack is all in https://github.com/markfinger/webpack-service/blob/master/lib/Bundle.js, if it's any help. Either way, I'll try to rig up a test case to replicate it

[owais]

Thanks Mark, if you are not able to find it this week, I'll try to debug it next weekend since my weekend is over and I must get back to official work :(

[owais]

@markfinger I also see a lot of Timeouts.

HTTPConnectionPool(host='127.0.0.1', port=9009): Read timed out. (read timeout=10.0)

Unfortunately, I have not been able to pin point the reason for this.

[markfinger]

There's a 10 second timeout on requests to the host. If it's building from a cold-boot, webpack can take a while.

You can bump up the the FUNCTION_TIMEOUT setting in python-js-host to give it more room to build

[rockymeza]

I tend to get a timeout under the two scenarios:

1. I installed a new package and tried to require it. I have to restart the js-host server in order for it to know.
2. I have a syntax error. It will often timeout. Then I restart the js-host server and it will start reporting the actual syntax errors to django.

[markfinger]

Sounds like this is probably webpack reporting a hard error, which is causing everything to break.

I've had test cases to handle broken files, but I never added a test case for broken watched files which were fixed.

It's been a while since I wrote this stuff, but it should be easy enough to test the following cases and see what's happening

My first guess is that the no op function being passed in by the watcher (we rely on plugin events to detect progress) is burying the errors. Maybe we need to force the compiler to restart after a hard error...?

@rockymeza @owais you should be able to hook in to the compiler's events which are emitted through the webpack-watcher wrapper.

You can add handlers around this line https://github.com/markfinger/webpack-service/blob/master/lib/Bundle.js#L188

Something like

this.watcher.onDone(function(err, stats) {
  console.info('done ' + this.opts.config, err);
}.bind(this);

this.watcher.onInvalid(function() {
  console.info('invalidated ' + this.opts.config);
}.bind(this));

this.watcher.onFailure(function(err) {
  console.error('failed ' + this.opts.config, err);
  // maybe try... 
  // this.watcher.invalidate()
}.bind(this));

[markfinger]

If this.watcher.invalidate() doesn't work, you might need to kill the watcher and the compiler, which should restart the whole process on the next request.

this.watcher.close()
this.watcher = null;
this.compiler = null;

I'll have some time over the weekend to poke around.

[markfinger]

Found a few minutes and hacked around. Looks like onFailure might only be called for more esoteric stuff.

Anyway, the following is a brute-force fix for the problem. I think the watcher wrapper might be failing to keep track of errors.

    this.watcher.onDone(function(err) {
      if (err) {
        this.invalidate();
        console.error(err);
        this.watcher.close();
        this.watcher = null;
        this.compiler = null;
      }
    }.bind(this));

I'll fix it properly when I get some more time

[markfinger]

The webpack-service master branch has the fix

[owais]

Thanks Mark, will give it a shot tomorrow and report back.

[rockymeza]

this did work better, but it just gave me a callback inside of webpack-watcher instead of giving me the actual build error.

The actual build error was spit out in the console though.

[markfinger]

Turned out the watcher was burying errors :( at least it gave me a change to rewrite some old stuff.

If you replace webpack-service with webpack-wrapper, it should be fixed now. I've updated the docs, but you can refer to 4c4f75f for the diff.