Next steps for 0.9

[sokra]

Webpack become bigger than initially excepted (like many other growing projects ;) ), and the current design is not longer suitable for planed features...

So a important step must be a big refactoring to a better design!

This will also result in a API change, because the current API is weird in some aspects and many options are not logical named/ordered/categorized. (I'll offer a deprecated backward compatible API, will warnings as hints for migration)

It's cool that we have a good bunch of tests so that we can test the new design. I'll to add only a few new features with 0.9, but I'll prepare the design for source maps and hot code replacement. It would be good if we could share more code with enhanced-require.

I'll move some stuff out of enhanced-resolve to keep this library more reuseable.

This (API) stuff is weird and will be changed:

• Why is the loaders option in the resolve object?
• The postprocess is weird. OO-Solution: Extend a factory class.
• Why are so many options in the options object? Make it more structured.

and this (internal) stuff:

• The parsing is not (good) extentable, not reusable and bad code.
• Extending the output source is weird
• Complex inter-module optimizing is difficult

I'll post more stuff on progress in the issue, so keep watching

[rdrey]

I'm looking forward to these changes. Let me know if I can help somehow.

[Phoscur]

I like that you are going to rethink the options object. Try to use convention over configuration when possible, find good default values.

[sokra]

Current state of design:

The webpack function will stay for backward compatiblity.

There is are new public Compiler class which encapsulate the overall process. It offers the convenience methods run and watch.

The Compiler creates a instance of Compilation for each run. It builds the dependency tree and splits it into chunks and other assets. It offers the methods process(context: String, entry: Dependency, callback) and getStats(). It also offers access to the properties entry: Module chunks: Chunk[] modules: Module[] assets: Object<String, String | Buffer>.

The Dependency class hierachy currently has the subclasses ModuleDependency (with subclasses for each type of apperance in source i.e. CommonJsRequireDependency, AMDRequireDependency, ...) and ContextDependency (also with subclasses).

In a Compilation you can map a class of Dependency to a ModuleFactory and a DependencyTemplate. The ModuleFactory creates a Module from a Dependency by resolving. The DependencyTemplate specifies the output source for a Depedency once the Module has a id. The Compiler do the setup for the mappings.

Some classes inherit from a Tapable interface, which means they offer a plugin interface for extending. Currently the classes Compiler, Compilation and Parser. I.e. the Parser is stupid and just ask plugins if it want to make a method call to a statically analysed method.

So the Compiler creates a Parser and map statically analysed methods to functions which returns Dependencys.

The Compilation asks plugins for optimizing the chunks. The Compiler adds some basic plugins which remove modules which are referenced in the parent module and remove empty chunks. The current maxChunks options will result in a plugin which merges modules. (Here is a good place to intergrate some stuff which is on the todo list, like leftover chunk)

MainTemplate, ChunkTemplate and ModuleTemplate will do the outputing, so you may subclass them to provide own stuff. (SingleMainTemplate, JsonpMainTemplate, JsonpChunkTemplate, FunctionModuleTemplate, EvalDevToolModuleTemplateDecorator are provided and set by the Compiler). The templates may also offer a plugin interface to extend them.

Also node.js / IO specific stuff is in the classes NodeCompiler, NodeFileEmitter, NodeResolver and NodeSubProcessExecutor. I hope somebody will implement a NodeThreadsAGoGoExecutor 😄 . That would be much better.

There is more stuff, this is just a overview. Any comments? Any stuff whichs should be extendable?

[sokra]

Which loaders should be included in webpack? Here is a list.

[jhnns]

I favor your attempts to simplify the API. The current options-object is very complex and seemed a little odd to me (e.g. the resolve/postprocess-thing).

I'm not sure if I got your explanation right, but I like your idea of a Compiler- and a Compilation-Class. Could you provide us some example code of how the webpack API will be?

Such as

var webpack = require("webpack"),
    Compiler = webpack.Compiler,
    ModuleTemplate = webpack.ModuleTemplate,
    MyModuleFactory = require("../MyModuleFactory.js"),
    myModuleFactory = new MyModuleFactory(),
    compiler = new Compiler();

var compilation = compiler.run("/my/folder");

compilation.on("dependency", function (dependency) {
    myModuleFactory.resolve(dependency);
    // etc
});

So what happens to loaders? How do they fit in the new architecture?
I don't think that webpack should ship more loaders. Your list is sufficient.

[sokra]

Loaders are the best thing :) and will stay.

I think I will ship webpack without the loaders, and you have to install every loader you need... So it stay more lightweight.

The standard usage of webpack will be (as before) the webpack method:

webpack({
  context: __dirname,
  entry: "./file.js",
  output: {
    path: path.join(__dirname, "public"),
    filename: "mybundle.js"
  }
}, function(err, stats) {
  if(err) throw err;
  console.log(stats.toString({
    colors: true
  }));
});

The options have been refactored. There is a plugin interface, so i hope many stuff can move into a plugin. (at options.plugins)

An example plugin for changing a path after resolving:

function MyPlugin() {
};
module.exports = MyPlugin;
MyPlugin.prototype.apply = function(compiler) {
  compiler.resolver.plugin("module-resolved", function(result, callback) {
    callback(null, result.replace(/x/g, "y"));
  });
};

var MyPlugin = require("./myPlugin.js");
webpack({
  // ...
  plugins: [new MyPlugin()]
}, callback);

The other (more advanded) way is the Compiler API, which is internally used by the webpack method:
(I'm not yet happy with this, but that is the current state)

var c = new Compiler();
c.context = __dirname;
c.entry = new SingleEntryDependency("./file.js");
// or: new SingleEntryPlugin(__dirname, "./file.js").apply(c);
c.options.output = {
  path: path.join(__dirname, "public"),
  filename: "mybundle.js"
};
// or: new OutputLocationPlugin(path.join(__dirname, "public"), "mybundle.js").apply(c);
new NodeEnvironmentPlugin().apply(c);
new JsonpTemplatePlugin().apply(c);
new FunctionModulePlugin().apply(c);
new EvalDevToolModulePlugin().apply(c);
new MyPlugin().apply(c);
c.run(function(err, stats) {
  // ...
});

The plugin system has the intend that a library could provide a plugin which does the configuration... I. e. a jade plugin which binds the extension to the correct loader. The user just have to use the plugin. (maybe with --use jade-plugin)

[sokra]

Progress:

• Core
• Resolver
• extract core into extra module
• contexts
• AMD
• enhanced-resolve
• optimizing chunks
• extract loaders into extra modules
• let enhanced-require also use it
• beta
• upgrade tools
• upgrade some loaders

[jhnns]

Seems reasonable to me. 😄 How does the compiler interface look like? Don't make it too granular and complicated. For me there are two important parts:

• Resolving modules (including compilation of modules like CoffeeScript)
• Outputting modules

I wouldn't separate path lookup and compilation (as it was with webpack 0.6.x). A loader may change the path and/or compile a module.

I like the idea of loaders, but I'm still not a big fan of enhanced-require. It is not CommonJS-style and makes the module unconsumable for other bundlers. It's ok to depend on a specific bundler when developing an application. But a library should not depend on a specific bundler.

[sokra]

Compiler interface:

var compiler = new Compiler();
// <- Attach plugins to the compiler
// Choose one of "run", "watch" oder "compile"
compiler.compile(function(err, compilation) { /* ... */ }); // compile and return a Compilation
compiler.run(function(err, stats) { /* ... */ }); // compile and write to files
compiler.watch(function(err, stats) { /* ... */ }); // like run, but rerun on change

At least you have to attach this plugins:

• an environment plugin, i. e. NodeEnvironment
• plugins for the output format, i. e. JsonpTemplatePlugin and FunctionModulePlugin
• a plugin for the entry, i. e. SingleEntryPlugin

Even better is if you attach a plugin for a require style, i. e. CommonJsPlugin ;)

Some of the plugins take options.

Example:

var compiler = new Compiler();
var context = __dirname; // root for resolving entry and shortening stuff
var plugins = [
  new NodeEnvironmentPlugin(outputPath, resolveOptions),
  new JsonpTemplatePlugin({ filename: "bundle.js" }),
  new FunctionModulePlugin(context),
  new SingleEntryPlugin(context, "./entry.js"),
  new CommonJsPlugin()
];
plugins.forEach(function(p) { p.apply(compiler) });
compiler.run(function(err, stats) {
  if(err) return console.error(err);
  console.log(stats.toString({ colors: true }));
});

This would be a very basic build.

The webpack function is a helper which decides by a passed options object, which plugins are attached.

[sokra]

Here is a bit of documentation for the current plugins: https://github.com/webpack/webpack/wiki/Plugins

[jhnns]

Looks good! But I'm not sure about the NodeEnvironment-plugin. Is webpack really designed to run in other environments than node?

[sokra]

Currently not, but I may want to add an online demo, so it would be cool if it could run in the browser (of couse bundled with itself ;) ). Maybe to worker loader also need another environment, not sure yet.

[jhnns]

Ahh headache ^^

[sokra]

Next week I may have some time to make good progress with 0.9

[jhnns]

Yeah! 🚀

[sokra]

require.context is still missing...

[sokra]

require.context and automatic context is now working...

some node.js tests are failing...

[sokra]

yeah...

• watch mode
• chunk numbers
• caching
• progress display
• executeable

[jhnns]

🚀 🚀 🚀