Future planning

[darobin]

I think that the architecture built for v3 has run its course, and it is time to start thinking about v4. Of notable interest, I think that RequireJS needs to be phased out in favour of something a lot simpler (and lighter).

As v2 showed however, it is impossible to build a second system (using that term deliberately) in parallel to an existing deployed one. Bug fixes have to be ported over to both while the next generation is being built, which inevitably fails. The only way to make this work is to approach the issue as some form of extreme refactoring, where the current version's architecture is changed while it is in flight (that's how v3 was built).

This entails no backwards-incompatible change. However, the thing that makes me most want to produce a new architecture is how painful it is in v3 to produce new profiles (I'm pretty much the only person who's really done it, despite there having been several requests, and I find it hard to explain without my notes). With the ability to produce new profiles easily it should be possible to keep shipping a version that matches today's (at the same URL) and a new one that drops obsolete stuff (like old school WebIDL support), without having to think about it.

Here are my notes about the things I'd like to do. Comments very welcome!

NPM all the things

Right now we have dependencies just dumped into the root, not even properly vendorised. In most cases we haven't updated them in ages. NPM didn't exist when ReSpec was started, but it does now. Let's use it, it will make things a lot easier.

How: We have all the dependencies installed through NPM and update the build script (and examples) to make use of them that way. This becomes 3.3.

Duct and Browserify

Right now the command-line interface is an afterthought and there is no library (i.e. require()) interface to speak of. Both of these things need to be solved because we have demand for it.

This involves several changes.

Browserify

Browserify replaces RequireJS. This means that we produce a Node-friendly version and use browserify to generate the build.

CLI & API

The Node version will still have to rely on PhantomJS, but we can do so through NightmareJS. This is easier to use, and makes it possible to expose the full processing pipeline as a reusable library that tools can make use of. We wrap PhantomJS so you don't have to.

Simpler building

Switching to browserify means that we can drop our build script and the crazy hoops it jumps through. It should just be npm build (and hopefully npm watch or npm release).

The current setup with branches is way too complicated and dates back from the era when people were still trying to figure out what git workflow worked best. We only need: master and topic branches. Forget anything else.

Duct

In order to replace RequireJS we will also need a way of running the pipeline in a manner that is Browserify-compatible. I have started experimenting with something called "Duct" that can do that. The good news is that it's a project completely separate from ReSpec so it can just be made to work orthogonally and swapped in.

Simpler module interface

The current interface to call into modules is overly complicated. It bears the traces of much rethinking. We can drop most of that complexity. This requires touching a lot of code, but it is very superficial in terms of functional changes.

Putting all of that together, this is the biggest step. But in effect once Duct is operational it isn't that much work, maybe one day, two at most. This takes us to 3.4.

Templating that makes sense

Handlebars is horrible. Picking a sane alternative and replacing all the templates (and helpers they use) would be very welcome. Any preferences?

This is 3.5.

Specialised jQuery

I'm not sure this is worth the work, but listing it just in case: we are currently importing the full jQuery, accounting for over 30% of our code size. We don't, however, use anywhere near all of it. It is possible to make builds of jQuery that only include some core bits. Shall we?

End of the FOUC

I haven't had time to experiment with this, but there are tricks out there to handle FOUCs. We should get rid of those, they annoy precisely the kind of people who are good at annoying others. Any preferred option? Animating in once the style has loaded might work best.

Easier testing

I find writing tests a fair bit harder than it needs to be in this project. I wonder if we could have a system in which 90% of tests could just be an HTML document (being a ReSpec document) and a set of assertions that might even be declarative (e.g. this selector matches, has three items, etc.). It matches what we have but with a lot less overhead. Right now the TS is painful enough that we don't even bother including regressions.

Cleaner configuration

Instead of using a global for configuration, we could just have <script type='respec'> containing some JSON. We could even have that support src and merging in case multiple configurations are supported.

Thoughts?

[marcoscaceres]

###NPM all the things

Agree with all the things.

###Templating that makes sense

Let's just use ES6. We can use ES6's template strings instead of handlebars and transpile the output if needed. Reliance on frameworks sucks.

Using ES6 would also clean up the code a lot in general.

Specialised jQuery

Maybe we could drop it?

End of the FOUC

I did some experimentation a little while ago with this (unfortunately, it was on my computer in Toronto that I can't access from Australia :( )... anyway, I got it working without a FOUC. It was pretty simple. Just add the style early... maybe this are the annoying kinds of hacks you are referring to?

Animating sounds like it could get annoying pretty quickly.

Easier testing

Always nice :)

Cleaner configuration

Sounds good. Like I said elsewhere, would like to deprecate otherLinks.

[marcoscaceres]

(cleaned up comment above)

[darobin]

Do you have examples in mind of places where using ES6 would clean things up? I'm not doubting, just curious what you're thinking about. Our templating needs are modest but I'm unsure ES6 templates are powerful enough. Admittedly, I haven't looked at any of the proposals in a while.

I don't think we can drop jQuery, browsers really aren't there yet. v1 actually used my own library (that's a lot smaller) but jQuery is much much nicer.

Concerning the FOUC, when I say animating I don't mean with some kind of horrible effect; just using animations (I read somewhere that that was a simple trick to get things to show only when the CSS was actually there). But yes it is entirely possible that simply brutally injecting the CSS synchronously as the first thing that the script does would actually take care of the FOUC. This needs testing.

[marcoscaceres]

Do you have examples in mind of places where using ES6 would clean things up?

Acknowledging that the changes would in many cases be cosmetic (unless we aimed to reduce our dependence on jQuery)...

• Anywhere an object is declared whose properties are functions. I'm going through the source tree and I see a lot of opportunity for cleanup there. It would be basic things like:

GLOBAL.respecEvents = {
    pub: function (topic) {...}
}

Becoming:

GLOBAL.respecEvents = {
     pub(topic) {...},
}

(not a huge change, but makes things a little nicer)

• Anywhere where a function can become a fat arrow. I see a lot of those... like .map(function(thing){...} and friends woud become map( thing => some + thing ).
• Anywhere a constant value is used. Again, a ton of of const could be added to aid in code legibility.
• Anywhere that deals with HTML by concatenating strings. Again, I see a lot of that.

This mess of single/double quotes and +:

 if (ref.href) output += '<a href="' + ref.href + '"><cite>' + ref.title + "</cite></a>. ";

Could be beautiful again (:dancer:):

 if (ref.href) output += `<a href="${ref.href}'"><cite>${ref.title}</cite></a>.`;

• Anywhere a default is expected in a function's args.
• Anywhere a for loop is used - either forEach (or appropriate array function, like map) or for(let i of whatever). This would make code much more comprehensible. For example, this one needs to be changed:
  https://github.com/w3c/respec/blob/develop/js/core/inlines.js#L48

And so on...