Faster, simpler and more robust debug/bench build setup

[mourner]

Closes #3014.

Greatly improves our debug and benchmark setup by using a set of NPM scripts that rely on watchify and st, removing the custom server.js script with express and browserify-middleware.

Introduces 3 independent server scripts:

• npm run start-debug watches gl-js code and runs a static server for debug pages in parallel; you access them through localhost:9966/debug/
• npm run start-bench downloads bench data, watches all benchmark-related code and runs a static server; you access benchmarks through localhost:9966/bench/; append #fps to run the FPS benchmark, etc.
• npm start combines the two above

With this setup, any code change is updated on the pages in 200-300ms instead of 20 seconds, making GL JS development pleasant again. It's also much simpler and more reliable.

• 👀 @lucaswoj @anandthakker
• bring back the ability to run single benchmarks
• update readmes

[anandthakker]

@mourner I ❤️ this change!

My only comment is that it might be worth considering budo (a lightweight dev server that wraps watchify) instead of the homegrown watchify/st combo. I am pretty sure it would allow exactly the same use cases, but with fewer helper npm scripts.

[mourner]

@anandthakker we used Budo before, but @lucaswoj dropped it as a part of #2231. I still prefer watchify + static server setup because it's way simpler, more flexible, and we don't really need live reload IMO.

[mourner]

It's worth noting that the main reason I split the servers back is that the debug server becomes ultra-fast — it starts from scratch in just a few seconds (the first watchify build takes ~1.7s). The benchmark server, which is needed less often, starts up with a build in ~15s.

However with this setup, I can actually change the scripts so that npm start sets up both debug and bench servers, and then have npm run start-debug and npm run start-bench if I want to run them separately. edit: did this, it's glorious!

[anandthakker]

I still prefer watchify + static server setup because it's way simpler, more flexible, and we don't really need live reload IMO.

I'm sold 👍

[anandthakker]

@mourner is there an easy way to increment the port until an open one is found? Asking because budo by default uses the same one, and I've definitely run into collisions while developing on gl-js.

[mourner]

I don't know of an easy way but I could do a PR to st or make a wrapper in a Node script.

[mourner]

Opened an st issue isaacs/st#81, will follow-up with a PR if the idea is approved.

[lucaswoj]

This looks great! I love how this simplifies our debug server into small, sharp, and composable tools.

Mixing generated and source files in the same directory adds clutter and is potentially confusing (e.g. bench/index.js vs bench/bench.js). What do you think about using a different directory for generated files, using to budo to generate these files, or trying to fix the original browserify-middleware bug?

The previous URL structure was cleaner. I appreciate that it's tricky to recreate without using a more sophisticated server like express. 💭

[mourner]

I agree about generated files — would it suffice to move bench/bench.js into bench/bundle/bench.js and debug/access-token.js into debug/bundle/access-token.js? Another option would be to create a thin wrapper script around st that would generate those on the fly, but the first is simpler.

What are your concerns about the new url structure? I find it cleaner than the previous one because it's explicit and predictable, without "magic" — for every referenced URL, it's clear where it comes from.

The original browserify-middleware issue is more of a design decision than a bug — it just isn't designed to handle cases like ours well, in addition to not being maintained anymore. So I'd avoid it if possible. I'm also hesitant about Budo because it's not suited for multiple target use case, and also seems like an overkill.

[mourner]

Just to be clear, I'd like to land this PR as soon as possible because the current situation is severely crippling my ability to work on GL JS productively, and it's hard to appreciate the elegance of the URL design when every page request takes half a minute. We can flesh out the nuances in follow-up PRs.

[mourner]

Switched to npm-run-all for Windows compatibility and simper parallel NPM scripts, and I think this looks pretty awesome: 203ab48