Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Delay building web.browser.legacy bundle until after development server restarts. #10055
Now that Meteor builds two client bundles, build times have naturally increased a bit. We are hard at work attacking this problem from multiple angles, such as #9983, and I'm confident that Meteor 1.7.1 build times will be shorter for most apps than they were before Meteor 1.7, despite building both
Nevertheless, since most testing in development happens using one bundle/browser at a time, it has been suggested (#9948) that we could skip the legacy build in development, as a cheap short-cut to faster rebuild times. To be honest, I don't love most of these ideas for disabling the legacy build with some sort of additional configuration, because supporting older browsers is important, configuration is a creeping cancer, and ignoring errors in the legacy build seems like a recipe for frustration when it's time to deploy your app.
I would vastly prefer a zero-configuration solution to this problem, though that's challenging in this case because skipping an important part of the build process seems like a decision the developer should make consciously and carefully.
As this pull request demonstrates, it's possible to remove the legacy build from the critical path to restarting the development server, so you can continue testing and developing your app while the legacy bundle is built in the background. There's nothing to configure, because the legacy bundle still gets built in roughly the same amount of time as before; you just don't have to wait for it to finish, if you aren't currently interested in testing legacy code.
How it works in detail:
In short, when we talk about zero-configuration here at the Meteor project, we mean something much deeper than just picking sensible defaults. No, we are striving to eliminate the need for configuration whenever and wherever we can. If you want to hear more about this cornerstone of the Meteor philosophy, check out this Meteor Night talk that I recently gave about the Meteor 1.7 modern/legacy bundling system (or read the slides).
Most importantly, we no longer return a copy of previousBuilders from bundler.bundle, but simply modify the input object over time. The copying approach was nice in theory, but incompatible with delaying the bundling of certain architectures (e.g. web.browser.legacy) until some time after bundler.bundle returns.
This is awesome @benjamn! One small thing:
@hwillson Great find! I've tracked this down to the code in
The problem is that
Since the client hash is the same for
This means the autoupdate package is no longer responsible for recomputing client hashes, and we can recompute the hashes whenever there's a new (or updated) client program, which enables delayed builds of architectures like web.browser.legacy (#10055).
Now that we're postponing the legacy build until after the first client refresh message is sent, there's a risk that changes to the legacy build will not be picked up until after the next rebuild. If we attempted to fix that problem by sending the refresh message after the legacy bundle is rebuilt, then we would lose most of the benefit of delaying the legacy build, because the client would not refresh until after the legacy build completed. The right way to fix the problem is by sending a second client refresh message after the legacy build finishes, but doing so with the current autoupdate implementation would very likely cause modern clients to reload a second time. The solution implemented by this commit is simple in theory: the autoupdate package should keep track of distinct versions for each client architecture, so that modern clients will refresh only when the modern versions change, and legacy clients will refresh only when the legacy versions change, which allows us to send two refresh messages without causing any clients to refresh more than once. In reality, this was a fairly major rewrite, since the ClientVersions collection has a totally different schema now. I've tested it as well as I can, though I'm not entirely sure what will happen if clients using the previous version of the autoupdate package begin receiving DDP messages from this version of the autoupdate server code.
Although tests are passing after my rewrite of the
I can make the delayed writing of the legacy bundle "more atomic" by populating a fresh temporary directory and then renaming that directory, but that's slower overall because it means rewriting everything rather than just the files that have changed. Writing the changed files in place (and deleting any files that were removed) is fundamentally less instantaneous than renaming a directory, but much faster because it doesn't require creating a temporary directory from scratch.
Here are some ideas I'm considering to address this read/write race:
I'm going to keep thinking about this until I have more confidence in one solution over the others.
This package is already importable because it's a dependency of request, npm, and http-signature, but it's a good idea to depend on it explicitly just in case those packages stop depending on it in the future.
Instead of having every message consumer listen to every message and act on the ones that seem relevant to its interests, we now have a single process.on("message", callback) hook that can dispatch messages to different Meteor packages running in the server process. Receiving packages should export an onMessage function. The onMessage function may be async, and its result will be delivered back to the build process as the result of the sendMessage Promise.
This is the solution I came up with for the problems I described here: #10055 (comment)
#10055 (comment) As I explained in this comment, Package._on(packageName, callback) was a bad API because it never called the callback if the package was not installed, which caused any app not using the autoupdate package to get stuck trying to communicate with the autoupdate package.
This partially reverts commit 99b79dc, which was added as part of PR #10055 in an effort to trigger hot reloads on the client when/if the definition of a "modern" browser happened to change, due to server code calling setMinimumBrowserVersions. Although changes in the minimum modern browser versions are pretty rare, it seemed important to incorporate this information into the client hash, because code sent to the client tends to be dramatically different depending on whether the client is considered modern. However, this change was made without updating the corresponding version calculations in CordovaBuilder#appendVersion in tools/cordova/builder.js, so the versions in program.json for Cordova apps disagreed with the versions served in manifest.json by the web server, leading to the problems described by @lorensr in this cordova-plugin-meteor-webapp issue: meteor/cordova-plugin-meteor-webapp#69 It would be nice to include the minimum versions hash in program.json for Cordova builds, but unfortunately these versions are not known at build time, because they are determined by calls to setMinimumBrowserVersions during server startup. In other words, if we wanted to access that information during Cordova builds, we would have to start the web server and run all server-side application initialization code just to find out if setMinimumBrowserVersions was called anywhere. In the future, we could consider including the minimum versions hash in manifest.json, so cordova-plugin-meteor-webapp could compare the current version to the new version whenever it fetches manifest.json. However, I think simply removing the minimum versions hash from the client version calculation is a fine solution in the meantime. If a developer needs to trigger a hot reload because they changed their minimum modern versions, they should just be sure to change their client code at the same time. Any change that would normally trigger a client reload will work.