fix: order files in asar for incremental updates #8128
Conversation
ASAR file begins with a header that list all files and an offset to each file in the rest of the file. When a file placed early in ASAR changes its length - it means that all subsequent file declarations in the header will have their offsets updated. While harmless by itself, this negatively affects the incremental download size as more of the installer binary is different from what it used to be. In this change we order files in asar such that: - Dependencies/node_modules come first (they change least often) - Main app files come last (they change more frequently) Additionally, files in asar are now ordered alphabetically within each fileset to guarantee stable output. All of above results in 2x improvement of incremental download size.
🦋 Changeset detectedLatest commit: 8a13d81 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
✅ Deploy Preview for car-park-attendant-cleat-11576 ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
This is so cool. |
|
Now that I think more about it - is there any point of having separate file sets? Could we merge them all together without a risk? |
|
@indutny-signal I don't quite understand; could you explain in more detail? Does this involve merging all files together? |
|
@beyondkmp no, only merging the file sets. The resulting ASAR should be the same if maybe ordered a bit more consistently. |
|
Nice work! Related note re: the filesets and merging them, I've been yet to figure out their true purpose for being separated. I've been working in my off time to migrate us to the official electron/asar, but still not to any success yet due to the way we handle filesets |
|
@mmaietta thanks! Yeah, it is... funky. I think we have separate file sets for separate matchers because matchers can specify "from" and "to" paths, but I'm not entirely sure why these have to be separate. If we ever migrate to electron/asar - that I mean that I should send a PR to them as well 😂 |
|
Blerg, sorry for snapshots failing. It makes sense that all offsets are changed, but I couldn't get tests running properly on my arm64 macbook. FWIW, I tested this by packaging our app which has ASAR integrity checks on, and verifying that it starts and runs without issues. |
|
@indutny-signal let me take a shot at updating them from my side on my arm64 mac |
|
@mmaietta ah, that's what I was missing. Let me run it real quick. I actually have docker installed now. |
|
@mmaietta done |
|
I'm actually not entirely sure why it is failing in CI. It seems to pass locally... |
|
Will make this into the next release once I can get all snapshots updated, I'll take a look! Related note re: differential updates, MacOS differential is now supported in electron-updater@6.3.0-alpha.1. As with all electron-updater changes, please make sure to thoroughly test from your side |
|
Aww, thanks so much! We use our own differential update system so we won't get a chance to test it :( |
|
Very cool! Out of curiosity, does your differential update system work with dmgs? |
|
@mmaietta nope, we use zip files! |
ASAR file begins with a header that list all files and an offset to each file in the rest of the file. When a file placed early in ASAR changes its length - it means that all subsequent file declarations in the header will have their offsets updated. While harmless by itself, this negatively affects the incremental download size as more of the installer binary is different from what it used to be.
In this change we order files in asar such that:
Additionally, files in asar are now ordered alphabetically within each fileset to guarantee stable output.
All of above results in 2x improvement of incremental download size for updates that don't change dependencies.