On Windows, Brim, Zui Insider, and Zui installers unpack to the same location

[philrz]

The symptom of this problem was first identified by a community user. I've most recently reproduced it with the GA Windows installer Brim-Setup-0.30.0.exe and the Zui Insiders installer Zui---Insiders-Setup-0.30.1-29.exe.

On all platforms, Brim and Zui Insiders maintain separate locations on the filesystem for data+config, which allow them to have separate lakes, separate settings, etc. They're supposed to also maintain separate locations for application binaries, and this is indeed true on macOS and Linux. The problem here is that on Windows they seem to share the same location under %USERPROFILE%\AppData\Local\Programs for application binaries. Here's repro steps to see what can happen.

1. Start with a "clean" Windows system that has no existing/prior Brim or Zui Insiders install.
2. Install Zui---Insiders-Setup-0.30.1-29.exe. At this point we see:

C:\>dir %USERPROFILE%\AppData\Local\Programs
 Volume in drive C has no label.
 Volume Serial Number is 0C07-A419

 Directory of C:\Users\Phil\AppData\Local\Programs

07/25/2022  10:36 AM    <DIR>          .
07/25/2022  10:36 AM    <DIR>          ..
01/20/2016  10:44 AM    <DIR>          Common
10/20/2021  03:35 PM    <DIR>          Python
07/25/2022  10:36 AM    <DIR>          zui-insiders
               0 File(s)              0 bytes
               5 Dir(s)  16,874,004,480 bytes free

C:\>%USERPROFILE%\AppData\Local\Programs\zui-insiders\resources\app.asar.unpacked\zdeps\zed.exe -version
Version: v1.2.0-11-gcd2f3a37

3. Quit Zui Insiders
4. Install Brim-Setup-0.30.0.exe. At this point the Zui Insiders launch icon disappears from the desktop and its entry is also gone from the installed Programs in Settings. A look at the filesystem shows that the binaries actually landed in the existing zui-insiders directory, proven by the fact that the zed.exe version went backwards to the one associated with the older GA Brim release.

C:\>dir %USERPROFILE%\AppData\Local\Programs
 Volume in drive C has no label.
 Volume Serial Number is 0C07-A419

 Directory of C:\Users\Phil\AppData\Local\Programs

07/25/2022  10:36 AM    <DIR>          .
07/25/2022  10:36 AM    <DIR>          ..
01/20/2016  10:44 AM    <DIR>          Common
10/20/2021  03:35 PM    <DIR>          Python
07/25/2022  10:39 AM    <DIR>          zui-insiders
               0 File(s)              0 bytes
               5 Dir(s)  16,879,833,088 bytes free

C:\>%USERPROFILE%\AppData\Local\Programs\zui-insiders\resources\app.asar.unpacked\zdeps\zed.exe -version
Version: v1.1.0

I've confirmed that neither macOS nor Linux seem to have this problem. When both apps are installed, they show separate "Brim" and "Zui - Insiders" filesystem paths for binaries.

[philrz]

As mentioned in #2409 (comment), on my first time using a "real" Zui install artifact (i.e., not Zui Insiders) I saw the same symptom of my Brim immediately being uninstalled.

[philrz]

Verified using artifacts at https://github.com/brimdata/brim/actions/runs/3101781827.

As a baseline, I did a re-test on a fresh Windows system by first installing Zui---Insiders-Setup-0.30.1-75.exe and then installing the old artifacts from 12 days ago in https://github.com/brimdata/brim/actions/runs/3025318796 which represent a draft build of "real" Zui v1.0.0. As with all the other test permutations, since the App IDs are shared between the two, the install of the Zui artifact made the Zui Insiders icon disappear from the desktop as a symptom of its uninstall, and the directory %USERPROFILE%\AppData\Local\Programs still showed only the zui-insiders folder as proof that the binaries stomped on each other.

After removing all that and starting fresh again, I once again first installed Zui---Insiders-Setup-0.30.1-75.exe but then installed the new draft build of Zui v1.0.0 from https://github.com/brimdata/brim/actions/runs/3101781827 which has the benefit of the unique App ID per #2547. As the following screenshot shows, now both app icons remain on my desktop and I was able to open both simultaneously. Also the folder %USERPROFILE%\AppData\Local\Programs contains both zui and zui-insiders as further proof that both sets of binaries can live safely side-by-side just like we have on macOS and Linux.

[philrz]

For the next test, I started fresh with an install of GA Brim v0.31.0, imported some data, and saved a query. I then installed the draft Zui v1.0.0 artifacts from https://github.com/brimdata/brim/actions/runs/3101781827. As shown in the following screenshot, the migration of my data succeeds just fine, but the Brim app binaries are not replaced by the Zui install. That means the Windows experience is similar to what we've seen on Linux (since the install/uninstall there is manual) but unlike macOS where the Brim app is automatically replaced by Zui upon upgrade.

This leaves the exposure that a user might accidentally/innocently re-open the Brim app and start working with their old data again and not realize there's two separate apps with independent data stores. (In the screenshot above I actually tried to open both at once and so the Brim one saw the "spinner" because of the conflicting Zed services, but each is perfectly capable of launching ok solo.)

I can think of a few possible directions we could go with this information.

1. We could go back to using com.electron.brim as the App ID for Zui, since that would give the "replace on upgrade" behavior we had before and hence avoid the hazard for the user. Of course the App ID for Zui Insiders should remain separate in any case.

2. We could keep it as is, but whenever the user launches Zui, check for the legacy Brim binaries in the well-known prior location and, if present, advise them to perform the manual uninstall. This would benefit Linux users as well.

3. If that prior idea was tempting but deemed too much effort, we could keep it as is, but just heavily document in the Release Notes, Downloads page, etc. that they should do this one-time manual uninstall as part of the Brim/Zui transition.

[philrz]

This issue was reopened because the last tests we ran with Zui Insiders revealed some challenges we couldn't resolve on the spot, so we backed out the #2547 change while we considered our next move.

The setup is on Windows with test artifacts:

• Zui---Insiders-Setup-0.30.1-74.exe
• Zui---Insiders-Setup-0.30.1-75.exe
• Zui---Insiders-Setup-0.30.1-76.exe

The first two (74, 75) are artifacts that were from the current build system where Zui Insiders shares an App ID with regular Brim/Zui. The third artifact (76) was a one-off build that had the change from #2547 such that its App ID differed from the one in the earlier artifacts. That artifact was pulled from the GitHub Releases page for Zui Insiders and has since been replaced with a "regular" artifact numbered 76 that once again was produced via the current build system, which should not be confused with the one used in the tests described here.

Starting with a fresh Windows system that has no prior remnants of Brim/Zui/Zui Insiders installs, after installing Zui---Insiders-Setup-0.30.1-74.exe, the Programs list shows the expected single entry.

After quitting this Zui Insiders and manually installing Zui---Insiders-Setup-0.30.1-75.exe, the Programs list shows just that single new entry. So far so good.

After quitting this Zui Insiders and manually installing the Zui---Insiders-Setup-0.30.1-76.exe that has the different App ID, now two entries are shown in the Programs list.

At this point I expect a user's natural instinct would be to try to uninstall the Zui---Insiders-Setup-0.30.1-75.exe entry. However, when we do this, the Zui---Insiders-Setup-0.30.1-76.exe entry is actually the one that disappears.

And at this point, the entry for Zui---Insiders-Setup-0.30.1-75.exe cannot be removed. If we attempt, we get:

The only way I've found to get rid of the orphaned entry at this point is to manually reinstall the Zui---Insiders-Setup-0.30.1-75.exe artifact, after which the uninstall does succeed.

In conclusion, it seems we still definitely want the App ID for Zui Insiders to change so we can deliver the benefits of side-by-side installations of Brim/Zui and Zui Insiders. However, I don't see a way around this problem without the user doing a manual uninstall of the Zui Insiders that has the old App ID. There may be ways to minimize the impact (I'll run some tests of current ideas) but given that we've intentionally presented Zui Insiders as beta-like, we may simply need to educate these power users via Slack/Twitter and walk them through the one-time repair to their Programs list like I showed above.

[philrz]

I've completed an exhaustive set of tests and have settled on what I think is a reasonable path forward. As predicted above, the tl;dr is that it'll likely consist of:

1. A new App ID just for Zui Insiders, with the App ID remaining the same for the transition from Brim to regular Zui
2. Some Release Notes and/or an article cautioning users of some corner cases during the transition to watch out for and how to handle them

To establish this, I tested all permutations of:

• Supported OS platforms: macOS, Windows, Linux
• Manual update & auto-update (where applicable, i.e., auto-update is not available on Linux)
• What would be the Brim->Zui transition
• What would be the ability to run Zui Insiders side-by-side next to Brim/Zui on all platforms, especially Windows (the original scope of this issue)

For completeness, in all tests, I populated a pool with data and a saved query before performing or triggering any upgrades, then ensured that these were migrated successfully into the newer version. This was successful in all permutations, so it is not mentioned in each detailed test result.

Repo & Builds

To have full control in all test permutations, I created a soft fork of the Brim repo into a separate repo https://github.com/brimdata/upgrade-test, making sure to bring over the branch rc-v0.31.0 since it would be used for baselining. I reconstructed all necessary secrets for creating production-style release packages on all platforms. The only base level change compared to the true Brim repo was setting repository in the package.json to https://github.com/brimdata/upgrade-test so the installed test releases would poll the correct location to auto-update.

Beyond that, here's what's unique about each test release at https://github.com/brimdata/upgrade-test/releases that was used.

Release	appId	version in package.json	Intended to be equivalent to
v0.31.0	com.electron.brim	0.31.0	The production GA Brim v0.31.0 that most users are running currently
v1.0.0	com.electron.brim	1.0.0	The production GA Zui v1.0.0 that we expect to release soon
v1.0.1	com.electron.brim	1.0.1	Two roles: 1. What would be the first follow-on Zui release after v1.0.0 2. The Zui Insiders that many users are running today that shares an App ID with GA Brim v0.31.0
v1.0.2	io.brimdata.zui	1.0.2	What would be the first Zui Insiders release that has a different App ID than GA Brim v0.31.0
v1.0.3	io.brimdata.zui	1.0.3	What would be the first follow-on Zui Insiders release after the App ID change

Test Results

Manual Install/Update For Brim->Zui Transition

Windows

Starting from a fresh system with no prior Brim/Zui software, installing the v0.31.0 release gets the expected result of a single entry in Programs.

After manually installing the v1.0.0 release, the "Brim" entry is gone from Programs and the Zui 1.0.0 entry appears.

This seems like a good result. The user lands in an automatically-launched new Zui with all their data intact. An orphaned User Data directory for Brim is left behind under %APPDATA%, but there's no real risk of a user accidentally accessing this old data, since the only remaining installed app is the Zui one that's pointing at the new User Directory for Zui inder %APPDATA% that holds the migrated data. However, it would be helpful to point this out in a Release Note and/or docs article to alert the user to the redundancy and advise them to archive/delete the older Brim User Data directory once they're satisfied that they see all their data intact in Zui.

If we wanted to be fancy, we could wire up functionality in Zui to look for the older Brim User Data directory in its expected location and, if present, pop up a notification to the user that they might want to archive/delete it. However, I expect this is probably overkill and docs would provide adequate coverage.

macOS

Starting from a fresh system with no prior Brim/Zui software, installing the v0.31.0 release gets the expected result of a single entry in Applications.

After manually installing the v1.0.0 release, the "Brim" entry is still present in the Applications list in addition to the new entry for Zui. Like we saw with Windows (and will see on all platforms) the User Data directory for Brim is also left behind.

This seems like an unexpected/undesirable result, but probably tolerable. The leftover Brim application does leave the possibility that a forgetful user might accidentally open Brim and make changes (load new pools, save new queries, etc.) and then later open Zui and wonder why they're not seeing their changes. As we'll see in later test results, thankfully we don't see this same result during auto-update between these two releases, therefore what we've observed here could probably just be mentioned in a Release Note and/or article indicating that a manual upgrade to Zui would then require a manual uninstall of Brim. Much as described above for the User Data directory, we could be fancy by having Zui check for the presence of older Brim app binaries in the well-known location and pop up a notice advising the user to uninstall, but this is probably overkill.

Linux

Starting from a fresh system with no prior Brim/Zui software, installing the v0.31.0 release gets the expected result of a single entry of installed software.

After manually installing the v1.0.0 release, what we see is basically the same as what we just saw on macOS: Both app icons are now present, and due to the separate data directories, it's still possible to relaunch the old Brim and work with pre-migrated data.

This result (and hence the conclusion) is equivalent to what's written above for Manual Upgrade on macOS.

Auto-Updated Brim->Zui Transition

All these tests start from the same state as the prior tests with v0.31.0 installed on a fresh system.

Windows

With the v1.0.0 release available as the latest available in the GItHub Releases page, we exit from Brim, re-launch, and let it sit for a minute to find the update, which it does.

After clicking Restart, we see the same result as for the Manual Upgrade test on WIndows: Brim is gone from the Programs list and only Zui 1.0.0 is present.

The conclusion is equivalent to what's written above for Manual Upgrade on Windows.

macOS

With the v1.0.0 release available as the latest available in the GItHub Releases page, we exit from Brim, re-launch, and let it sit for a minute to find the update, which it does.

After clicking Restart, I could see the entry for old Brim was gone from the Applications list and only Zui is now present. Contrast this with the Manual Upgrade test for macOS shown above where the entry for old Brim was left behind.

Since auto-update is available on macOS, we expect most (all?) users will make the Brim->Zui transition via auto-update when we post the first Zui release, which helps rationalize not putting in a bunch of special code to deal with the unfortunate result we saw for the Manual Upgrade on macOS. The orphaned Brim data directory is still a consideration, though.

Linux

N/A, since auto-update is not available on Linux. However, the notification of the new release does appear as expected, which would prompt the user to begin the Manual Upgrade workflow we described above.

Auto-Update To Follow-On Zui Release

On each platform a test was performed starting from the state of the prior tests such that only Zui v1.0.0 was installed, then Zui v1.0.1 was made available as the most recent on the GitHub Releases page. On both Windows and macOS we saw the desirable result where:

1. The notice of availability of the v1.0.1 release came up in about a minute after launching the app
2. Clicking "Restart" caused thev1.0.1 release to be installed
3. Only the a single entry for Zui remained in the Programs/Applications list, repsectively

For Linux, the notification popped up as it always does, but Manual Upgrade was necessary as it always is.

In all cases saved data was preserved between releases, and nothing in the logs indicated any attempt to re-migrate old data.

Auto-Update To Side-by-Side Zui Insiders

On each platform a test was performed starting from the state of the prior tests such that v1.0.1 was installed. As that has the appId set to com.electron.brim, it can act as a stand-in for Zui Insiders releases that users are running today that unfortunately clash with the appId for Brim. Since Zui Insiders is almost entirely intended to be upgraded via auto-update (since new releases are published daily), the so-called v1.0.2 was then made available as the most recent on the GitHub Releases page, it having the appId set to io.brimdata.zui and hence is a stand-in for a Zui Insiders that would no longer clash as we saw on Windows and hence could run side-by-side with Brim or regular Zui.

Windows

After seeing the auto-update notification and clicking Restart, the v1.0.2 starts automatically. An entry for the older v1.0.1 is left behind in the Programs list.

If a user saw this state and had not received any additional guidance, I expect what they'd now try to do is uninstall v1.0.1 entry, since it's older. Unfortunately, this actually ends up removing the newer v1.0.2 from the list, leaving behind just the older v1.0.1 entry. If the user once again tries to uninstall that, they receive an error message.

At this point, the App Binaries directory for Zui is empty, so the error message we're seeing seems related to orphaned remnants (in the Registry? I don't claim perfect Windows expertise here...) However, the contents of the separate User Data directory are fully intact. My testing then revealed a simple set of steps for getting out of this state:

1. Manually install the release that matches the orphaned entry (e.g., v1.0.1 in this case)
2. Manually uninstall the release (once again, v1.0.1 in this case) which will leave the Programs list empty of Zui entries
3. Manually install the newest release (v1.0.2 in this case, but in practice it would be the newest Zui Insiders)

At this point I ran another permutation where I started once again back at a clean v1.0.1 install and let it auto-update to v1.0.2. In this case my goal was to simulate a user that's blissfully ignorant of the orphaned entry in the Programs list as more Zui Insiders releases come out that trigger auto-update. Therefore once v1.0.2 was installed, I made v1.0.3 available as the latest in GitHub Releases. Once I next relaunched the app, saw the auto-update notice, and clicked Restart, the v1.0.3 app comes up and now the Programs list looks like:

At this point it's effectively the same as we saw the prior test, i.e., the user would uninstall the newer entry in Programs (either intentionally by clicking Uninstall on the newer one, or unintentionally by clicking Uninstall on the older one), then have to manually install the older one (v1.0.1 in this case), at which point they'd be free to install the newest Zui Insiders and it would come up with all their data still intact. However, the most important detail revealed by this test is that the orphaned/older entry appears to be effectively benign. That is, if the user is not bothered by the extra clutter in their Programs list, they'd be free to just keep accepting auto-updates to the newer Zui Insiders releases. If they ever wanted to get rid of the orphaned entry, they could do the manual multi-uninstall/reinstall at any time. This all becomes something we'd want to explain to the (thankfully smaller, and having set "beta" expectations with them already) Zui Insiders user community before/after the transition to the new appId.

macOS

After seeing the auto-update notification and clicking Restart, the app does not actually restart. An entry appears in the main.log that implies this is due to the change in appID.

[2022-10-21 13:23:20.903] [error] There was a problem updating the application: Error: Could not locate update bundle for com.electron.brim within file:///Users/phil/Library/Caches/com.electron.brim.ShipIt/update.Xujyjd4/

There's still only one Zui entry in the Applications list. Clicking it to relaunch, it comes up yet again in v1.0.1, once again pops up an auto-update notification, and the cycle repeats.

At this point the user is effectively stuck at the old version until they get the hint that they need to manually uninstall to get rid of the old version, then manually install the newer version. Once I did this uninstall and manually installed v1.0.2, launching put me in the desired state of running v1.0.2 with all of my data still intact. So while it may have the potential to briefly confuse, this actually seems like a decent state for things to be in, since a confused user would hopefully look to the Releases Notes and/or Slack for guidance, or just attempt the manual uninstall/reinstall themselves and see it succeed.

Once in v1.0.2, making v1.0.3 the latest available has the user back at the typical/desirable auto-update workflow: The auto-update notice arrives, then clicking Restart automatically installs the new release and restarts the app into v1.0.3 with all data intact.

Linux

As with the Brim->Zui tests, Linux is unique in that auto-update is not available. Performing manual upgrades, testing showed no user-facing impact of the change in appId. Installing v1.0.1, v1.0.2, and v1.0.3 in sequence with dpkg had each installed "over" the prior, with only one entry ever present in the list of installed programs and the User Data always remaining intact between versions. If an upgrade is attempted via the Files utility (e.g., when v1.0.1 was installed and the .deb packages for v1.0.2 was double-clicked), the information for the newer release is shown but an Install button is not offered, which would hopefully nudge the user toward the recommended workflow of uninstalling the prior release before installing the newer one.

Action Plan

Summarizing all the results above, assuming no reviewers find holes in the results or propose alternatives, here's my intended action plan.

For Brim->Zui Transition

I'd plan to write an article in the Support Resources section of the new Zui docs to set user expectations. It would touch on:

1. The Brim User Data being left behind, and advising the user to manually archive/delete this once they're satisfied that they see all their migrated data intact in Zui.
2. Advice to manually uninstall Brim on macOS if they manually installed Zui.
3. Advice to manually uninstall Brim on Linux.

When we draft Release Notes for Zui v1.0.0 we could link to this article. However, as the testing reveals that nearly all users should expect a seamless transition, I don't think we should go out of our way to draw excess attention to it.

For Zui Insiders Side-By-Side Transition

I'd plan to draft Release Notes that I'd post first in the #zui-insiders channel on Slack. These would explain:

1. The presence of the limitation that has prevented the running of Brim and Zui Insiders side-by-side on Windows
2. The need for us to establish a unique application identifier in Zui Insiders to resolve the issue
3. That the change to the new application identifier will result in some one-time manual steps to complete the transition
4. Once the transition is complete, Brim/Zui and Zui Insiders should be able to run side-by-side on all platforms, and no problems with Zui Insiders updates are expected going forward

And the per-platform guidance would be:

• Windows - The presence of the benign entry in the Programs list for the old Zui Insiders, which could be purged through the manual reinstall/uninstall of the old version before installing the newest Zui Insiders
• macOS - The need to manually uninstall the old Zui Insiders before a manual install of the new Zui Insiders will succeed
• Linux - No extra steps expected. The recommendation is to manually uninstall the old Zui Insiders before manually installing the newer one, as always.

After giving users a few days to absorb this info and ask any questions, the appId change would be merged in the Brim repo and a new Zui Insiders build triggered. Once created, the Release Notes for that new version would be populated with the same text posted on Slack, and we'd also Tweet an announcement briefly mentioning the change as relevant to Zui Insiders users and linking to the Release Notes on GitHub.

[philrz]

Verified in Zui Insiders 116.

Now that the appID transition is complete, this screenshot shows that on Windows we can have Zui Insiders and Brim open at the same time with their respective separate Zed lake storage. The entries for both are visible in the Programs list.

[philrz]

I'm revisiting this topic one more time because I've just been confronted with a subtle Windows-specific problem with the way things are headed right now for the Zui v1.0 release.

Just to recap, the appId change for Zui Insiders was a success and users have been able to run Brim v0.31.0 and Zui Insiders side-by-side successfully for several months. Meanwhile the appId has not been changed for the app that lives in the Brim repo (soon to be the Zui repo) because I felt the observed "replace on upgrade" behavior described in #2459 (comment) was a good thing. However, the first video below shows a bad side-effect from this: While we do get the desirable behavior of the "Brim" entry going away from the installed Programs and the "Zui" one appearing in its place, the Application Binaries directory does not get renamed from "brim" to "zui" during the upgrade.

Repro.mp4

I also confirmed in a similar test in a personal fork that the same happens with auto-update.

Meanwhile, as shown in the attached video, if a user is installing Zui for the first time on a fresh system, their Application Binaries directory is named "zui" as we might expect.

Repro-just-Zui.mp4

If a user never had a reason to care about the location of the Application Binaries directory, this might be a non-issue. However, as proven by the existence of that Filesystem Paths article, there's already precedent here, e.g., when users want to find the Zed CLI tools that are compatible with their installed app so they can do some scripted operations from the shell against their Zed lake. More recently, this all became apparent because I've been writing some Zed lake migration tools to help users get through a storage format change, and the tools needed a way to find those same binaries to help with the migration.

After consulting with the team, it seems the best way forward is to move to a new/unique appId for Zui after all. Looking at the closing remarks above, indeed this option was always on the table. Back then I did not relish the idea of telling users they had to manually uninstall the old Brim and leaving them at risk of forgetting to uninstall and accidentally relaunching it and working with old data. But having users out there with a mix of installation directories seems worse. Given that the storage format change is already leading us to point users toward some manual steps during the upgrade, this is feeling like the cleaner way forward.