Updater improvements

[tumbletree]

The problem you're addressing (if any)
Qubes Updater is:

• opaque
• uninformative
• unresponsive
• prone to crashing, still. About once a month, hard reboot required.

It can be disruptive and distracting, but its essential. It should be better.

Updating is not trivial. At best, restarting all relevant qubes (sys-whatever + dependent vms) individually is very time-consuming, so a system restart is almost always required. Since restart takes so long (> 5 minutes), updating is not trivial.

The user is not given any tools by which they can manage the intrusion onto their workflow. There is no indication of the urgency, gravity and/or security of the updates - no idea of the packages at all. How can a user decide when to update? Is this "drop everything" or "when you can / overnight" priority?

Describe the solution you'd like

• It shouldn’t crash the machine.
• “Details” should display full process in real time, just like sudo dnf/apt upgrade dialog
• “Cancel” should do exactly that. It doesn’t. It still chugs along anyway.
• Security / urgent updates labelled as such. Even a "traffic light" system.
• You shouldn’t need to click a checkbox to enable clicking on templates you want to force an upgrade. That should be enabled automatically.
• mouse-over each template should reveal what packages are going to be upgraded.
• Qubes Security Bulletins (or their notification) should be distributed via the Updater. This is important information that should be "pushed", not rely on external services to "pull". [edit: added 2020-12-31]

Where is the value to a user, and who might that user be?

Everyone benefits, particularly

• those on older/lower-spec hardware (specifically thinking of the restart times, but maybe also non-responsive issues)
• those for whom disruption to workflow represents a higher burden than others.
• (perhaps better compliance with updates? Which might be better for those who filed questions from unhappy users?)

Describe alternatives you've considered
sudo apt/dnf update && sudo apt/dnf upgrade

Additional context

Relevant documentation you've consulted

I gather salt is an issue. I can't understand ( or devote the time to understand) that matter, but surely there has to be something.

Related, non-duplicate issues

Updater fail

[andrewdavidwong]

I've been thinking about this lately too. Ideally, I think Qubes should install all mandatory updates (especially mandatory critical security updates) automatically, in the background, without any user awareness (much less action) required. There should be options for not downloading updates over metered connections and for advanced users to opt out entirely. There should also be options for downloading (but not installing) non-mandatory updates and automatically installing all updates. (The appropriate defaults for non-mandatory updates are more open to debate.)

One of the biggest pain points is that updating can take a very long time, especially with many templates, especially over Tor. There's no reason that a human should be have to sit and wait while update data is being downloaded. It should be downloaded in the background (when appropriate, as unobtrusively as possible).

Another problem is users not knowing what exactly to do or not taking the appropriate action at the right time when a critical security update is released, such as when a new QSB is issued. For virtually all normal users, we already know that they want to install security updates when those updates are stable (and some even earlier, i.e., from security-testing). So, the logical default is for all stable security updates to be installed automatically. Qubes should be secure by default and keep itself that way by default.

Xen and kernel updates will require a dom0 reboot, but requiring a reboot after certain updates is nothing new for an operating system. Just handle this in the usual way: by notifying the user that a reboot is required and letting them choose when to do it. (Of course, the system should never forcibly reboot. The user gets to control the system; not the other way around. Which is, again, why there should also be an option to opt out of all of this.)

[DemiMarie]

Xen and kernel updates will require a dom0 reboot, but requiring a reboot after certain updates is nothing new for an operating system. Just handle this in the usual way: by notifying the user that a reboot is required and letting them choose when to do it. (Of course, the system should never forcibly reboot. The user gets to control the system; not the other way around. Which is, again, why there should also be an option to opt out of all of this.)

I agree. That said, it would also be nice to have an option for, “if it’s 2AM (say), reboot automatically if needed”.

The hardest part of this is going to be Salt. Salt is designed for configuration management at massive scale, where watching individual updates happen is neither practical nor desirable. That certainly is not the case for QubesOS! To be honest, I wonder if Salt is the wrong tool for the job here. It can enforce extremely complex configurations, but updates require only a tiny fraction of that complexity. Furthermore, it is designed for unattended operation, whereas in the case of QubesOS feedback is highly desirable and sometimes even required. Arch Linux officially does not support unattended upgrades, and not looking at the output from Pacman can cause problems.

A better approach is to run a shell script in the VM, with its stdin/out/err connected to an xterm. Cancellation is then just a matter of Ctrl-C, and update progress is clearly visible. The overhead of Salt and the management VM is eliminated. Since the script would be supplied by dom0, vulnerabilities in the VM’s package manager can be mitigated as needed. And we can make updates interactive for those who desire that.

[SvenSemmler]

On 12/22/20 12:17 PM, Demi Marie Obenour wrote: I wonder if Salt is the wrong tool for the job here.

Very much agree.

A better approach is to run a shell script in the VM, with its stdin/out/err connected to an xterm.

I always wondered why Salt was chosen for the update process. I used the updater a few times but found it slow/unresponsive and difficult to understand what's going on. Since then I simply use it as an indicator: when I see the little start in the toolbar, I open a dom0 terminal and kick my bash script/minimize the window. The updater GUI could run similar scripts and be way more responsive that way, plus one could easily see which packages will be updated.

[andrewdavidwong]

I always wondered why Salt was chosen for the update process. I used the
updater a few times but found it slow/unresponsive and difficult to
understand what's going on. Since then I simply use it as an indicator:
when I see the little start in the toolbar, I open a dom0 terminal and
kick my bash script/minimize the window.

Well, the problem is that the Salt updates do more than just run dnf upgrade or apt upgrade in the VM. They can also apply other security fixes:

https://groups.google.com/g/qubes-devel/c/n14n3Q295rw/

[DemiMarie]

I always wondered why Salt was chosen for the update process. I used the
updater a few times but found it slow/unresponsive and difficult to
understand what's going on. Since then I simply use it as an indicator:
when I see the little start in the toolbar, I open a dom0 terminal and
kick my bash script/minimize the window.

Well, the problem is that the Salt updates do more than just run dnf upgrade or apt upgrade in the VM. They can also apply other security fixes:

https://groups.google.com/g/qubes-devel/c/n14n3Q295rw/

Indeed they can, which is why I suggested running a shell script provided by dom0 instead.

[andrewdavidwong]

I always wondered why Salt was chosen for the update process. I used the
updater a few times but found it slow/unresponsive and difficult to
understand what's going on. Since then I simply use it as an indicator:
when I see the little start in the toolbar, I open a dom0 terminal and
kick my bash script/minimize the window.

Well, the problem is that the Salt updates do more than just run dnf upgrade or apt upgrade in the VM. They can also apply other security fixes:
https://groups.google.com/g/qubes-devel/c/n14n3Q295rw/

Indeed they can, which is why I suggested running a shell script provided by dom0 instead.

Right, I was just replying specifically to @SvenSemmler's quoted remarks there, because I used to do exactly the same thing before asking about it in the linked thread.

[ninavizz]

+1 to all of this! I've been stuck in updater-hell for the past 2 weeks on my own Qubes laptop... and rethinking the updater to make it more user-friendly, to better manage tasks between torified and non-torified network connections, to more efficiently use dom0 disk space w/o requiring disk space partitioning rejiggers, to automagically manage downloads and reboots when I'm not using the machine, and to offer me the option of seeing EVERYTHING in real-time and in a historic view, would be awesome.

The technical particulars aren't my game, so I'm not going to delve into the Google Groups chats on the topic. But, I would like to approach an updater-ecosystem GUI rethink at some point in 2021.

@andrewdavidwong could this issue also get a UX tag on it?

[andrewdavidwong]

@andrewdavidwong could this issue also get a UX tag on it?

Added!

[tumbletree]

Added a point re QSBs.

Notification of QSB (link to the should (IMHO) be pushed out in the Updater.

They're important, legitimate, system-relevant parcels of info. A webpage I need to specially go check, (I'm not going to remember) or a third party distribution service (and as of writing, I still only know of @andrewdavidwong's relaying this stuff to the subreddit) might have been okay for the early days of an experimental OS, but not now.

If its important info on running the OS, it should be delivered in the OS (trust, efficiency, fidelity, etc).