Implement a more intuitive Qubes Manager #1870

bnvk · Mar 23, 2016

While the current Qubes VM Manager works alright, there are a handful of bugs and it is not nearly as intuitive as it could be. Additionally, there are configuration settings that are missing from it and others that are a bit technical and overwhelming. Lastly, there are new big ideas like Recipes and installing apps that aren't taken into account at all.

UX Ideas / New Features

Splits up VMs into App Qube, Networking, Devices, Templates, Standalones
Choose applications to run by default on starting a Qube
Dropdown to choose Open URLs in [Qube Name]
Explain Power State of a Qube like Running, Paused, Dying, Crashed
Intuitive options to install software packages into Qube or Templates
Integration of Recipes that use the salt configuration #1939
Opportunistically offer links to Docs / explainer content #1774
Last started / run
Make "updating" VM action simple / less steps (restart)

Require Modifications to API

Add Icon field instead of all VMs having a lock icon #1873
Add Description field so instead of anon-whonix there is also Anonymous Web Browsing #899
Add available_disk_space for resizing of qubes sizes #2016

Considerations & Issues

UpdateVM system defaults should only list NetVMs and ProxyVMs #1831
run suspend / resume scripts also on VM pause / resume #1855
Already assigned devices should not be listed as "available" under VM Settings > Devices of other VMs #1838
stop using abbreviations for whonix templates #1778
change default proxy-vm: sys-whonix -> system-tor #1777
stop using abbreviations for default vm names #1776
change default appvm name: anon-whonix -> anonymous-browsing #1775

...there is probably loads more things and will update accordingly 😄

Added ticket reference to the VM description idea.

marmarek · Mar 23, 2016

Added ticket reference to the VM description idea.

@bnvk the issue #899 is quite important for me. I have many many VM's and I would like to have a VM description similar to what VirtualBox offers. I created some simple GUI mock-up under: https://groups.google.com/d/msg/qubes-devel/t32l-0BjlLs/w3Rir4LhAAAJ . If I can help you in any way or work together with you on issue #899 please let me know.

Note: VM description means for me a text box not a descriptive VM title.

Jeeppler · Mar 24, 2016

@bnvk the issue #899 is quite important for me. I have many many VM's and I would like to have a VM description similar to what VirtualBox offers. I created some simple GUI mock-up under: https://groups.google.com/d/msg/qubes-devel/t32l-0BjlLs/w3Rir4LhAAAJ . If I can help you in any way or work together with you on issue #899 please let me know.

Note: VM description means for me a text box not a descriptive VM title.

@Jeeppler thanks for the input and mocksup. I hadn't given much thought to Last started value, but I definitely see your point. I will try to work that into the new UI. I'm not that familiar with the VirtualBox UI, but I would be happy to checkout screenshots if you could take some describing what you mean.

I wonder if adding a user facing "group" value to VMs, such as Web Browsing or Work whereby a user would see all their web browsing related VMs (anonymous, social accounts, banking, etc...) would help. Not sure if this is ideal, but I wanted to jot the idea down!

bnvk · Mar 24, 2016

@Jeeppler thanks for the input and mocksup. I hadn't given much thought to Last started value, but I definitely see your point. I will try to work that into the new UI. I'm not that familiar with the VirtualBox UI, but I would be happy to checkout screenshots if you could take some describing what you mean.

I wonder if adding a user facing "group" value to VMs, such as Web Browsing or Work whereby a user would see all their web browsing related VMs (anonymous, social accounts, banking, etc...) would help. Not sure if this is ideal, but I wanted to jot the idea down!

@bnvk VirtualBox provides a description box like shown here: https://software.grok.lsu.edu/image/19970.gif and https://programmaticponderings.files.wordpress.com/2013/04/view-of-oracle-vm-virtualbox-manager-for-windows.png , but it is not perfect and I was already designing my own idea as mock up. I would like to see the Last started value since this could be a good indicator for the question: Do I really need this machine?

I think furthermore that markdown support in the VM description would be nice. Maybe not full markdown support, but lists, code, bold text, italic text and heading support would be nice.

Jeeppler · Mar 24, 2016

@bnvk VirtualBox provides a description box like shown here: https://software.grok.lsu.edu/image/19970.gif and https://programmaticponderings.files.wordpress.com/2013/04/view-of-oracle-vm-virtualbox-manager-for-windows.png , but it is not perfect and I was already designing my own idea as mock up. I would like to see the Last started value since this could be a good indicator for the question: Do I really need this machine?

I think furthermore that markdown support in the VM description would be nice. Maybe not full markdown support, but lists, code, bold text, italic text and heading support would be nice.

"I wonder if adding a user facing "group" value to VMs, such as Web Browsing or Work whereby a user would see all their web browsing related VMs (anonymous, social accounts, banking, etc...) would help." I completely agree with you. This would be a nice idea. I would use the "groups" for filtering and define that:

each VM can be part of 0 to N groups

for example:

python-dev is part of: "Python", "Programming", "Development", "Work"
zololo is part of: "Python", "Programming", "Development", "Work", "Customer", "Project"

Jeeppler · Mar 24, 2016

"I wonder if adding a user facing "group" value to VMs, such as Web Browsing or Work whereby a user would see all their web browsing related VMs (anonymous, social accounts, banking, etc...) would help." I completely agree with you. This would be a nice idea. I would use the "groups" for filtering and define that:

each VM can be part of 0 to N groups

for example:

python-dev is part of: "Python", "Programming", "Development", "Work"
zololo is part of: "Python", "Programming", "Development", "Work", "Customer", "Project"

~~not seeing in the list:~~

~~know network state from first level of Network Manager~~

~~the user should not have to go to VM Settings of a particular VM to know if the VM is non-networked, going through sys-whonix, sys-firewall, etc.~~

~~I guess "grouping" VMs would be a potential workaround, if you create groups based on their networking state. But then they are not being based on user domain.~~

please ignore, hadn't adequately explored the existing Qubes Manager > View menu...

mfc · Apr 28, 2016

~~not seeing in the list:~~

~~know network state from first level of Network Manager~~

~~the user should not have to go to VM Settings of a particular VM to know if the VM is non-networked, going through sys-whonix, sys-firewall, etc.~~

~~I guess "grouping" VMs would be a potential workaround, if you create groups based on their networking state. But then they are not being based on user domain.~~

please ignore, hadn't adequately explored the existing Qubes Manager > View menu...

User suggestions for improvements to the current Qubes Manager:

https://groups.google.com/d/msgid/qubes-users/57384F52.40907%40gmail.com

andrewdavidwong · May 15, 2016

User suggestions for improvements to the current Qubes Manager:

https://groups.google.com/d/msgid/qubes-users/57384F52.40907%40gmail.com

@andrewdavidwong thanks! Added that to my list + had already been working that exact thing into new Qubes Manager :)

bnvk · May 15, 2016

@andrewdavidwong thanks! Added that to my list + had already been working that exact thing into new Qubes Manager :)

a few of my ideas:
1) add collumns for

vitual cpu cores
system storage max
system storage current
system storage current usage (bar similar to cpu usage)
private storage max
private storage current
private storage current usage (bar similar to cpu usage)
mac
initial memory
max memory
one or multiple columns for vm control (buttons for start, stop, pause and update)
this is useful if the user has a long list of vms and wants to start/stop/update multiple vms without moving the mouse to the top of the window or opening the context menu several times.
maybe a column for the "run command in vm" option. (a line edit that executes the command on pressing enter)

(basically everything that could be useful)
2) sort the list when selecting a new netvm. (maybe keep none + default always at the top)
3) use a stable sorting algorithm when sorting the rows or allow secondary sort criteria (e.g. first sort by Template, then by name)
4) when setting the netvm of a running vm A to some proxy-vm B currently not running, an error message is displayed.
add a button to this message to start vm B and set it as netvm.
(the user will probably close the warning, close settings, start B, open settings of A again and set the netvm again)
5) allow the user to add an additional drive to other vms than hvms.
this enables the user to have a fast vm stored on a ssd and have some "long-term storage" as a drive located on a big hdd.
use-case:
i have a vm holding big files (videos / raw images). i only use a few files at once.
the files i use should have fast access (stay on my ssd).
the files i currently don't use should not waste space on my ssd (it only has 128 GB)
with this change i could copy the files i use to my ssd-backed drive and store files i don't use on my hdd-backed drive.
6) allow the user to constrict the list of possible netvms for a vm.
use-cases:
a) i have multiple vpn-proxyvms for work / as endpoints for my vpn-provider(s).
if i change one of the vpns i don't want to be able accidentally select sys-firewall and leak some data.
b) i only want dispvms to be able to use either tor or some vpn-A as netvm.
again, i don't want to be able to accidentally select sys-firewall or vpn-work as netvm.
7) make the qubes-manager interactive:
e.g.: it does not only display the netvm, but instead show a drop down menu to select it.
the same can be done for any setting (number of virtual cpus, disk storage, ram ....)
this would enable the user to quickly change settings without clicking through menus (think about changing the dispvms netvm).
this behavior should be optional per vm, so add some check box in the settings dialog (and maybe a column with this option)
8) make it possible to set the default app shortcuts for vms created from one template and/or add the option to change the shortcut settings for multiple vms at once (set all shortcuts, remove a certain shortcut if present, add a shortcut to the list (if the appvm provides it)).
usecase:
a) i don't want to set thunderbird as only shortcut for all of my mail-vms manually, but all at one go. (set all shortcuts)
b) i want to use terminator instead of xterm as my terminal and don't want to go through every vm, to change the setting. it want to change this for all vms at one go. (remove a certain shortcut + add a shortcut to the list => this could be done by a "replace shortcut" option)
9) don't lock the qubes-manager when opening a dialog.
usecase:
a) i want to compare the settings of two vms. (but i can't open both dialogs at the same time and put them next to each other)
b) i want to update all of my template-vms. if i press update, a dialog opens and i can't click on update for the next vm.
10) add some button to update all vms (i will be doing this anyways)
11) add an option (shut down after update) to the update dialog, displayed when the template to update is not running.
this option should be a check box and be enabled by default (i will probably shut it down after the update).
11.2) maybe add an such an option to the "run command in vm" option, too.
in this case, the check box should be enabled if the vm is currently not running and vice versa.

john-david-r-smith · May 19, 2016

a few of my ideas:
1) add collumns for

vitual cpu cores
system storage max
system storage current
system storage current usage (bar similar to cpu usage)
private storage max
private storage current
private storage current usage (bar similar to cpu usage)
mac
initial memory
max memory
one or multiple columns for vm control (buttons for start, stop, pause and update)
this is useful if the user has a long list of vms and wants to start/stop/update multiple vms without moving the mouse to the top of the window or opening the context menu several times.
maybe a column for the "run command in vm" option. (a line edit that executes the command on pressing enter)

(basically everything that could be useful)
2) sort the list when selecting a new netvm. (maybe keep none + default always at the top)
3) use a stable sorting algorithm when sorting the rows or allow secondary sort criteria (e.g. first sort by Template, then by name)
4) when setting the netvm of a running vm A to some proxy-vm B currently not running, an error message is displayed.
add a button to this message to start vm B and set it as netvm.
(the user will probably close the warning, close settings, start B, open settings of A again and set the netvm again)
5) allow the user to add an additional drive to other vms than hvms.
this enables the user to have a fast vm stored on a ssd and have some "long-term storage" as a drive located on a big hdd.
use-case:
i have a vm holding big files (videos / raw images). i only use a few files at once.
the files i use should have fast access (stay on my ssd).
the files i currently don't use should not waste space on my ssd (it only has 128 GB)
with this change i could copy the files i use to my ssd-backed drive and store files i don't use on my hdd-backed drive.
6) allow the user to constrict the list of possible netvms for a vm.
use-cases:
a) i have multiple vpn-proxyvms for work / as endpoints for my vpn-provider(s).
if i change one of the vpns i don't want to be able accidentally select sys-firewall and leak some data.
b) i only want dispvms to be able to use either tor or some vpn-A as netvm.
again, i don't want to be able to accidentally select sys-firewall or vpn-work as netvm.
7) make the qubes-manager interactive:
e.g.: it does not only display the netvm, but instead show a drop down menu to select it.
the same can be done for any setting (number of virtual cpus, disk storage, ram ....)
this would enable the user to quickly change settings without clicking through menus (think about changing the dispvms netvm).
this behavior should be optional per vm, so add some check box in the settings dialog (and maybe a column with this option)
8) make it possible to set the default app shortcuts for vms created from one template and/or add the option to change the shortcut settings for multiple vms at once (set all shortcuts, remove a certain shortcut if present, add a shortcut to the list (if the appvm provides it)).
usecase:
a) i don't want to set thunderbird as only shortcut for all of my mail-vms manually, but all at one go. (set all shortcuts)
b) i want to use terminator instead of xterm as my terminal and don't want to go through every vm, to change the setting. it want to change this for all vms at one go. (remove a certain shortcut + add a shortcut to the list => this could be done by a "replace shortcut" option)
9) don't lock the qubes-manager when opening a dialog.
usecase:
a) i want to compare the settings of two vms. (but i can't open both dialogs at the same time and put them next to each other)
b) i want to update all of my template-vms. if i press update, a dialog opens and i can't click on update for the next vm.
10) add some button to update all vms (i will be doing this anyways)
11) add an option (shut down after update) to the update dialog, displayed when the template to update is not running.
this option should be a check box and be enabled by default (i will probably shut it down after the update).
11.2) maybe add an such an option to the "run command in vm" option, too.
in this case, the check box should be enabled if the vm is currently not running and vice versa.

A suggestion: simplify PCI passthrough. According to the documentation[1] you have to issue several commands to get a VM direct access to a PCI(-e) device, e.g. graphic cards. This is useful for people that play games (everyday computer) and engineers using CAD software.

[1] https://www.qubes-os.org/doc/assigning-devices/

ghost · May 19, 2016

A suggestion: simplify PCI passthrough. According to the documentation[1] you have to issue several commands to get a VM direct access to a PCI(-e) device, e.g. graphic cards. This is useful for people that play games (everyday computer) and engineers using CAD software.

[1] https://www.qubes-os.org/doc/assigning-devices/

@dumbl3d0re: GPU passthrough isn't currently supported.

andrewdavidwong · May 19, 2016

@dumbl3d0re: GPU passthrough isn't currently supported.

Arqwer:

R3.0
1 After I open backup dialog, qubes manager fades out. It is
uncomfortable: I might want to use it. For example, I selected which VMs
i'm going to backup, and realized, that my external drive is not connected
now. I want to use qubes manager to attach external drive, but it is
unavailable. Or I want to shut down some VMs, or start VM to send backup
there.

2 Backup process does not block me from turning on VM that is now in
backup process. That makes no sense. If VM should be turned off during
backup, I should not be able to turn it on. At least without --force
flags.

andrewdavidwong · May 30, 2016

Arqwer:

R3.0
1 After I open backup dialog, qubes manager fades out. It is
uncomfortable: I might want to use it. For example, I selected which VMs
i'm going to backup, and realized, that my external drive is not connected
now. I want to use qubes manager to attach external drive, but it is
unavailable. Or I want to shut down some VMs, or start VM to send backup
there.

2 Backup process does not block me from turning on VM that is now in
backup process. That makes no sense. If VM should be turned off during
backup, I should not be able to turn it on. At least without --force
flags.

@bnvk could you maybe publish a screenshot of your current development state?

Jeeppler · May 30, 2016

@bnvk could you maybe publish a screenshot of your current development state?

Suggestion regarding VM organization in Qubes Manager:
https://groups.google.com/d/msgid/qubes-users/CAPzH-qCmFqNvxVCwuCsgMsZZ2ufHExmiLs%3DuqWRiN0n1bf1K0g%40mail.gmail.com

andrewdavidwong · May 30, 2016

Suggestion regarding VM organization in Qubes Manager:
https://groups.google.com/d/msgid/qubes-users/CAPzH-qCmFqNvxVCwuCsgMsZZ2ufHExmiLs%3DuqWRiN0n1bf1K0g%40mail.gmail.com

@john-david-r-smith @dumbl3d0re @Jeeppler thank you for your feedback and suggestions. Great stuff in there @andrewdavidwong thanks for pulling in stuff from the mailing lists.

I made progress towards this issue, however, as of last week I am not officially working on it any longer as per @marmarek feedback / decision. Reasoning being: I am not a competent / fastest enough Python + Gtk developer (yet) and Qubes does not have the budget to hire additional engineers.

All the code I created exists in this repo and is signed with this key. I don't have wireframes or design comps (I was instructed to go straight to code) of what I was aiming for, so I will explain a bit:

An intuitive window that feels closer to a file system (as opposed to a list of VMs) that is categorized by qube type (AppVM, NetVM, TemplateVM, etc...). There is a secondary bar relevant to selected qube type (App Qube --> Attach Microphone) that are relevant to the current selected qube type. The biggest missing aspect is "qube state" (Running, Halted, etc...) as there were a few discussions / issues that would affect how this should be implemented best, IMHO see #1015 #1788 #1873

Doubling-clicking on a qube icon in the manager (pictured above) would open the following window:

This window contains common qube type specific actions () and is similar to the "Basic" tab in the current VM Manager. This view was not meant to fully configure a qube, as those actions would happen via the multi-tabbed Advanced window:

The other high level "sections" of the Qubes Manager are common, but discreetly different things, a user may interact with (Recipes, Backups, Global Settings) of which I implemented the later.

Most of these interfaces are still quite rough visually and have no integration with the actual Qubes dom0 API because the fedora-20 is not compatible with Glade (the Gtk GUI app) and fedora-23 not available for dom0 when I started. There is a rough start towards Recipes #1939 interface here:

In my repo is a list of ToDo Items that @marmarek and I outlined as per this current implementation.

At present I am unsure of how the team wants to move forward, and who will keep working through these issues. Given my primary skillset being a designer who's most competent at web engineering, but is now moderately familiar with Gtk, Glade, and Python, the few directions I see this going are:

I do not touch any of the code (python or Glade) while the core team finds someone else handle that. In which case, I can provide feedback & ideas via sketches and UI compositions
I only work within Glade to add elements and polish how things look (margins, fonts, icons, etc...)
I still do a bit of python as well as Glade, but is subject to review

All that said, since working on this will no longer be part of my paid salary, I am unsure how much time I will be able to commit to this, but I can try to help. I was very excited about where this is going and would love to see it implemented.

bnvk · Jun 1, 2016