Qubes-Whonix 14 SaltStack state files required

[adrelanos]

I am not sure it was a good idea to go for versioned (suffixing -14) Whonix template names.

• We now have the issue that salt uses non-versioned hardcoded values.
• However, [instructions for testing newly built Qubes-Whonix 14 templates] need to refer to the versioned template names.
• Qubes-Whonix manual setup got to complicated that we now advice users to use salt for it. (#3447) So telling users to do that manually is not an option either.

Perhaps we should revert this for the next template release and go back to non-versioned template names? Do you see a better solution for this?

[mirrorway]

Isn't the natural solution to just modify and version the salt too? qubesctl state.sls qvm.anon-whonix-14 dom0 In the future, the salt setup could be different for different major versions of whonix.

There are so many benefits to having the version in the names:

• Can see at a glance which package or template is 13 vs 14. Mis-identification could mean spending an hour downloading the wrong package, or testing the wrong version.
• Consistency with fedora and debian template naming.
• Can easily install new whonix templates to test without worrying about overwriting the old ones that work. In the long run, wouldn't this simplify the installation? fewer VMs to rename and fewer RPMs to remove.

[mirrorway]

Isn't the natural solution to just modify and version the salt too? qubesctl state.sls qvm.anon-whonix-14 dom0 In the future, the salt setup could be different for different major versions of whonix.

There are so many benefits to having the version in the names:

• Can see at a glance which package or template is 13 vs 14. Mis-identification could mean spending an hour downloading the wrong package, or testing the wrong version.
• Consistency with fedora and debian template naming.
• Can easily install new whonix templates to test without worrying about overwriting the old ones that work. In the long run, wouldn't this simplify the installation? fewer VMs to rename and fewer RPMs to remove.

[adrelanos]

Shall I just duplicate all files with names containing whonix on https://github.com/QubesOS/qubes-mgmt-salt-dom0-virtual-machines/tree/master/qvm and add -14 to the file names and file content where appropriate? @marmarek

[adrelanos]

Shall I just duplicate all files with names containing whonix on https://github.com/QubesOS/qubes-mgmt-salt-dom0-virtual-machines/tree/master/qvm and add -14 to the file names and file content where appropriate? @marmarek

[marmarek]

Salt can use variables, and those can be set pillars (separate files, also enabled/disabled with top.enable/top.disable). We use this for setting either separate sys-usb, or combined with sys-net, then when referencing USB VM, use {{ salt['pillar.get']('qvm:sys-usb:name', 'sys-usb') }} (sys-usb here is default value if it isn't defined in pillar). pillar file

We can do the same with Whonix - use variable like qvm:whonix:version_suffix (with either empty value or -14, later -15 etc). Then use it everywhere when referencing whonix-gw or whonix-ws templates (and probably also whonix-ws-dvm name). And provide files to set some particular value. This way, using non-default version would require just qubesctl top.enable qvm.whonix-14 before qubesctl state.sls qvm.anon-whonix.

We can do the same with other Whonix-related VMs (sys-whonix, anon-whonix), but IMO by default those should stay with current names. Separate variable?

[marmarek]

Salt can use variables, and those can be set pillars (separate files, also enabled/disabled with top.enable/top.disable). We use this for setting either separate sys-usb, or combined with sys-net, then when referencing USB VM, use {{ salt['pillar.get']('qvm:sys-usb:name', 'sys-usb') }} (sys-usb here is default value if it isn't defined in pillar). pillar file

We can do the same with Whonix - use variable like qvm:whonix:version_suffix (with either empty value or -14, later -15 etc). Then use it everywhere when referencing whonix-gw or whonix-ws templates (and probably also whonix-ws-dvm name). And provide files to set some particular value. This way, using non-default version would require just qubesctl top.enable qvm.whonix-14 before qubesctl state.sls qvm.anon-whonix.

We can do the same with other Whonix-related VMs (sys-whonix, anon-whonix), but IMO by default those should stay with current names. Separate variable?

[adrelanos]

We can do the same with other Whonix-related VMs (sys-whonix, anon-whonix), but IMO by default those should stay with current names.

Sounds good!

We can do the same with other Whonix-related VMs (sys-whonix, anon-whonix), but IMO by default those should stay with current names.

I am divided. Should we go for

---

(A)

just keep

• sys-whonix
• anon-whonix

Questions:

• What would happen if someone major upgrades (Whonix 14 -> 15) using salt? No issues?
• Template whonix-gw-14 and whonix-ws-14 would simply be no longer used? Salt would change anon-whonix and sys-whonix to whonix-(gw|ws)-15?

Advantages:

• easy to persist Tor entry guards (important anonymity providing feature)

Disadvantages:

• Doesn't allow parallel usage of two Whonix versions at once. For testers which are very important. Would be possible in theory but instructions would look very complicated / messy. (Instructions: "First install using salt. (Required due to #3447.) Then create sys-whonix-15, then somehow set salt settings, then set back the template of your sys-whonix-14 to whonix-gw-14.)
• Installation of newer Whonix version using salt would change (sys|anon)-whonix templates to whonix-(gw|ws) without asking the user.

---

or
(B)

• sys-whonix-14
• anon-whonix-14

And later...

• sys-whonix-15
• anon-whonix-15

Disadvantages:

• not easily persisting Tor entry guards
• messy instructions for persisting Tor entry guards (Instructions: "let your sys-whonix-14 VM use whonix-gw-15 as its TemplateVM")
• Perhaps we tell users how to copy over their Tor state data folder (for Tor entry guards) and Tor config using qvm-copy-to-vm instead? (Instructions: "Start sys-whonix-15. Click nothing in anon-connection-wizard. Start sys-whonix-14. Run these qvm-copy-to-vm commands etc. for migration.")
• looks strange when someone does an in-place major (Whonix 14 -> 15) upgrade since the old names would persist (whonix-gw-14, whonix-ws-14, sys-whonix-14, anon-whonix-14)
  • --> Or should we just explain in in-place upgrade instructions to change TemplateBasedVM names and TemplateVM names? Is that even possible for templates installed using rpm?

Advantages:

• can provide easy, clean, clear call for testers blog posts
• easy new installation without persisting of data
• sys-whonix-15 cleanly uses whonix-gw-15 etc.

---

I think we should go for (B).

[adrelanos]

We can do the same with other Whonix-related VMs (sys-whonix, anon-whonix), but IMO by default those should stay with current names.

Sounds good!

We can do the same with other Whonix-related VMs (sys-whonix, anon-whonix), but IMO by default those should stay with current names.

I am divided. Should we go for

---

(A)

just keep

• sys-whonix
• anon-whonix

Questions:

• What would happen if someone major upgrades (Whonix 14 -> 15) using salt? No issues?
• Template whonix-gw-14 and whonix-ws-14 would simply be no longer used? Salt would change anon-whonix and sys-whonix to whonix-(gw|ws)-15?

Advantages:

• easy to persist Tor entry guards (important anonymity providing feature)

Disadvantages:

• Doesn't allow parallel usage of two Whonix versions at once. For testers which are very important. Would be possible in theory but instructions would look very complicated / messy. (Instructions: "First install using salt. (Required due to #3447.) Then create sys-whonix-15, then somehow set salt settings, then set back the template of your sys-whonix-14 to whonix-gw-14.)
• Installation of newer Whonix version using salt would change (sys|anon)-whonix templates to whonix-(gw|ws) without asking the user.

---

or
(B)

• sys-whonix-14
• anon-whonix-14

And later...

• sys-whonix-15
• anon-whonix-15

Disadvantages:

• not easily persisting Tor entry guards
• messy instructions for persisting Tor entry guards (Instructions: "let your sys-whonix-14 VM use whonix-gw-15 as its TemplateVM")
• Perhaps we tell users how to copy over their Tor state data folder (for Tor entry guards) and Tor config using qvm-copy-to-vm instead? (Instructions: "Start sys-whonix-15. Click nothing in anon-connection-wizard. Start sys-whonix-14. Run these qvm-copy-to-vm commands etc. for migration.")
• looks strange when someone does an in-place major (Whonix 14 -> 15) upgrade since the old names would persist (whonix-gw-14, whonix-ws-14, sys-whonix-14, anon-whonix-14)
  • --> Or should we just explain in in-place upgrade instructions to change TemplateBasedVM names and TemplateVM names? Is that even possible for templates installed using rpm?

Advantages:

• can provide easy, clean, clear call for testers blog posts
• easy new installation without persisting of data
• sys-whonix-15 cleanly uses whonix-gw-15 etc.

---

I think we should go for (B).

[marmarek]

(A)

Questions:

What would happen if someone major upgrades (Whonix 14 -> 15) using salt? No issues?
Template whonix-gw-14 and whonix-ws-14 would simply be no longer used? Salt would change anon-whonix and sys-whonix to whonix-(gw|ws)-15?

Depending how salt is written - currently it will not touch existing sys-whonix and anon-whonix, but it is trivial to change that. And IMO it is a good idea to adjust it.

Doesn't allow parallel usage of two Whonix versions at once. For testers which are very important. Would be possible in theory but instructions would look very complicated / messy. (Instructions: "First install using salt. (Required due to #3447.) Then create sys-whonix-15, then somehow set salt settings, then set back the template of your sys-whonix-14 to whonix-gw-14.)

We can create yet another pillar for such cases ("whonix-vms-suffix" or such).

(B)

And later...

sys-whonix-15
anon-whonix-15

This option also means the need for manually changing netvm sys-whonix-14->sys-whonix-15 for all other qubes using whonix.

Or should we just explain in in-place upgrade instructions to change TemplateBasedVM names and TemplateVM names? Is that even possible for templates installed using rpm?

It isn't possible to rename (in 4.0 not even for non-templates), but it is always possible to clone.

[marmarek]

(A)

Questions:

What would happen if someone major upgrades (Whonix 14 -> 15) using salt? No issues?
Template whonix-gw-14 and whonix-ws-14 would simply be no longer used? Salt would change anon-whonix and sys-whonix to whonix-(gw|ws)-15?

Depending how salt is written - currently it will not touch existing sys-whonix and anon-whonix, but it is trivial to change that. And IMO it is a good idea to adjust it.

Doesn't allow parallel usage of two Whonix versions at once. For testers which are very important. Would be possible in theory but instructions would look very complicated / messy. (Instructions: "First install using salt. (Required due to #3447.) Then create sys-whonix-15, then somehow set salt settings, then set back the template of your sys-whonix-14 to whonix-gw-14.)

We can create yet another pillar for such cases ("whonix-vms-suffix" or such).

(B)

And later...

sys-whonix-15
anon-whonix-15

This option also means the need for manually changing netvm sys-whonix-14->sys-whonix-15 for all other qubes using whonix.

Or should we just explain in in-place upgrade instructions to change TemplateBasedVM names and TemplateVM names? Is that even possible for templates installed using rpm?

It isn't possible to rename (in 4.0 not even for non-templates), but it is always possible to clone.

[adrelanos]

Marek Marczykowski-Górecki:

(A) > Questions: > > What would happen if someone major upgrades (Whonix 14 -> 15) using salt? No issues? > Template whonix-gw-14 and whonix-ws-14 would simply be no longer used? Salt would change anon-whonix and sys-whonix to whonix-(gw|ws)-15? Depending how salt is written - currently it will not touch existing sys-whonix and anon-whonix, but it is trivial to change that. And IMO it is a good idea to adjust it.

Ok.

> Doesn't allow parallel usage of two Whonix versions at once. For testers which are very important. Would be possible in theory but instructions would look very complicated / messy. (Instructions: "First install using salt. (Required due to #3447.) Then create sys-whonix-15, then somehow set salt settings, then set back the template of your sys-whonix-14 to whonix-gw-14.) We can create yet another pillar for such cases ("whonix-vms-suffix" or such).

Since we're scarce on developer time, and since I don't speak `salt`, I suggest to go for the shortest / easiest implementation path. Also the term the most easy to understand solution (including for me when getting support requests) would be preferred.

(B) > And later... > > sys-whonix-15 > anon-whonix-15 This option also means the need for manually changing netvm sys-whonix-14->sys-whonix-15 for all other qubes using whonix.

Understood. Makes sense. So this would need to be covered in upgrade instructions. Wondering which would be the last confusing outcome of the user... I guess.... Can/should there be a salt command to change 14->15 netvm setting with this as well? Perhaps a separate ticket? I prefer (B). Could you implement this please? Implementation details (A) vs (B), etc. are of course up to you.

[adrelanos]

Marek Marczykowski-Górecki:

(A) > Questions: > > What would happen if someone major upgrades (Whonix 14 -> 15) using salt? No issues? > Template whonix-gw-14 and whonix-ws-14 would simply be no longer used? Salt would change anon-whonix and sys-whonix to whonix-(gw|ws)-15? Depending how salt is written - currently it will not touch existing sys-whonix and anon-whonix, but it is trivial to change that. And IMO it is a good idea to adjust it.

Ok.

> Doesn't allow parallel usage of two Whonix versions at once. For testers which are very important. Would be possible in theory but instructions would look very complicated / messy. (Instructions: "First install using salt. (Required due to #3447.) Then create sys-whonix-15, then somehow set salt settings, then set back the template of your sys-whonix-14 to whonix-gw-14.) We can create yet another pillar for such cases ("whonix-vms-suffix" or such).

Since we're scarce on developer time, and since I don't speak `salt`, I suggest to go for the shortest / easiest implementation path. Also the term the most easy to understand solution (including for me when getting support requests) would be preferred.

(B) > And later... > > sys-whonix-15 > anon-whonix-15 This option also means the need for manually changing netvm sys-whonix-14->sys-whonix-15 for all other qubes using whonix.

Understood. Makes sense. So this would need to be covered in upgrade instructions. Wondering which would be the last confusing outcome of the user... I guess.... Can/should there be a salt command to change 14->15 netvm setting with this as well? Perhaps a separate ticket? I prefer (B). Could you implement this please? Implementation details (A) vs (B), etc. are of course up to you.

[adrelanos]

Related:
whonix-ws-based VMs lack the anon-vm tag
#3595

[adrelanos]

Related:
whonix-ws-based VMs lack the anon-vm tag
#3595

[adrelanos]

Do you think you could help with this one? @viq

[adrelanos]

Do you think you could help with this one? @viq

[viq]

Yeah, I'll need to dig some into how qubes does things under the mask, but I should be able to help some here. But first, let me refresh the discussion a bit, to make sure I have the details right, and understand the approach.

1. Template versioning
   For me it makes most sense to treat it like we do fedora, debian etc, so I have no issue with having whonix-gw-14 next to whonix-gw-15, which also potentially makes it easier to test new versions.

2. AppVM versioning
   Unless what lives on /rw/ changes in significant ways (does it? I don't know enough about whonix currently, and what changes 14 brings) then I would keep with the theme of other AppVMs and not use versioning

3. Testing new whonix versions
   I don't see much issue here, either clone or set up a fresh sys-whonix-next (and there could be states for that) and on a couple AppVMs set that as NetVM. As for what to use for updating machines through, I'd need to find where and how it's set, and again that could be automated - but we'd need to find a decent place and way to set the value of that variable for salt to use.

[viq]

Yeah, I'll need to dig some into how qubes does things under the mask, but I should be able to help some here. But first, let me refresh the discussion a bit, to make sure I have the details right, and understand the approach.

1. Template versioning
   For me it makes most sense to treat it like we do fedora, debian etc, so I have no issue with having whonix-gw-14 next to whonix-gw-15, which also potentially makes it easier to test new versions.

2. AppVM versioning
   Unless what lives on /rw/ changes in significant ways (does it? I don't know enough about whonix currently, and what changes 14 brings) then I would keep with the theme of other AppVMs and not use versioning

3. Testing new whonix versions
   I don't see much issue here, either clone or set up a fresh sys-whonix-next (and there could be states for that) and on a couple AppVMs set that as NetVM. As for what to use for updating machines through, I'd need to find where and how it's set, and again that could be automated - but we'd need to find a decent place and way to set the value of that variable for salt to use.

[adrelanos]

Thanks so much for helping out! viq:

Yeah, I'll need to dig some into how qubes does things under the mask, but I should be able to help some here. But first, let me refresh the discussion a bit, to make sure I have the details right, and understand the approach.

Great!

1. Template versioning For me it makes most sense to treat it like we do fedora, debian etc, so I have no issue with having `whonix-gw-14` next to `whonix-gw-15`, which also potentially makes it easier to test new versions.

Alright.

2. AppVM versioning Unless what lives on `/rw/` changes in significant ways (does it? I don't know enough about whonix currently, and what changes `14` brings) then I would keep with the theme of other AppVMs and not use versioning

Actually, it doesn't change in incompatible ways. So let's drop AppVM versioning.

3. Testing new whonix versions I don't see much issue here, either clone or set up a fresh `sys-whonix-next` (and there could be states for that) and on a couple AppVMs set that as NetVM. As for what to use for updating machines through, I'd need to find where and how it's set, and again that could be automated - but we'd need to find a decent place and way to set the value of that variable for salt to use.

Ok.

[adrelanos]

Thanks so much for helping out! viq:

Yeah, I'll need to dig some into how qubes does things under the mask, but I should be able to help some here. But first, let me refresh the discussion a bit, to make sure I have the details right, and understand the approach.

Great!

1. Template versioning For me it makes most sense to treat it like we do fedora, debian etc, so I have no issue with having `whonix-gw-14` next to `whonix-gw-15`, which also potentially makes it easier to test new versions.

Alright.

2. AppVM versioning Unless what lives on `/rw/` changes in significant ways (does it? I don't know enough about whonix currently, and what changes `14` brings) then I would keep with the theme of other AppVMs and not use versioning

Actually, it doesn't change in incompatible ways. So let's drop AppVM versioning.

3. Testing new whonix versions I don't see much issue here, either clone or set up a fresh `sys-whonix-next` (and there could be states for that) and on a couple AppVMs set that as NetVM. As for what to use for updating machines through, I'd need to find where and how it's set, and again that could be automated - but we'd need to find a decent place and way to set the value of that variable for salt to use.

Ok.

[adrelanos]

Should salt run qvm-features-request whonix-ws=1 as well? (#3595) (T791) @marmarek

[adrelanos]

Should salt run qvm-features-request whonix-ws=1 as well? (#3595) (T791) @marmarek

[marmarek]

No, but some whonix-ws package should place a file in /etc/qubes/post-install.d that would call it. This way it will be done automatically during template installation and update, regardless of used method. No action for this needed from salt.
The only requirement is that qubes-core-admin-addon-whonix is installed.

[marmarek]

No, but some whonix-ws package should place a file in /etc/qubes/post-install.d that would call it. This way it will be done automatically during template installation and update, regardless of used method. No action for this needed from salt.
The only requirement is that qubes-core-admin-addon-whonix is installed.

[viq]

@marmarek you mean the file and package should be present in dom0, or in a template?

[viq]

@marmarek you mean the file and package should be present in dom0, or in a template?

[marmarek]

/etc/qubes/post-install.d/(something).sh in the template, qubes-core-admin-addon-whonix in dom0.

[marmarek]

/etc/qubes/post-install.d/(something).sh in the template, qubes-core-admin-addon-whonix in dom0.

[adrelanos]

The appropriate package would be https://github.com/Whonix/qubes-whonix.

[adrelanos]

The appropriate package would be https://github.com/Whonix/qubes-whonix.

[adrelanos]

https://github.com/QubesOS/qubes-core-agent-linux/tree/master/post-install.d

[adrelanos]

https://github.com/QubesOS/qubes-core-agent-linux/tree/master/post-install.d

[adrelanos]

Just now added. Let me know if file name and contents looks alright.

https://github.com/Whonix/qubes-whonix/blob/master/etc/qubes/post-install.d/30-whonix-ws.sh

Looked easy and I like sorting the easy things asp.
Later some commentary can be added (low priority). If it looks alright, I'll upload and upgraded package.

[adrelanos]

Just now added. Let me know if file name and contents looks alright.

https://github.com/Whonix/qubes-whonix/blob/master/etc/qubes/post-install.d/30-whonix-ws.sh

Looked easy and I like sorting the easy things asp.
Later some commentary can be added (low priority). If it looks alright, I'll upload and upgraded package.

[marmarek]

Should be qvm-features-request whonix-ws=1 (= not space)

[marmarek]

Should be qvm-features-request whonix-ws=1 (= not space)

[adrelanos]

Marek Marczykowski-Górecki:

Should be `qvm-features-request whonix-ws=1` (`=` not space)

Fixed.

[adrelanos]

Marek Marczykowski-Górecki:

Should be `qvm-features-request whonix-ws=1` (`=` not space)

Fixed.

[viq]

QubesOS/qubes-mgmt-salt-dom0-virtual-machines#10 is my attempt at it

[viq]

QubesOS/qubes-mgmt-salt-dom0-virtual-machines#10 is my attempt at it

[adrelanos]

fix, run "qvm-features-request whonix-ws=1" only on Whonix-Workstation:
Whonix/qubes-whonix@5d5aa4e

Should further qvm-features-request whonix-ws=1 be only run in TemplateVM? If so, I could add another test making sure it will only be run in TemplateVM. Or not required?

[adrelanos]

fix, run "qvm-features-request whonix-ws=1" only on Whonix-Workstation:
Whonix/qubes-whonix@5d5aa4e

Should further qvm-features-request whonix-ws=1 be only run in TemplateVM? If so, I could add another test making sure it will only be run in TemplateVM. Or not required?

[marmarek]

Should further qvm-features-request whonix-ws=1 be only run in TemplateVM?

Shouldn't matter. IMO better leave as is, let dom0 extension decide.

[marmarek]

Should further qvm-features-request whonix-ws=1 be only run in TemplateVM?

Shouldn't matter. IMO better leave as is, let dom0 extension decide.

[adrelanos]

QubesOS/qubes-mgmt-salt-dom0-virtual-machines#10 was merged.

Current status: Waiting for package upgrade.

[adrelanos]

QubesOS/qubes-mgmt-salt-dom0-virtual-machines#10 was merged.

Current status: Waiting for package upgrade.

[marmarek]

Package is already in current-testing (QubesOS/updates-status#543), I've uploaded it just after merging it.
The missing part is versioned template(s) in templates-community. Should I build a new one, or move the one from unstable?

[marmarek]

Package is already in current-testing (QubesOS/updates-status#543), I've uploaded it just after merging it.
The missing part is versioned template(s) in templates-community. Should I build a new one, or move the one from unstable?

[adrelanos]

Do you think you could provide instructions on how to test this? @viq

https://forums.whonix.org/t/qubes-whonix-14-saltstack-state-files-testers-wanted/5324

[adrelanos]

Do you think you could provide instructions on how to test this? @viq

https://forums.whonix.org/t/qubes-whonix-14-saltstack-state-files-testers-wanted/5324

[adrelanos]

Does https://forums.whonix.org/t/qubes-whonix-14-saltstack-state-files-testers-wanted/5324/2 suffice?

[adrelanos]

Does https://forums.whonix.org/t/qubes-whonix-14-saltstack-state-files-testers-wanted/5324/2 suffice?

[awokd]

Are these salt commands ready for testing? I updated a stock 4.0 setup to current-testing and tested above salt commands. No -14 templates installed to begin with which caused errors since I didn't have unstable enabled. Manually installed templates, but the commands are still failing with One or more requisite failed: qvm-whonix-ws-dvm.whonix-ws-14-dvm.

[awokd]

Are these salt commands ready for testing? I updated a stock 4.0 setup to current-testing and tested above salt commands. No -14 templates installed to begin with which caused errors since I didn't have unstable enabled. Manually installed templates, but the commands are still failing with One or more requisite failed: qvm-whonix-ws-dvm.whonix-ws-14-dvm.

[adrelanos]

Ready for testing, yes, see: https://forums.whonix.org/t/qubes-whonix-14-saltstack-state-files-testers-wanted

Do you have the same issue like me in #4082?

Anyhow. I suggest to close this initial implementation ticket and to create new bugs for any issues.

[adrelanos]

Ready for testing, yes, see: https://forums.whonix.org/t/qubes-whonix-14-saltstack-state-files-testers-wanted

Do you have the same issue like me in #4082?

Anyhow. I suggest to close this initial implementation ticket and to create new bugs for any issues.