Qubes 3.2.1 Build

[awokd]

Qubes OS version:

R3.2

Affected TemplateVMs:

fedora-23, debian-8

---

Steps to reproduce the behavior:

Install Qubes 3.2

Expected behavior:

Will have somewhat recent kernel and templates

Actual behavior:

Kernel and templates are dated

General notes:

Current kernel is now Linux 4.9.56-21.pvops.qubes.x86_64
Current templates Fedora 26 and Debian 9

---

Related issues:

https://groups.google.com/forum/?_escaped_fragment_=msg/qubes-users/fE2HCAdF-U0/eLovum3xAgAJ#!msg/qubes-users/fE2HCAdF-U0/eLovum3xAgAJ

[awokd]

Preliminary build steps/notes, with line numbers on the file edits:

Qubes 3.2 build 22Dec2017

dom0:

sudo qubes-dom0-update qubes-template-fedora-25

[from https://www.qubes-os.org/doc/building-archlinux-template/]

Create standalone appVM from fedora 25 template named dev25
Set private storage to 60000MB, more if additional templates beyond default list
Increase CPU and RAM, disable memory balancing as desired

dev25:

sudo dnf upgrade
sudo reboot
sudo dnf install git createrepo rpm-build make wget rpmdevtools dialog rpm-sign gnupg dpkg-dev debootstrap python2-sh

gpg --keyserver pgp.mit.edu --recv-keys 0xDDFA1A3E36879494
Verify its fingerprint, set as ‘trusted’. This is described here https://www.qubes-os.org/doc/VerifyingSignatures.
gpg --edit-key 0x36879494
fpr
trust
5
q
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
gpg --keyserver pgp.mit.edu --recv-keys 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA

git clone git://github.com/QubesOS/qubes-builder.git qubes-builder
mkdir qubes-builder/keyrings
mkdir qubes-builder/keyrings/git
cp .gnupg/pubring.gpg qubes-builder/keyrings/git/
cp .gnupg/trustdb.gpg qubes-builder/keyrings/git/
cd qubes-builder
git tag -v git describe

gedit example-configs/qubes-os-r3.2.conf
13: DISTS_VM = fc26 stretch

./setup
Select 3.2
Stable
No for a full build
Select builder-fedora, builder-debian, template-whonix, mgmt-salt
Y to download
Select fc26, stretch, whonix-gateway, whonix-workstation

make install-deps
make get-sources

gedit qubes-src/installer-qubes-os/conf/comps-qubes.xml
1164: qubes-template-fedora-26
1174: debian-9
1175: Debian 9 (stretch) template
1179: qubes-template-debian-9
1211: debian-9
gedit qubes-src/template-whonix/builder.conf
22: WHONIX_TBB_VERSION ?= 7.0.11
gedit qubes-src/installer-qubes-os/qubes-anaconda-addon/firstboot-qubes-text
107: qubes-prefs --set default-template 'fedora-26'
gedit qubes-src/installer-qubes-os/qubes-anaconda-addon/org_qubes_os_initial_setup/gui/spokes/qubes_os.py
230: self.default_template = 'fedora-26'

make qubes
make iso

[awokd]

Preliminary build steps/notes, with line numbers on the file edits:

Qubes 3.2 build 22Dec2017

dom0:

sudo qubes-dom0-update qubes-template-fedora-25

[from https://www.qubes-os.org/doc/building-archlinux-template/]

Create standalone appVM from fedora 25 template named dev25
Set private storage to 60000MB, more if additional templates beyond default list
Increase CPU and RAM, disable memory balancing as desired

dev25:

sudo dnf upgrade
sudo reboot
sudo dnf install git createrepo rpm-build make wget rpmdevtools dialog rpm-sign gnupg dpkg-dev debootstrap python2-sh

gpg --keyserver pgp.mit.edu --recv-keys 0xDDFA1A3E36879494
Verify its fingerprint, set as ‘trusted’. This is described here https://www.qubes-os.org/doc/VerifyingSignatures.
gpg --edit-key 0x36879494
fpr
trust
5
q
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
gpg --keyserver pgp.mit.edu --recv-keys 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA

git clone git://github.com/QubesOS/qubes-builder.git qubes-builder
mkdir qubes-builder/keyrings
mkdir qubes-builder/keyrings/git
cp .gnupg/pubring.gpg qubes-builder/keyrings/git/
cp .gnupg/trustdb.gpg qubes-builder/keyrings/git/
cd qubes-builder
git tag -v git describe

gedit example-configs/qubes-os-r3.2.conf
13: DISTS_VM = fc26 stretch

./setup
Select 3.2
Stable
No for a full build
Select builder-fedora, builder-debian, template-whonix, mgmt-salt
Y to download
Select fc26, stretch, whonix-gateway, whonix-workstation

make install-deps
make get-sources

gedit qubes-src/installer-qubes-os/conf/comps-qubes.xml
1164: qubes-template-fedora-26
1174: debian-9
1175: Debian 9 (stretch) template
1179: qubes-template-debian-9
1211: debian-9
gedit qubes-src/template-whonix/builder.conf
22: WHONIX_TBB_VERSION ?= 7.0.11
gedit qubes-src/installer-qubes-os/qubes-anaconda-addon/firstboot-qubes-text
107: qubes-prefs --set default-template 'fedora-26'
gedit qubes-src/installer-qubes-os/qubes-anaconda-addon/org_qubes_os_initial_setup/gui/spokes/qubes_os.py
230: self.default_template = 'fedora-26'

make qubes
make iso

[awokd]

MBR/AMD Test notes:

• Installer warning of pre-release/testing version
• MBR corrupted, fixed with https://groups.google.com/forum/?_escaped_fragment_=msg/qubes-users/TS1zfKZ7q8w/JQFkVF4xBgAJ#!msg/qubes-users/TS1zfKZ7q8w/JQFkVF4xBgAJ
• Stretch template MIA, need to fix group name in qubes-src/installer-qubes-os/conf/comps-qubes.xml and qubes-src/installer-qubes-os/conf/qubes-kickstart.cfg to "debian".
• Hypervisor command line on post-installed system is "placeholder", causing dom0 to take up most RAM
• No splash screen on boot
• All qubes based on Fedora 26 only have Thunderbird shortcut
• Passthrough appears to work
• Mapping USB block devices works
• Network and sys-whonix works
• Fedora DNF
  salt requires dnf-utils, but that conflicts with the installed yum-utils
  repaired with "dnf --allow-erasing --best upgrade"
  I should be able to find this in the Fedora 26 list of required packages

[awokd]

MBR/AMD Test notes:

• Installer warning of pre-release/testing version
• MBR corrupted, fixed with https://groups.google.com/forum/?_escaped_fragment_=msg/qubes-users/TS1zfKZ7q8w/JQFkVF4xBgAJ#!msg/qubes-users/TS1zfKZ7q8w/JQFkVF4xBgAJ
• Stretch template MIA, need to fix group name in qubes-src/installer-qubes-os/conf/comps-qubes.xml and qubes-src/installer-qubes-os/conf/qubes-kickstart.cfg to "debian".
• Hypervisor command line on post-installed system is "placeholder", causing dom0 to take up most RAM
• No splash screen on boot
• All qubes based on Fedora 26 only have Thunderbird shortcut
• Passthrough appears to work
• Mapping USB block devices works
• Network and sys-whonix works
• Fedora DNF
  salt requires dnf-utils, but that conflicts with the installed yum-utils
  repaired with "dnf --allow-erasing --best upgrade"
  I should be able to find this in the Fedora 26 list of required packages

[andrewdavidwong]

This is a tracking issue for tasks related to Qubes 3.2.1. It was prompted by this thread:
https://groups.google.com/d/topic/qubes-users/KYl6FWhP_IQ/discussion

[andrewdavidwong]

This is a tracking issue for tasks related to Qubes 3.2.1. It was prompted by this thread:
https://groups.google.com/d/topic/qubes-users/KYl6FWhP_IQ/discussion

[awokd]

UEFI/Intel specific test notes

• original 3.2 required use of Refind to install on this machine, this build does not
• still needed mapbs and noexitboot lines

[awokd]

UEFI/Intel specific test notes

• original 3.2 required use of Refind to install on this machine, this build does not
• still needed mapbs and noexitboot lines

[awokd]

Resolved:

• missing Stretch template
• AMD/MBR "placeholder" hypervisor command line due to coreboot and #2553. Would cause no splash screen too. IMHO not a release blocker, but should test as is MBR install on non-coreboot.
• fepitre addressed DNF conflicts in Fedora template

Pending:

• test MBR install on non-coreboot
• Should QubesOS/qubes-installer-qubes-os@62cb1ca or any others from QubesOS/updates-status#117 be backported to 3.2.1?
• Stretch template updater broken. Might still be #2615. I remember some mailing list discussion too; will research.
• Fixed one bug but default shortcuts still missing, researching.
• I need to write more descriptive commit messages.

[awokd]

Resolved:

• missing Stretch template
• AMD/MBR "placeholder" hypervisor command line due to coreboot and #2553. Would cause no splash screen too. IMHO not a release blocker, but should test as is MBR install on non-coreboot.
• fepitre addressed DNF conflicts in Fedora template

Pending:

• test MBR install on non-coreboot
• Should QubesOS/qubes-installer-qubes-os@62cb1ca or any others from QubesOS/updates-status#117 be backported to 3.2.1?
• Stretch template updater broken. Might still be #2615. I remember some mailing list discussion too; will research.
• Fixed one bug but default shortcuts still missing, researching.
• I need to write more descriptive commit messages.

[marmarek]

Stretch template updater broken. Might still be #2615. I remember some mailing list discussion too; will research.

Maybe this helps?
QubesOS/updates-status#323

Should QubesOS/qubes-installer-qubes-os@62cb1ca or any others from QubesOS/updates-status#117 be backported to 3.2.1?

Most commits are specific to r4.0 or fc25 in dom0. But probably some applies (including the one you've pointed). I can't test it right now, but can prepare some pull request with selected commits.

[marmarek]

Stretch template updater broken. Might still be #2615. I remember some mailing list discussion too; will research.

Maybe this helps?
QubesOS/updates-status#323

Should QubesOS/qubes-installer-qubes-os@62cb1ca or any others from QubesOS/updates-status#117 be backported to 3.2.1?

Most commits are specific to r4.0 or fc25 in dom0. But probably some applies (including the one you've pointed). I can't test it right now, but can prepare some pull request with selected commits.

[awokd]

Stretch template issue seemed to have been caused by #2909 / #3229. Deleting /usr/lib/xorg/Xorg.wrap per the latter resolved it. dbus-x11 was already installed and I left /etc/X11/Xwrapper.config sitting there with allowed_users=console. May already be fixed somewhere in testing but I can't find code mentioning "Xorg.wrap".

[awokd]

Stretch template issue seemed to have been caused by #2909 / #3229. Deleting /usr/lib/xorg/Xorg.wrap per the latter resolved it. dbus-x11 was already installed and I left /etc/X11/Xwrapper.config sitting there with allowed_users=console. May already be fixed somewhere in testing but I can't find code mentioning "Xorg.wrap".

[marmarek]

See gui-agent-linux changes I've pushed today, should address Xorg.wrap issue.

[marmarek]

See gui-agent-linux changes I've pushed today, should address Xorg.wrap issue.

[awokd]

On this last build, I got the following. I skipped "make builder" and continued with the remaining components and they seemed to work so hopefully it wasn't too critical. Not sure what I did differently this time.

$ make help
[...]
COMPONENT can be one of:
vmm-xen core-libvirt core-vchan-xen core-qubesdb linux-utils core-admin core-admin-linux core-agent-linux linux-kernel artwork gui-common gui-daemon gui-agent-linux gui-agent-xen-hvm-stubdom app-linux-split-gpg app-linux-tor app-thunderbird app-linux-pdf-converter app-linux-img-converter app-linux-input-proxy app-linux-usb-proxy app-yubikey mgmt-salt mgmt-salt-base mgmt-salt-base-topd mgmt-salt-base-config mgmt-salt-base-overrides mgmt-salt-dom0-qvm mgmt-salt-dom0-virtual-machines mgmt-salt-dom0-update meta-packages linux-template-builder desktop-linux-kde desktop-linux-xfce4 desktop-linux-i3 desktop-linux-awesome manager linux-pvgrub2 installer-qubes-os linux-yum linux-deb vmm-xen-windows-pvdrivers antievilmaid builder builder-fedora builder-debian template-whonix

You can also specify COMPONENTS="c1 c2 c3 ..." on command line
to operate on subset of components. Example: make COMPONENTS="gui" get-sources
$ make builder
make: *** No rule to make target 'builder'. Stop.

[awokd]

On this last build, I got the following. I skipped "make builder" and continued with the remaining components and they seemed to work so hopefully it wasn't too critical. Not sure what I did differently this time.

$ make help
[...]
COMPONENT can be one of:
vmm-xen core-libvirt core-vchan-xen core-qubesdb linux-utils core-admin core-admin-linux core-agent-linux linux-kernel artwork gui-common gui-daemon gui-agent-linux gui-agent-xen-hvm-stubdom app-linux-split-gpg app-linux-tor app-thunderbird app-linux-pdf-converter app-linux-img-converter app-linux-input-proxy app-linux-usb-proxy app-yubikey mgmt-salt mgmt-salt-base mgmt-salt-base-topd mgmt-salt-base-config mgmt-salt-base-overrides mgmt-salt-dom0-qvm mgmt-salt-dom0-virtual-machines mgmt-salt-dom0-update meta-packages linux-template-builder desktop-linux-kde desktop-linux-xfce4 desktop-linux-i3 desktop-linux-awesome manager linux-pvgrub2 installer-qubes-os linux-yum linux-deb vmm-xen-windows-pvdrivers antievilmaid builder builder-fedora builder-debian template-whonix

You can also specify COMPONENTS="c1 c2 c3 ..." on command line
to operate on subset of components. Example: make COMPONENTS="gui" get-sources
$ make builder
make: *** No rule to make target 'builder'. Stop.

[awokd]

Tested with full build and install on MBR.
Resolved:

• I applied the cherry-picked .diff directly to source code and coreboot install now works properly, with correct Xen command line construction
• fixed missing default application shortcuts, will PR my hack fix shortly
• starting applications in Debian Stretch template works now without messing with Xorg.wrap etc.
• going to ignore that "make builder" glitch for now

Pending:

• error on default template deployment "qubes-prefs --force-root default-template fedora-26 failed, no such option --force-root". Probably one of the backported ones, I will isolate.
• now that default application shortcuts are working properly, I need to update outdated ones in Stretch template
• will test again on MBR and UEFI once I figure out the --force-root item

[awokd]

Tested with full build and install on MBR.
Resolved:

• I applied the cherry-picked .diff directly to source code and coreboot install now works properly, with correct Xen command line construction
• fixed missing default application shortcuts, will PR my hack fix shortly
• starting applications in Debian Stretch template works now without messing with Xorg.wrap etc.
• going to ignore that "make builder" glitch for now

Pending:

• error on default template deployment "qubes-prefs --force-root default-template fedora-26 failed, no such option --force-root". Probably one of the backported ones, I will isolate.
• now that default application shortcuts are working properly, I need to update outdated ones in Stretch template
• will test again on MBR and UEFI once I figure out the --force-root item

[marmarek]

error on default template deployment "qubes-prefs --force-root default-template fedora-26 failed, no such option --force-root". Probably one of the backported ones, I will isolate.

Yes, this one

[marmarek]

error on default template deployment "qubes-prefs --force-root default-template fedora-26 failed, no such option --force-root". Probably one of the backported ones, I will isolate.

Yes, this one

[awokd]

Yes, that was it. I rolled it back by hand because there was another .diff to that same file. Finally finished rebuilding the templates and installer and iso and:

[awokd]

Yes, that was it. I rolled it back by hand because there was another .diff to that same file. Finally finished rebuilding the templates and installer and iso and:

[awokd]

MBR/AMD Test Results
Resolved:

• Default template deployment works correctly with zero errors
• USB Qube not marked as experimental, created DispVM by default
• Stretch applist whitelists working properly (will PR soon with my updated template whitelists)

Pending:

• test UEFI/Intel install
• cosmetic issue: on the very first boot after template deployment, the network indicator doesn't display. A reboot restores normal function, and it continues working after that for repeated reboots. Not sure if it's worth looking into.

[awokd]

MBR/AMD Test Results
Resolved:

• Default template deployment works correctly with zero errors
• USB Qube not marked as experimental, created DispVM by default
• Stretch applist whitelists working properly (will PR soon with my updated template whitelists)

Pending:

• test UEFI/Intel install
• cosmetic issue: on the very first boot after template deployment, the network indicator doesn't display. A reboot restores normal function, and it continues working after that for repeated reboots. Not sure if it's worth looking into.

[awokd]

UEFI/Intel Test Results
Resolved:

• Same as MBR/AMD
• Performed a rescue mode edit of /boot/efi/EFI/qubes/xen.cfg. Would like to claim this was planned but it was because I forgot to add mapbs=1 and noexitboot=1 at the end of install!

[awokd]

UEFI/Intel Test Results
Resolved:

• Same as MBR/AMD
• Performed a rescue mode edit of /boot/efi/EFI/qubes/xen.cfg. Would like to claim this was planned but it was because I forgot to add mapbs=1 and noexitboot=1 at the end of install!

[awokd]

Build Reproduction

• Follow ./setup steps above
• Apply PRs
  QubesOS/qubes-builder#42
  QubesOS/qubes-installer-qubes-os#18
  QubesOS/qubes-linux-template-builder#6
  QubesOS/qubes-builder-debian#13
• Apply QubesOS/qubes-installer-qubes-os#19 minus QubesOS/qubes-installer-qubes-os@1967168
• Fix Fedora 26 DNF conflict with QubesOS/qubes-builder-rpm#15

Caveats:

• network indicator missing on very first boot, full reboot or restarting sys-net restores it for good
• not sure if "No rule to make target 'builder'" was anything
• No package or file manager in default Stretch template
• Fedora 26 template testing was performed by editing builder-fedora/template_scripts/packages_fc26.list and replacing yum-utils with dnf-utils, not the above PR

[awokd]

Build Reproduction

• Follow ./setup steps above
• Apply PRs
  QubesOS/qubes-builder#42
  QubesOS/qubes-installer-qubes-os#18
  QubesOS/qubes-linux-template-builder#6
  QubesOS/qubes-builder-debian#13
• Apply QubesOS/qubes-installer-qubes-os#19 minus QubesOS/qubes-installer-qubes-os@1967168
• Fix Fedora 26 DNF conflict with QubesOS/qubes-builder-rpm#15

Caveats:

• network indicator missing on very first boot, full reboot or restarting sys-net restores it for good
• not sure if "No rule to make target 'builder'" was anything
• No package or file manager in default Stretch template
• Fedora 26 template testing was performed by editing builder-fedora/template_scripts/packages_fc26.list and replacing yum-utils with dnf-utils, not the above PR

[awokd]

Looks like QubesOS/qubes-desktop-linux-xfce4@ae0ddcb#diff-e2c130c91a59d0c63daf7efc7a753d81 fixes the network indicator. Will attempt to backport to 3.2.1 and test.

[awokd]

Looks like QubesOS/qubes-desktop-linux-xfce4@ae0ddcb#diff-e2c130c91a59d0c63daf7efc7a753d81 fixes the network indicator. Will attempt to backport to 3.2.1 and test.

[awokd]

Tests OK, PRing. That resolves the network indicator caveat. That is all I can think of besides branding and signing, etc. Please let me know if there's anything else.

[awokd]

Tests OK, PRing. That resolves the network indicator caveat. That is all I can think of besides branding and signing, etc. Please let me know if there's anything else.

[marmarek]

Thank you very much!
I'll review and merge it in the meantime, but more time I'll probably have only after releasing 4.0-rc4. But maybe I'll manage to build 3.2.1 while waiting for some test build of 4.0 ;)

[marmarek]

Thank you very much!
I'll review and merge it in the meantime, but more time I'll probably have only after releasing 4.0-rc4. But maybe I'll manage to build 3.2.1 while waiting for some test build of 4.0 ;)

[awokd]

No trouble, hopefully this helped towards getting 4.0 out a little bit faster or for you all to get some extra time off for the holidays!

[awokd]

No trouble, hopefully this helped towards getting 4.0 out a little bit faster or for you all to get some extra time off for the holidays!

[awokd]

For documentation purposes, with the added scope of recent security mitigations will the next version of R3 released be R3.3?

[awokd]

For documentation purposes, with the added scope of recent security mitigations will the next version of R3 released be R3.3?

[andrewdavidwong]

For documentation purposes, with the added scope of recent security mitigations will the next version of R3 released be R3.3?

My understanding is that the next version will be R3.2.1 (not R3.3), since the final decision regarding the Meltdown mitigation was to use the less drastic XPTI solution, as explained in QSB #37.

@marmarek, please confirm.

[andrewdavidwong]

For documentation purposes, with the added scope of recent security mitigations will the next version of R3 released be R3.3?

My understanding is that the next version will be R3.2.1 (not R3.3), since the final decision regarding the Meltdown mitigation was to use the less drastic XPTI solution, as explained in QSB #37.

@marmarek, please confirm.

[andrewdavidwong]

Do we still expect Qubes 3.2.1 to be released soon? It's scheduled to reach EOL in ~10 months on 2019-03-28:

https://www.qubes-os.org/doc/supported-versions/#qubes-os

It might seem strange if it reaches EOL shortly after being released.

[andrewdavidwong]

Do we still expect Qubes 3.2.1 to be released soon? It's scheduled to reach EOL in ~10 months on 2019-03-28:

https://www.qubes-os.org/doc/supported-versions/#qubes-os

It might seem strange if it reaches EOL shortly after being released.

[marmarek]

There is already preliminary build in testing here: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=3.2&build=20180522&groupid=1
It includes also Fedora 28 template. Failures on UEFI tests seems to be limitation of test environment, but other failures are still to be resolved. For anyone interested - it's possible to download ISO image from there - select any test and go to assets tab.

@andrewdavidwong @rootkovska Should we follow the full release candidate cycle (which means at least 5 weeks), or use some shorter schedule?

[marmarek]

There is already preliminary build in testing here: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=3.2&build=20180522&groupid=1
It includes also Fedora 28 template. Failures on UEFI tests seems to be limitation of test environment, but other failures are still to be resolved. For anyone interested - it's possible to download ISO image from there - select any test and go to assets tab.

@andrewdavidwong @rootkovska Should we follow the full release candidate cycle (which means at least 5 weeks), or use some shorter schedule?

[andrewdavidwong]

Should we follow the full release candidate cycle (which means at least 5 weeks), or use some shorter schedule?

I don't have an informed opinion about this, other than what I previously noted (that it might seem strange if it reaches EOL shortly after being released).

[andrewdavidwong]

Should we follow the full release candidate cycle (which means at least 5 weeks), or use some shorter schedule?

I don't have an informed opinion about this, other than what I previously noted (that it might seem strange if it reaches EOL shortly after being released).