New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debian in dom0 #1919

Open
mfc opened this Issue Apr 18, 2016 · 51 comments

Comments

Projects
None yet
@mfc
Copy link
Member

mfc commented Apr 18, 2016

We have discussed this numerous times but don't have an issue to track these discussions. It would be worth understanding what would be needed to change dom0 from Fedora to Debian (say Debian 8). Benefits include:

  • increased hardware compatibility
  • incorporate serious work taken towards reproducible builds
  • better firstboot installer
  • better (slower) release cycle than Fedora with longer-term support
  • other things?

This ticket does not encompass modifying the desktop environment.

@andrewdavidwong

This comment has been minimized.

Copy link
Member

andrewdavidwong commented Apr 19, 2016

The other issue about changing/upgrading dom0's OS: #1807

@TNTBOMBOM

This comment has been minimized.

Copy link

TNTBOMBOM commented Apr 19, 2016

the problem is , why anyone would take fedora which is very known for its issues (from about 2000) and their enterprise lockin on the code (redhat sucking stuff...) ?

is there anyone can give me good reasons why taking fedora at the first place ? why not debian ? why not even gentoo (which is regarding of code easiness/freedomcy better than fedora)?

im afraid the one who took the first (lets call it "bad") decision is still continuing to take these decisions which will make qubes a disaster even if u shifted to debian or X distro.

so i think consultations on these decisions is very important. and fedora to be the core of qubes or any other linux distro is a mistaken step.

@woju

This comment has been minimized.

Copy link
Member

woju commented Apr 19, 2016

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 10, 2016

What exactly is the issue with GPU passthrough and Xen?

@marmarek

This comment has been minimized.

Copy link
Member

marmarek commented May 10, 2016

Take a look at this thread - it's a good summary of GPU passthrough and PCI passthrough to HVM at all: https://groups.google.com/d/topic/qubes-devel/mvZBIUuyjv0/discussion

@minad

This comment has been minimized.

Copy link

minad commented May 13, 2016

It would be really nice to have a choice here. Like @woju I think it would be best to have a minimal distribution. However it might be nice to try something more declarative like NixOS.

I guess you wouldn't want to maintain different distributions, at least not for the official distribution? Do you have an idea how many Fedora specific configuations are in the qubes builder and the applications?

@marmarek

This comment has been minimized.

Copy link
Member

marmarek commented May 13, 2016

I guess you wouldn't want to maintain different distributions, at least not for the official distribution?

Exactly.

Do you have an idea how many Fedora specific configuations are in the qubes builder and the applications?

During the current Fedora 20 -> 23 upgrade I'm trying to collect that list. Actually two lists:

  • customization of GUI domain (which currently is the same as dom0) - this list is almost completed based on #1784
  • things customized in dom0 in general

I think the only really Fedora specific thing is update mechanism, other parts are just some system configuration which may need some adjustment when porting to other distribution (like config file locations etc).

Yes, we're considering NixOS here, but no decision has been made yet.

@unman

This comment has been minimized.

Copy link
Member

unman commented May 13, 2016

I think the only really Fedora specific thing is update mechanism, other parts are just some system configuration which may need some adjustment when porting to other distribution (like config file locations etc).

I've got some experience here as I've been running Debian in dom0 for a while. It's an unholy mix of packages, alien and hand crafted stuff,(hand crafted like the Borja Jesus), and not full Qubes. But it works well enough for my slight requirements. I'm hoping that the upgrade to 3.1 will be somewhat closer to full Qubes.
So far I would say there is almost nothing Fedora specific that comes outwith the normal repackaging process.

It's interesting that @mfc suggests that Debian provides increased hardware compatibility, while @woju says Fedora has best compatibility with graphic cards. Is there any evidence either way?

@minad

This comment has been minimized.

Copy link

minad commented May 13, 2016

The Borja jesus 😂 That still sounds somehow encouraging! What are your reasons for prefering Debian on the dom0? I just started using qubes and I had much more experience with Debian, Arch, ... but basically none with Fedora. It took a short while to get accustomed - but in the end it feels all the same with some minor package manager differences. Since qubes takes a more radical approach in general, I think it would also make sense to consider a change to something conceptually "better" or more advanced like nix.

@h01ger

This comment has been minimized.

Copy link

h01ger commented May 18, 2016

unman, could you please share more details about your setup, maybe even commits? ;-) Are you running this on a jessie based dom0 or stretch? How did you build+install the qubes specific stuff?

I'm still pondering doing the same, and looking at the cubes git repos I've identified these as being needed for dom0:

  • qubes-core-admin
  • qubes-core-admin-linux
  • qubes-core-agent-linux
  • qubes-core-libvirt
  • qubes-core-vchan-xen

Is that correct? Which are missing? Cause once dom0 is working one should be able to deploy VMs from a backup… ;-)

@marmarek

This comment has been minimized.

Copy link
Member

marmarek commented May 18, 2016

On Wed, May 18, 2016 at 09:03:43AM -0700, Holger Levsen wrote:

I'm still pondering doing the same, and looking at the cubes git repos I've identified these as being needed for dom0:

  • qubes-core-admin
  • qubes-core-admin-linux
  • qubes-core-agent-linux
  • qubes-core-libvirt
  • qubes-core-vchan-xen

Is that correct? Which are missing?

In addition to above:

  • qubes-vmm-xen (yes, you need this, not native Debian packages because
    of Qubes-specific patches, and parts in Debian packages - libxenvchan,
    stubdomains)
  • qubes-gui-common (required to build gui-daemon)
  • qubes-gui-daemon
  • qubes-core-qubesdb
  • qubes-manager

And mgmt-salt stuff (everything with "base" and "dom0").

Some of them already have debian packaging.

Packaging Xen for dom0 would be tricky because of stubdomains. It
consists of some external libraries/tools built as part of Xen build and
statically linked. Something that AFAIR is against Debian policy (for
example there are multiple source tarballs, from different projects).

And template packages needs to be converted to deb. Or not packaged at
all and installed some other way - which I think would be even better
option (installing templates as packages have a lot of side effects...).

Cause once dom0 is working one should be able to deploy VMs from a backup… ;-)

Yes, it should be possible.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

@h01ger

This comment has been minimized.

Copy link

h01ger commented May 18, 2016

Marek, thanks for the hints!

For a start I don't mind violating Debian policy if needed. That said, while statically linking and embedded code copies are discouraged in general, there are also exceptions to these rules.

But really, at first I just want to be able to use a Debian based system as dom0.

@unman

This comment has been minimized.

Copy link
Member

unman commented May 19, 2016

@h01ger
Worked through the build logs from fedora and the spec files. I followed the same build order and worked through each package in turn. I started with a base jessie install, and built up from there.
I tried cherry picking Xen packages from sid and applying the Qubes patches, and that worked reasonably but I found difficult to incorporate stubdomains, and got pretty lost. In the end I basically built everything from source. It was relatively easy to identify Debian equivalents for the fedora requirements, and referencing the native Debian packages also helped, particularly for Xen.
I had some success using alien for simpler stuff, although some paths and scripts needed to be bodged by hand.
I didn't build manager or salt stack, but I cant imagine they would be difficult.
Some stuff just doesn't work at all - can't boot from iso in HVM, for example. And nothing is really packaged so it's probably fragile on updates.

I'm currently working on 3.1, and trying to do the right thing, packaging properly and (sometimes) using qubes-builder framework, but I have limited time before my next trip. When I have anything sensible to share I will certainly commit, but I'm a way off that at present.

When I started there was a clear thought that Qubes would be moving to Debian, and I wanted to see what would be involved. Now, of course, that's no longer the case, so I suspect time is better spent working on Debian issues on the vm side.

andrewdavidwong added a commit that referenced this issue Jun 7, 2016

andrewdavidwong added a commit that referenced this issue Jun 12, 2016

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented Jun 14, 2016

I thought one of the main reasons to switch from Fedora to Debian are reproducible builds. Debian made and make a lot of progress in this area.

The other question which was discussed before is moving from KDE to GTK (Gnome) for the desktop.

@woju proposed above a switch to Alpine Linux. I understand that Alpine Linux has nice security features, but it seems to be a radical switch. A switch to Alpine would be less radical if the management GUI and dom0 would be separated. In the case the separation between GUI and dom0 would be possible and implemented, dom0 would be Alpine, but what about the GUI domain?

Why not going step by step. First moving from Fedora to Debian, because of the ability to create reproducible builds. Then moving from KDE to GTK (Gnome). The final step would be to separate Dom0 into separate domains. The management domain would be Alpine whereas the graphical engagement domain would be still Debian.

Another good point of @woju is the problem with drivers, older kernel and drivers in Debian. This is true for Debian stable, but Debian has a testing, unstable and experimental repository which contain more up to date packages.

Debian and Linux in general has many distribution derivatives. Why not using one of the derivatives which fulfills the requirements:

  • based on Debian
  • static release model
  • up-to-date driver

Tanglu is a distribution which fulfills all the requirements. Tanglu is based on Debian, but uses software packages from testing, unstable and sometimes experimental to provide the newest drivers and other software packages, but is still based on Debian stable. Tanglu has three main editions KDE, GNOME and core. In addition Tanglu uses Calamares the independent system installer.

andrewdavidwong added a commit that referenced this issue Jun 18, 2016

@rootkovska rootkovska removed the C: label Jun 30, 2016

@adrelanos

This comment has been minimized.

Copy link
Member

adrelanos commented Sep 14, 2016

@h01ger

This comment has been minimized.

Copy link

h01ger commented Sep 15, 2016

https://wiki.debian.org/Qubes/Devel is the relevant URL for this issue.

https://wiki.debian.org/Qubes just describe how to use Qubes from the point of view of someone familar with Debian - and that wiki page is also a bit outdated.

andrewdavidwong added a commit that referenced this issue Nov 6, 2016

@adrelanos

This comment has been minimized.

Copy link
Member

adrelanos commented Nov 25, 2016

@unman in #1781 (comment)

I don't think that #1919 is still a target.

How did you come to that conclusion?

In any case it seems to me that most users should be kept well away from dom0 and so what is running there is irrelevant. If anyone has the nous to tinker in dom0 then they should be able to handle the differences.

I don't think Fedora can be as transparent as that. Once one wants to do certain things, one example is #1375 and there are others, one is back to learning about Fedora. Same when issues happen or when debug info is requested. And that probably is not going to go away. Also higher effort for developers to know two systems. For example makes getting grsecurity in Qubes harder since an rpm package / repository is required.

@v6ak

This comment has been minimized.

Copy link

v6ak commented May 25, 2017

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 25, 2017

Sorry, for using the term outdated. Unsupported is the right term. Fedoras release model is as following:
"Release X is supported until one month after the release of Release X+2." [1] as a result Fedora 24 will reach it's end of life (EOL) approx. one month after the Fedora 26 release. The current supported versions are Fedora 24 and 25. Therefor Fedora 23 which is currently used for Dom0 is unsupported by the Fedora team.
Unsupported does not mean Fedora is outdated. Fedora comes always a lot of new software in every release. The hardware support is still superior compared to other distribution.

Kalilinux is doing pretty well since they based their OS on the Debian "Testing" branch. Other distributions are successful doing the same. The Debian "Testing" branch is far more conservative compared to Arch Linux. Arch Linux has new software shortly after it is available. With other words Arch Linux follows a pretty aggressive rolling release model. Debian "Testing" on the other hand accepts packages only if they have been proven to work in the Debian "Unstable" branch [2]. As a result, I would consider Debian "Testing" as having conservative rolling release model. In addition, I think it fits into Qubes OS release model. Qubes OS receives minor upgrades and features even during the lifetime of a major release. For example, Qubes VM Manager updates or kernel updates during the lifetime of Qubes OS 3.2.

[1] https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle
[2] https://wiki.debian.org/DebianReleases

@v6ak

This comment has been minimized.

Copy link

v6ak commented May 25, 2017

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 25, 2017

@v6ak I do not get your point of Xfwm/Kwin. Please elaborate on that a little bit more.

@adrelanos

This comment has been minimized.

Copy link
Member

adrelanos commented May 25, 2017

Whonix was based on Debian testing. (now on stable) The short summary "testing was absolutely horrible". Long summary:

https://www.whonix.org/wiki/Dev/Operating_System#Why_is_Whonix_based_on_Debian_Stable.2C_not_Debian_Testing.3F

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 25, 2017

@adrelanos that make sense. Thanks. I thought Debian "testing" is more reliable.

@tasket

This comment has been minimized.

Copy link

tasket commented May 25, 2017

An Ubuntu-based distro (not Ubuntu itself) could be a better choice if long-term support in the form of patches is a priority, which I think it should be. There is also the benefit of Ubuntu's hardware testing program... they're the only distro I'm aware that has validated an extensive list of PC desktop systems and components and done the tweaking necessary to get them working smoothly.

Some of this effort trickles down to Debian, but the trend is unreliable. Debian does not really care if desktop users are stymied by a server-centric kludge or bad UI choice or naive defaults from upstream.

Trisquel Linux specifically tracks Ubuntu LTS releases. But if FOSS-only is too confining (and I'm not sure it is), it may be worthwhile to remove the few offending components from Ubuntu ourselves and put the distro under a Qubes moniker.

OTOH, Fedora is really a problem: It is the only major distro that does not sign its repo metadata as a full set, allowing attacks that selectively hold-back patches -- yes, this is a big deal! Yet Fedora-based distros like RHEL and CentOS do fully sign. The reason I surmise for the disconnect is Red Hat's desire to direct users to paid RHEL subscriptions as the "serious choice with security". Fedora also lacks considerably in package selection. Its a testbed distro, not for serious use.

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 25, 2017

@tasket Trisquel would be a nice choice. However, it is 100% free and libre software.

I like the idea of an Ubuntu-based distro. However, removing offending components from Ubuntu is not worth the effort. There are several other distros based on Ubuntu. My favorite among them would be Linux Mint. Another advantage would be that Linux Mint offers Cinnamon, Mate, Xfce and KDE flavors out of the box. Other desktop environments can be installed afterwards (i3, GNOME etc.). All Linux Mint 18.x editions are based on Ubuntu 16.04 LTS [1].

[1] http://blog.linuxmint.com/?p=3145

@tasket

This comment has been minimized.

Copy link

tasket commented May 25, 2017

@Jeeppler How many components are there to remove from Ubuntu: Amazon and the Ubuntu font and...?

Linux Mint switched to Debian years ago and had a history of negligence re: not passing along security patches. So I take it they switched back?

@jpouellet

This comment has been minimized.

Copy link
Contributor

jpouellet commented May 25, 2017

Mint got owned and served backdoored install ISOs not to long ago, and yet even after that they still don't make sigs available on their downloads page. I don't know anything about their internal processes or track record, but this as my first impression suggests there are likely serious security concerns about their development process and their goals as a project are not aligned with Qubes', especially for use in dom0.

Remember also that ideally we want to move towards a completely reproducibly built system. Debian has been making lots of progress on that front. Is this a priority for Ubuntu or derivatives as well?

@v6ak

This comment has been minimized.

Copy link

v6ak commented May 25, 2017

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 25, 2017

@tasket @marmarek once pointed out that the problem about Ubuntu is that the Ubuntu license prevents the Qubes team from redistributing a modified Ubuntu under the Ubuntu name. Even if you remove all the privacy concerning parts of Ubuntu you still end up with the trademark issues.

Linux Mint is based on Ubuntu. I think you got confused between Linux Mint and Linux Mind Debian Edition (LMDE). LMDE is based on Debian.

@jpouellet your security concerns are maybe valid. The data breach which happened to them is one thing, that can happen to anybody. Linux Mint is a community project, Qubes OS developers and users can always work together with them to improve their security standard.

Debian and GuixSD are the two distributions which made some serious progress towards reproducible builds. Ubuntu on the other hand is not even listed in the reproducible-builds.org [1] page.

[1] https://reproducible-builds.org/who/

@tasket

This comment has been minimized.

Copy link

tasket commented May 26, 2017

I don't think re-branding is a hurdle... more of a threshold. But I wasn't aware Ubuntu was ignoring reproducability. That, together with full repo signing and alternative architectures, makes Debian the more interesting choice.

@v6ak

This comment has been minimized.

Copy link

v6ak commented May 26, 2017

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented May 26, 2017

@v6ak qubuntu or qubesuntu ;-)

@msusi

This comment has been minimized.

Copy link

msusi commented Jul 4, 2017

Until Fedora remains the default dom0 for QubesOS it would be nice to deblob it using for example packages from freed-ora: https://www.fsfla.org/ikiwiki/selibre/linux-libre/freed-ora.en.html

@andrewdavidwong andrewdavidwong added the task label Apr 3, 2018

@ghost

This comment has been minimized.

Copy link

ghost commented Apr 5, 2018

Related: #833

Dom0 should definitely have the smallest possible TCB, so something like Alpine would make sense.

@tasket

This comment has been minimized.

Copy link

tasket commented Apr 5, 2018

I'm a bit skeptical about alternatives like Alpine. They are rarely discussed or deployed, and the fact that the Qubes community has not adopted it for domU use may indicate a lack of flexibility or utility in the distro. I sometimes find even Fedora's selection of CLI tools too limiting.

Further, it adds a third Linux flavor that contributors and admins must become familiar with. This saps attention and energy away from exploring true alternative operating systems.

@v6ak

This comment has been minimized.

Copy link

v6ak commented Apr 6, 2018

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented Apr 7, 2018

My biggest issue with Fedora as Dom0 is, that the version which is used is in Qubes OS is already unsupported by the Fedora community. Qubes 4.0 is a good example. Qubes 4.0 was released on the 29th March 2018. Qubes 4.0 ships with Fedora 25. The end of life for Fedora 25 was the 12th December 2017. I would prefer to have a long term distribution as basis for Qubes OS. Both Debian and CentOS would be better options, because of there long term support. However, the problem with both is the lack of up-to-date driver for new computer models. In case of Debian this could be avoided by using Debian 'testing'.

@v6ak

This comment has been minimized.

Copy link

v6ak commented Apr 7, 2018

@adrelanos

This comment has been minimized.

Copy link
Member

adrelanos commented Apr 7, 2018

@tasket

This comment has been minimized.

Copy link

tasket commented Apr 7, 2018

Seems like it would be easy for Debian stable instances to pull in drivers (especially) from testing. I've done this with various packages over the years with few issues.

@DemiMarie

This comment has been minimized.

Copy link

DemiMarie commented Apr 8, 2018

It seems that the problem is twofold:

  • Most LTS distros are aimed at enterprise systems and servers, which have a relatively limited range of hardware. Qubes, on the other hand, is aimed at end-users, so it needs to support pretty much any hardware that has the necessary virtualization support. That means a recent kernel and drivers, which the LTS distros avoid.
  • On the other hand, Qubes cannot tolerate frequently upgrading Dom0. Why is that? I know that most recent Linux distributions can do this non-destructively.

One way around this is that the Linux kernel has a fantastic backwards-compatibility story when it comes to its userspace APIs. Perhaps we could drop-in a newer kernel? No, that won’t work.

@Jeeppler

This comment has been minimized.

Copy link

Jeeppler commented Apr 9, 2018

It would not be a huge issue to upgrade Dom0 once in a while. The upgrades should not break anything. Rolling release would be to fast and LTS is to slow. The best would probably be a semi-rolling release model. I think, having every 2-3 months a kernel upgrade would be acceptable.

@DemiMarie

This comment has been minimized.

Copy link

DemiMarie commented Apr 10, 2018

@v6ak

This comment has been minimized.

Copy link

v6ak commented Apr 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment