Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port Qubes to ppc64 [2 bitcoin bounty] #4318

Open
Rspigler opened this issue Sep 18, 2018 · 34 comments

Comments

@Rspigler
Copy link

commented Sep 18, 2018

QubesOS is the most secure operating system available, by far. However, it unfortunately only runs on the x86 instruction set, which runs on unauditable and insecure firmware. The Power Architecture is a much more secure ISA. Products like the Talos II (edit: and now much more affordable Blackbird) with the Power9 CPU are fully open, with auditable schematics, firmware, and software - and being able to run QubesOS on such devices would be a huge win for the infosec community.

There are various ways to achieve this compatibility, so I thought that this issue could be a way to track them/discuss.

1 - Xen could have a ppc64 port (Raptor Computing Systems has offered free hardware to incentivize)
2 - Using the seL4 microkernel (#3894), which is already looking into supporting the Power Architecture
3 - Qubes' Hypervisor Abstraction Layer (HAL), which utlizes libvirt to support multiple hypervisors, yet currently only supports Xen, could be expanded to support KVM, to run on ppc64.

@madscientist159

This comment has been minimized.

Copy link

commented Sep 18, 2018

My vote would be seL4. Not only are they responsive to the threat posed by closed, highly privileged firmware, but the kernel is already being used in very high security installations.

@Rspigler

This comment has been minimized.

Copy link
Author

commented Sep 19, 2018

I would truly love seL4 as well and hope that is the end goal; however, I believe that that course of action would take the longest of all 3. I would argue that porting Qubes for ppc64 (by way of 1 or 3) is of higher priority than working on substituting the seL4 microkernel for Xen, since that offers the quickest route to Qubes running a truly open platform. The security benefit of seL4 could then come after.

I could be wrong though.

@madscientist159

This comment has been minimized.

Copy link

commented Sep 19, 2018

Of the two, I suspect 3 is the fastest and probably a decent stopgap solution. 1 would be duplicating effort that could go into the seL4 port IMO.

@justinlynn

This comment has been minimized.

Copy link

commented Sep 19, 2018

If needed, I can provide a significant quantity of build and test infrastructure for this effort.

@andrewdavidwong

This comment has been minimized.

Copy link
Member

commented Sep 19, 2018

There are various ways to achieve this compatibility, so I thought that this issue could be a way to track them/discuss.

The qubes-devel mailing list is the appropriate place for this sort of discussion. By contrast, qubes-issues is reserved for actionable issues as described in our issue reporting guidelines. If this issue is to remain open, it needs to be action-oriented rather than just a place for tracking and discussion. I'll change the title to reflect this.

@andrewdavidwong andrewdavidwong changed the title Issue Tracker for ppc64 Port Port Qubes to ppc64 Sep 19, 2018

@andrewdavidwong andrewdavidwong added this to the Far in the future milestone Sep 19, 2018

@awokd

This comment has been minimized.

Copy link

commented Sep 23, 2018

I humbly propose this thread: https://groups.google.com/d/msg/qubes-devel/IFLyyCUbLmQ/_HtgOJK4AQAJ as a good place to continue the discussion!

@Peter-Easton

This comment has been minimized.

Copy link

commented Sep 26, 2018

I have had some difficulty getting to Google Groups, but I'd like to put my vote in as well for a port to the Talos II, whichever the professionals decide it will take the form of. I'm not well-versed enough in hypervisor design to be able to make any meaningful comment on which one would be best, but on the PPC64el, Debian runs KVM quite well with Libvirt for running other Debian and CentOS guests and is happily usable as a desktop with a minimum of work. I'd be very happy to beta-test a port to the PPC64, if you guys need testers.

Not only that but the Talos Lite is now here, which is about the price of high-end business grade or one of those high-end name-branded gaming laptops, and offers far better upgradeability for the future than a gaming laptop even when we don't look at the owner control/openness aspect of it. For many people interested in and serious about security, the price equivalent of a good quality laptop isn't a high one to pay, especially if the user isn't a firmware hacker that has the skills and know how to create their own "trusted stick" like what Joanna Rutkowska was suggesting for a stateless, verified boot x86 laptop.

@tlaurion

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2018

The OTF funding form is here.

Time to tag developers, fund grant filling people (@mfc!) that might help so the discussion can follow on google groups and the information for the grant gets filled for the next OTF round!

@mfc

This comment has been minimized.

Copy link
Member

commented Sep 27, 2018

hi all, i don't have any capacity to help out right now, but realistically i don't think OTF would fund such effort.

their target group is non-tech-savvy users in at-risk environments. these users have the current computers they own, they don't have the money/luxury of buying new hardware. Qubes' existing hardware requirements are already a very high barrier for these users and their target audience.

so if anything they would be more interested in funding Qubes development onto phones than Qubes on ppc64.

not stopping you all from submitting a proposal to OTF of course. maybe this is more something for a crowdfunding approach, for example.

@Rspigler

This comment has been minimized.

Copy link
Author

commented Sep 30, 2018

Understandable. Going ahead with both might be possible. If a crowdfunding approach were to be done, what would be the estimated target goals for the KVM approach, and the seL4 approach? (seL4 could possibly be set as a stretch goal).

@marmarek

This comment has been minimized.

Copy link
Member

commented Sep 30, 2018

As for KVM:

Well, the first non-trivial task would be research what really needs to be done. There are two main parts here:

  1. add KVM support in general
  2. solve ppc64 specific problems (including verification of devices isolation etc)

The first one is much more predictable, I think it basically is:

  • implementing vchan using KVM-specific mechanism
  • implement "driver domains" (running a disk/net/usb backend in separate VM, instead of the host) - this is major feature missing in KVM compared to Xen; there are possible workarounds sacrificing some security
  • add support for KVM to GUI agent/daemon - one hypervisor specific part there is accessing actual window composition buffers, using shared memory mechanism provided by particular hypervisor
  • adjust startup scripts, libvirt configuration etc
  • test everything and fix bugs

I think we should assume 50-100kEUR goal for this part alone. Note it may be also beneficial for other architectures, not only ppc64.

The second part may take several months of work, but also may turn out to be trivial. Require someone knowing ppc64 architecture (which I don't know).

For seL4, I don't even try to provide any estimate, without doing some research on the current state of it and what features crucial for Qubes are missing. See FAQ for some hints.

@Rspigler

This comment has been minimized.

Copy link
Author

commented Oct 13, 2018

Thank you for your detailed response and all your work on Qubes.

What would your estimate be for approach #1? (ppc64 port of Xen?)

@marmarek

This comment has been minimized.

Copy link
Member

commented Oct 13, 2018

What would your estimate be for approach #1? (ppc64 port of Xen?)

Take a look at this xen-devel thread

@tlaurion

This comment has been minimized.

Copy link
Contributor

commented Feb 12, 2019

GSoC proposal, anyone?

@marmarek

This comment has been minimized.

Copy link
Member

commented Feb 12, 2019

In theory deadline for setting up GSoC idea page was Feb 6, but in practice period for students to choose projects haven't started yet. Feel free to open PR against gsoc.md. It's ok to have a proposal with very basic description, although this ticket already contain content that could be re-used.

@tlaurion

This comment has been minimized.

Copy link
Contributor

commented Feb 12, 2019

Added here.

@Rspigler

This comment has been minimized.

Copy link
Author

commented Feb 13, 2019

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I am placing 1 bitcoin as a bounty for completing this task in porting Qubes.

As multiple developers will most likely be involved, I will divide the bounty between based on work done as I see fit.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwsYOJ56G8Q1Wl3glNc4P5sIUGCMFAlxjj+AACgkQNc4P5sIU
GCNDWw//YeGQtVH2xc6+qNRezvcBG3M4Ql0DaPFndSPqQ3tBaHsaBoAH7BT6o3Ih
EU/pTCBWDAItC13yot4FejYp2eLRY4VtHPd3wHB+e5WVCZdKbW7Z1F2SWsPJutDo
NSKC+Rb8CEZO0sxSe9CYP7plc6a6GK+WF+ZjgiSN0ISeBf/VMr1V7aXJgZ+cIjSp
eaJCzQ0XSah0oO2brJXUTvPOHUKsFHv3vJLcZag+N5T3HZ5P6Zt6jYorCGRtRBzK
aWVaqGb8MVR9CQUOcLJ8c0ubyRHwwpTWMgr0QQ0QUzwRTBKMyt8jAfPVshI1l/T5
xzr067/sX1cUIWXGp5kpvK32aX9Gb+Sh7e+BqYour4hGiSpq2Vzz2UAakLutM2s4
hkFfnZA1HiZsqS0bxR9/iNi1+dvEBc2U4KUd6ou++p4osve+PpHqWXJGd5Acewgf
fcdhNDPSSif5ON56gZRzvEMhtbE4MXfIdEI8NQGihZfdyO+R+muKxDRDPq9HFK3s
hH1RcAYx6jC1T7E4UENZMzw2n9xKIcBnbkURmx6byzRhZACcX/I6YMciZks5Xxdb
yIItqpTqjW7IdabmDxEYUlG7g+9G8kzZMNFNiqbbxmvsd83O4OUuPOcYj9XAIRtw
w5QmHMVKbPMoKPN9s15i3KXGTnpIQ80E0nGWai8NykgzzzMOH3c=
=Tehn
-----END PGP SIGNATURE-----

@Rspigler Rspigler changed the title Port Qubes to ppc64 Port Qubes to ppc64 [1 bitcoin bounty] Feb 14, 2019

@shawnanastasio

This comment has been minimized.

Copy link

commented Feb 17, 2019

As for KVM:

Well, the first non-trivial task would be research what really needs to be done. There are two main parts here:

1. add KVM support in general

2. solve ppc64 specific problems (including verification of devices isolation etc)

The first one is much more predictable, I think it basically is:

* implementing vchan using KVM-specific mechanism

I have been working on a reimplementation of vchan using KVM with the ivshmem shared memory backend, available here (comments welcome!).

I originally started it to assist in porting individual Qubes projects (mainly qubes-gui-daemon), but it would be great if it was useful for a port of Qubes as a whole.

@marmarek

This comment has been minimized.

Copy link
Member

commented Feb 17, 2019

I have been working on a reimplementation of vchan using KVM with the ivshmem shared memory backend, available here (comments welcome!).

Here you are: shawnanastasio/libkvmchan#1

@Rspigler

This comment has been minimized.

Copy link
Author

commented Feb 18, 2019

In the Qubes Architecture spec (Version 0.3), in the rational for choosing Xen over KVM, the authors mention how KVM relies on the Linux kernel to provide isolation. They also argue that Xen is much easier to audit due to a smaller code base, and that the I/O emulator, networking code, and drivers can be moved out of Dom0, further minimizing TCB.

This document was written 2010. These concerns still present?

@marmarek

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

Mostly yes. In the meantime, some extra precautions were developed to isolate device emulator (I think the most relevant is seccomp filtering), so it is slightly better now.
But the arguments of a) Linux vs Xen code size, and b) backend drivers on the host system still stands.
The "b" point is especially bad, as it makes it hard to have host system isolated from the outside world (networking, usb etc).

On the other hand, a lot of development have happened on KVM since 2010, some of it very relevant for Qubes. For example much better support for GPU passthrough. And (the main point of this discussion) support for other architectures.

@marmarek

This comment has been minimized.

Copy link
Member

commented Feb 20, 2019

Very interesting development: KVM support for Xen guests, including PV drivers. Given that Xen PV drivers supports backends in another guest, there is a chance this mechanism will support that too. Especially those patches emulates grant tables and event channels, which are designed with non-host backend in mind.

@leo-lb

This comment has been minimized.

Copy link

commented Apr 11, 2019

I'll add up another Bitcoin to the bounty, split across people same as above..

@Rspigler Rspigler changed the title Port Qubes to ppc64 [1 bitcoin bounty] Port Qubes to ppc64 [2 bitcoin bounty] Apr 12, 2019

@leo-lb

This comment has been minimized.

Copy link

commented Jun 22, 2019

At current market rate, the bounty is worth more than $20,000 - I think that's good money.

@shawnanastasio

This comment has been minimized.

Copy link

commented Jul 4, 2019

A quick update. My vchan implementation for KVM is nearing completion. Once it's in a usable state, I plan on testing it with qrexec and other miscellaneous qubes utilities. If all goes well I'll look into getting qubes-gui working which will require a mechanism to map guest pages from the host which I'm still not sure how to (safely) do on KVM.

@0spinboson

This comment was marked as off-topic.

Copy link

commented Jul 5, 2019

it's y'all's money of course, but as someone who doesn't see ppc ever becoming mainstream, while the project is rather under-staffed (considering the number of open issues on the tracker doubled since late 2016), I do think it's a strange priority to not want to give this to the Qubes project in general, but only to someone willing & able to port Qubes to a yuppie platform.

@leo-lb

This comment was marked as off-topic.

Copy link

commented Jul 5, 2019

it's y'all's money of course, but as someone who doesn't see ppc ever becoming mainstream, while the project is rather under-staffed (considering the number of open issues on the tracker doubled since late 2016), I do think it's a strange priority to not want to give this to the Qubes project in general, but only to someone willing & able to port Qubes to a yuppie platform.

I am not interested in furthering Qubes OS support on x86. The architecture is doomed in regards to providing freedom to it's users. OpenPOWER is not a "yuppie" platform, I use it as my main workstation for day to day programming and administrative work. It would not benefit me at all to donate to the Qubes OS project if it's only goal is supporting x86 because I don't use x86 hardware anymore. So when Qubes OS is ported to ppc64[le] then I can explore donating for the rest of the project, otherwise, it's of no use to me..

A quick update. My vchan implementation for KVM is nearing completion. Once it's in a usable state, I plan on testing it with qrexec and other miscellaneous qubes utilities. If all goes well I'll look into getting qubes-gui working which will require a mechanism to map guest pages from the host which I'm still not sure how to (safely) do on KVM.

Awesome!

@0spinboson

This comment was marked as off-topic.

Copy link

commented Jul 5, 2019

to clarify: by 'yuppie' I mean 'a platform for the quite well-off (who tell themselves Must Have Total Platform Security Or I Might As Well Move In With The NSA)', as 1300$ (ex VAT, shipping) for a 4c CPU in 2019 just means you have money to burn. Not that you aren't allowed to tell yourself these things, of course, but the thing you're afraid of is the govt and intelligence complex (which is of course interwoven with the Tech companies), not the arch and its shortcomings.
Anyway, I've said my piece, so I'll leave you to it.

@leo-lb

This comment was marked as off-topic.

Copy link

commented Jul 5, 2019

to clarify: by 'yuppie' I mean 'a platform for the quite well-off (who tell themselves Must Have Total Platform Security Or I Might As Well Move In With The NSA)', as 1300$ (ex VAT, shipping) for a 4c CPU in 2019 just means you have money to burn. Not that you aren't allowed to tell yourself these things, of course, but the thing you're afraid of is the govt and intelligence complex (which is of course interwoven with the Tech companies), not the arch and its shortcomings.
Anyway, I've said my piece, so I'll leave you to it.

The CPU is cheaper than Intel in that regard, what costs money is the motherboard, and if I don't order RaptorCS offerings it will last even longer at that price, so that's meant to evolve of course. Personally I'm not afraid of surveillance or anything like that, I just think that by principle a machine you own means a machine you can hack. Not the case with Intel, AMD etc. You can't hack it.

@tasket

This comment was marked as off-topic.

Copy link

commented Jul 5, 2019

I don't agree with the simplistic "thing you're afraid of". Governments are claiming justification for their expanded reach partly on the great success criminal syndicates have had in expanding theirs. And the former will (these days) typically pay ransoms for expediency over ethics, thus feeding a vicious circle.

Price points are an issue, but there are examples about how open source products can gain traction in the marketplace and eventually reduce costs via economies of scale. What we should avoid is becoming obsessed with attaining price-parity vs WinTel (the platform that offers us cheap-and-nasty, or cheap-and-nasty-on-steroids).

@0spinboson

This comment was marked as off-topic.

Copy link

commented Jul 6, 2019

I don't agree with the simplistic "thing you're afraid of". Governments are claiming justification for their expanded reach partly on the great success criminal syndicates have had in expanding theirs. And the former will (these days) typically pay ransoms for expediency over ethics, thus feeding a vicious circle.

With respect, 'organized crime' has nothing to do with why the intelligence complex is asking Intel and AMD to add backdoors to the IME/PSP. The only reason they do, is because they want to be able to spy on citizens, corporations and foreign governments. They don't care about organized crime piggybacking (not least because they're not infrequently working together). The fact that lower govts are bothered by the same doesn't bother them either, "National security" trumps all.

@andrewdavidwong

This comment has been minimized.

Copy link
Member

commented Jul 6, 2019

Reminder: This is an issue tracker, not a forum for discussion. The discussion thread for this issue is here. Please keep comments here on-topic and actionable for any developers who might work on this issue. Please see our issue tracker guidelines for more information.

@Rspigler

This comment has been minimized.

Copy link
Author

commented Jul 29, 2019

In the discussion thread, @marmarek made clear that the Qubes team has no intention of porting to an open platform, and wants to keep focus on expanding the user base and hardware compatibility. However, he did state that if someone could provide enough funding to work on the port without sacrificing x86 support, Qubes would happily help with the port. I wonder if it would be possible to get an estimate on such a number?

I agree with @leo-lb - thanks to him and I, there are considerable funds available now available to developers willing to start working on this. @shawnanastasio has already started working on KVM support, although there is a long way to go. I would really like to start seeing more donations from the wider community. Spreading this on Twitter and Reddit I think would go along way. We probably need a website, with a secure donations page.

I am willing to match donations up to another 1 bitcoin. However, I am very concerned with what seems to be the consensus in choice of KVM over Xen. I understand that KVM has much better support for more architectures, but I have many security concerns. As I wrote before:

In the Qubes Architecture spec (Version 0.3), in the rational for choosing Xen over KVM, the authors mention how KVM relies on the Linux kernel to provide isolation [and therefore the isolation quality is limited to that of the Linux kernel]. They also argue that Xen is much easier to audit due to a smaller code base, and that the I/O emulator, networking code, and drivers can be moved out of Dom0, further minimizing TCB.

I would like this to be further discussed.

@marmarek

This comment has been minimized.

Copy link
Member

commented Jul 29, 2019

I wonder if it would be possible to get an estimate on such a number?

See #4318 (comment)

I would like this to be further discussed.

If host system would be sufficiently isolated (no external devices, no networking, possibly also no GUI), I would be less worried about KVM kernel part. Although it's still part of monolithic kernel there is very little direct interface to other kernel parts. We could have a separate kernel build with most features (not needed on host) disabled. It's still less ideal than Xen architecture, but acceptable. On the other hand, I/O emulator (aka qemu) is a bigger issue. There are multiple approaches to this issue:

  • sandbox it as much as possible (seccomp filter, non-root user, etc)
  • replace with something smaller - like crosvm - should be good enough for Linux qubes, not necessary other OSes
    Above points can (and should) be combined.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.