Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider support for Android VMs #2233

Open
andrewdavidwong opened this issue Aug 7, 2016 · 37 comments

Comments

Projects
None yet
@andrewdavidwong
Copy link
Member

commented Aug 7, 2016

Over the years, many Qubes users on the mailing lists have expressed the desire to be able to run some kind of Android VM (e.g., HVM) under Qubes OS.

@JohnnyCalavera

This comment has been minimized.

Copy link

commented Sep 18, 2016

Did someone try to create an android-x86 VM on qubes? I was planning on doing it myself in the near future. (by the way: the stable Marshmallow release is out now)

@grote

This comment has been minimized.

Copy link

commented Sep 18, 2016

I tried running RemixOS, but without success. There might be a general problem of running Android as a guest in a Xen hypervisor.

@Jeeppler

This comment has been minimized.

Copy link

commented Sep 20, 2016

@andrewdavidwong can you reference to those mailing list entries?

@andrewdavidwong

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2016

@Jeeppler: This is based on my memory of the mailing lists over the years, so I don't have an exhaustive list. However, here are some examples:

@Jeeppler

This comment has been minimized.

Copy link

commented Sep 23, 2016

According to the posts, people want to run Android x86 to do application development.

However, it seems the Android emulator uses the CPU virtualization extension. The problem could also be solved by using nested virtualization on Xen.

@andrewdavidwong

This comment has been minimized.

Copy link
Member Author

commented Nov 8, 2016

Cross-linking new discussion thread:

https://groups.google.com/d/topic/qubes-users/frK8xaBh9pI/discussion

@entr0py

This comment has been minimized.

Copy link

commented Nov 8, 2016

@grote I have not had any issues running RemixOS under Qubes. But it suffers from the same Android-x86 mouse issue mentioned in the thread that Andrew just linked.

Qubes is actually a convenient platform for running Android emulators because most of them don't allow static IP addresses. Qubes' mini-DHCP server works just fine.

@grote

This comment has been minimized.

Copy link

commented Nov 8, 2016

@3n7r0p1 which version did you use? Could you send me the link to the download?

Please also let me know if you find a solution for the mouse issue.

@entr0py

This comment has been minimized.

Copy link

commented Nov 9, 2016

I used an older version of RemixOS so I checked again with the latest release and it also installs and boots fine.

version: 3.0.206, 64-bit
filename: Remix_OS_for_PC_Android_M_64bit_B2016101201.zip
download: http://www.jide.com/remixos-for-pc#downloadNow
https://www.fosshub.com/Remix-OS.html

when booting iso, add INSTALL=1 to kernel parameters
create partition, gpt: no, efi: no
if boot hangs, add vga=ask to kernel parameters (may need debug mode on)
allow 10-15 mins for first boot, 10-15 mins for desktop after setup

Confirmed mouse behavior hasn't changed. Not all functionality can be replicated from keyboard. Unusable without mouse fix.

IMO, RemixOS is more resource-heavy than Android-x86 (especially graphics).
I've Googled the heck out of the mouse issue. Don't have the expertise to take it any further. :(

@mfc mfc referenced this issue Jan 31, 2017

Closed

create GSOC 2017 Ideas List #2607

2 of 2 tasks complete
@Wikinaut

This comment has been minimized.

Copy link

commented Jun 26, 2017

👍 yes, please

@micahflee

This comment has been minimized.

Copy link

commented Jul 17, 2017

I just tried android-x86 in an HVM, and it actually works great (the only change I had to do was increase the RAM -- I'm using 4GB, but it probably will work with less). It doesn't have the same mouse issues as RemixOS (which, as of today, is a discontinued project).

screenshot_2017-07-17_13-28-13

I downloaded an Android 4.4.4 iso from https://sourceforge.net/projects/android-x86/ -- but the more I look into it, it appears there are later versions of android-x86 available, including 7.1: http://www.android-x86.org/releases/releasenote-7-1-rc1

@micahflee

This comment has been minimized.

Copy link

commented Jul 17, 2017

It would be cool to make an Android template. I'm not quite sure how the partitioning could work so that android-x86 AppVMs could have private images though.

@micahflee

This comment has been minimized.

Copy link

commented Jul 17, 2017

Update: I tried installing android-x86 7.1 in an HVM, and it doesn't work nearly as well as 4.4.4. I could boot in live mode, but I had the same mouse issues that exist in RemixOS, and when I install to the hard disk and try booting, it never finishes.

@nm8800

This comment has been minimized.

Copy link

commented Jul 29, 2017

@micahflee How did you get an AndroidVM to work in Qubes? Currently trying to figure out how to use the Android-x86 vm to do the same.

@aseralis

This comment has been minimized.

Copy link

commented Feb 15, 2018

I push the topic gently upwards, an android HVM in qubes would be great! Unfortunately it's very painful to get Android x86 running, but I'm going to test a bit by myself and share results.

@kuzega

This comment has been minimized.

Copy link

commented Mar 15, 2018

So hows going with android vm ?

My testing results on qubes r4.0-current-testing are that none of android works so far.
Tested iso images : remixOS, android x86 latest cyanogenmod 7.1 release1 and normal 7.1r1, 4.4r5, phoenixOS.
None of them detect the hard drive.
So another test I installed centos7minimal then andro x86 latest cyanogenmod in rpm package so it added grub entry, it does not boot
Most of the tests showed familiar boot screen but did not run, VM killed itself.

But i remember it was working on qubes 3.2, cant remember what android did i run but probably it was remixOS, when i was testing this like a year ago or so. It was install-able but did not boot from installation, had to use iso and all the changes were gone. The performance was really choppy and i gave up, that probably was because of SLAT feature missing on previous laptop.

PS: was not editing grub entries at all, only tested different VESA modes when available

Hint: try to test preinstalled android VM images and just put them in /var/lib/qubes . Personally i couldnt get this method to work, first have to convert given preinstalled images to qcow RAW format

@tonsimple

This comment has been minimized.

Copy link

commented Mar 24, 2018

@micahflee
Sorry for nagging, but could you please kindly step-by-step creation of android HVMs under Qubes R3.2 for the less talented of us?

can't seem to make it boot/install (tried Android 4.4.4 and later)

@tschakram

This comment has been minimized.

Copy link

commented Jul 4, 2018

Hello,
any solution to get a android hvm running on Qubes 4 ? Booting a live iso worked for all Versions of Androidx86,RemixOS and CM. But installtion failed because of the error "No hard drive avalible". Possible to prepare the hvm drive before installing?
thanks

@ptitdoc

This comment has been minimized.

Copy link

commented Aug 7, 2018

Hello,

Here are my limited success to deal with an Android HVM in Qubes R4:
1/ Download Android x86 ISO (tested with android-x86-7.1-r2.iso)
2/ Create an HVM
3/ Give at least 4000MB of RAM to the HVM and disable memory balancing (testing with 2000MB caused the VM to crash during startup)
4/ In the QubesVM advanced settings, select "Boot qubes from CDROM" and use "from file in Qubes" to select the ISO you downloaded
5/ When the VM boots, select Advanced options > Live CD VESA mode - No GPU hardware acceleration. Other options are working but this is the most stable I found. (Press space when android boots up)

Here are the issues I'm also facing:

  • The mouse works strangely because of QubesOS libvirt settings. For this reason, it is needed to use drag and drop all the time inside the Android window in order to use the mouse. To improve this it is possible to edit in dom0 /usr/share/qubes/templates/libvirt/xen.xml and to change the line:
    <input type="tablet" bus="usb" />
    to
    <input type="mouse" bus="usb" />
  • It is still not possible to install the android VM using recent Android version because the hard disk drive is not detected. Apparently Android does not supports disks presented as Xen block devices. Maybe the problem is similar to #3068.
@airelemental

This comment has been minimized.

Copy link

commented Aug 7, 2018

@ptitdoc

This comment has been minimized.

Copy link

commented Aug 8, 2018

Thanks for the pointer.

By the way, people are talking about problems with networking, but networking works out of the box in my case.

@mat-rex

This comment has been minimized.

Copy link

commented Aug 8, 2018

ptitdoc, what version of android did you try? I think I shot myself in the foot with 7.1.2, as might have a bug with ethernet and dhcp.

I did the following 2 changes:

  1. Changed 1 line in "init" inside initrd
    from:
    for device in ${ROOT:-/dev/[hmnsv][dmrv][0-9a-z]}; do
    to:
    for device in ${ROOT:-/dev/[hmnsvx][dmrv][0-9a-z]
    }; do
    That allows the XEN hard drive to be used at initialization, once the kernel has the driver.
    I would appreciate if this change would go into the repo there if you (enyone) are an android-x86 developer.

  2. I configured the kernel

@mat-rex

This comment has been minimized.

Copy link

commented Sep 21, 2018

A bit of a time passed, and on the sideline I have a config with which the mouse is a mouse, but you have two pointers. If you change the cursor to large, you can get used to it. Network configuration is stil manual and a pain.

Once you build android with that you get a CD (The best if you use the android common build environment... it took me days to get a build on 16G with qubes... but can be done). If that is too much, I can provide you the CD somehow (but why would you trust someone to provide you a CD ... but if in that VM, you have nothing to loose, you do not risk too much :-) ).

Create a new HVM. Boot a recent linux CD/DVD/thumbdrive (I was using the openly available, but unannounced knoppix 8,2).

boot your VM from the linux CD. Create an EXT2 partition to boot (50 or 500 megs? do not remember). install the bootloader from the cd.

Shutdown linux. Boot android.

Install Android to the free space, creating a new partition "ext4". reboot.

Boot linux again. Mount both partitions, copy kernel and inititial ramdisk, into the boot directory. Do NOT use the same name as what the android-x86 install created on the EXT4 partition.

google:
andoid-x86 boot grub

update grub/menu.lst according to the instructions found, modify directory name.

reboot end enjoy if you can without a network.

If you need networking, google android-x86 static ip.Use the IP quebes allocated your vm. Or find instructions on how to setup dhcp from the command line - I did not figure that. I have no clue why it does not autodetect.

If some succeeds, please post detailed instructions and CD. I posted this without the actual system at hand, so steps are as I remember from top of my head from many weeks ago.

Please post if you fail as well, when I'll check here (maybe soon, maybe a few month) if there are failure reports I'll try to post a detailed instructions with the system at hand.

@mat-rex

This comment has been minimized.

Copy link

commented Oct 28, 2018

The behaviour of the mouse in qubes 4 appears to come from:

/usr/share/qubes/templates/libvirt/xen.xml

Replace the word " tablet " with " mouse ", and mouse will become an emulated mouse in the HVMs. You get two mouse pointers - but if you change the size (I set it to max of 48) of your desktop mouse (Dom0 - system tools - settings manager - Mouse and touchpad - Theme) so the two can be distinguished, it is usable.

This allowed me to run a BSD gui environment in a HWM. I'll see how this changes things for Android.

@Thovthe

This comment has been minimized.

Copy link

commented Nov 3, 2018

I hear that @thestinger may, eventually, be working on this.

@thestinger

This comment has been minimized.

Copy link

commented Nov 3, 2018

I'll be using the x86 support in the Android Open Source Project rather than the Android x86 fork though. If there are relevant patches, they can be cherry-picked. I'm not comfortable with some of their changes from a security perspective and I don't want / need additions like an installer as I'll be outputting the images in the appropriate format directly from the builds. I don't want to lose support for block-based updates and verified boot compared to a standard Android installation, but there are other options beyond the traditional way of implementing it.

I also want to have the latest stable release of Android, without a long delay of many months. I intend to land what I can upstream to aid with future porting and maintenance as I've done with some of my hardening work on Android in the past. It will be helpful even if only a few basic things can be landed.

The work involved for full integration as a proper AppVM goes beyond what I'll be able to do within the current scope of the project, but I can get started on it.

@Thovthe

This comment has been minimized.

Copy link

commented Nov 3, 2018

Even a nice standalone_vm would be amazing. Have you planned what qubes utilities you are going to implement?

@v6ak

This comment has been minimized.

Copy link

commented Nov 3, 2018

@thestinger

This comment has been minimized.

Copy link

commented Nov 3, 2018

Android looks like it was designed for TemplateVMs – it has separate immutable /system and mutable /data. (Yes, they are few other directories like /vendor etc., but it seems all of them are like /data or like /system…)

There's the boot image (read-only, verified by bootloader), system (read-only, verified by kernel) and data (encrypted and wiped by a factory reset). Having dtb (read-only, verified by bootloader) split from boot is optional to reuse a boot image across devices with different device trees. Similarly, vendor (read-only, verified by kernel) is split out from system on modern devices so that system can be device independent, although forward looking devices split it long before the hardware abstraction was implemented. Modern devices don't need a cache partition and other partitions are either verified firmware partitions or very limited firmware-related state (misc, persist, etc.).

On an A/B update device, there are two actual partitions for each read-only / verified partition, i.e. the OS and firmware providing A and B slots. One is the active set of slots verified on boot and optionally with hardware read-only enforcement. The other can be written by the OS as part of updating (usually a delta from the active set), and then it can verify that it was all written out correctly (hashes match), mark it as the active set and reboot. If it fails to boot, including a verified boot failure, it will roll back to the old update. Once it makes it to late boot, it marks the new active set as good and rollback is disabled.

The legacy update mechanism involves having a separate recovery partition, which is essentially another boot image and verified in the same way. The OS has to pass the update to recovery via a partition with unencrypted state (cache, which is no longer needed with A/B updates), although it can be done without copying it there via their uncrypt hack.

Updates don't necessarily need to be done with these existing mechanisms. There are major robustness and usability advantages to A/B updates, so those are worth having, even if it's not implemented in exactly the usual way.

In a standalone VM, you are probably missing some non-modifiable bootloader. Without that, an attacker can just disable the verified boot.

The primary purpose of verified boot is defending against a compromise of the OS by making it difficult to persist with a high level of privileges. The protection against attackers from outside (generally physical tampering, but a bit broader in this case) doesn't apply without QubesOS having it and chaining it along which would be far out-of-scope and not what I mean by preserving it.

  • In a TemplateVM, you have immutable /system by design. Verified boot seems redundant.

Not redundant, but not useful due to it not being chained from higher up. If the standard layout of a modern device is followed, /system can be immutable in more than just a template.

@thestinger

This comment has been minimized.

Copy link

commented Nov 3, 2018

It would be completely reasonable to only have updates of the base images externally with them always read-only within the OS. That's what I mean by handling it another way, i.e. not actually handling updates within the OS, and sharing the entire OS images even after updates rather than copy-on-write since they're always identical across installations. I don't know the best approach, but it would certainly be easier to avoid dealing with all the bootloader complexity for A/B updates, especially dealing with verified boot and rollback protection.

@v6ak

This comment has been minimized.

Copy link

commented Nov 3, 2018

@thestinger

This comment has been minimized.

Copy link

commented Nov 3, 2018

The existing update systems (A/B updates or the legacy recovery system) require some bootloader support that I'd need to implement so that's why it's worth figuring out the best approach in advance. I think it'd be easier to implement the legacy system, but it's on the way out and has serious robustness disadvantages. It seems the Android-x86 fork doesn't provide updates at all, so there's not an existing implementation of the bootloader support.

@fgvhcw

This comment has been minimized.

Copy link

commented Nov 16, 2018

@mat-rex Can you please upload the working kernel config? The config that you posted in qubes-users got truncated when you copied it between VMs.

@mat-rex

This comment was marked as outdated.

Copy link

commented Dec 13, 2018

@mat-rex

This comment has been minimized.

Copy link

commented Dec 13, 2018

@mat-rex

This comment has been minimized.

Copy link

commented Dec 13, 2018

@marmarek

This comment has been minimized.

Copy link
Member

commented Feb 23, 2019

A post with detailed setup (including build from source) instruction for Android x86 in a Standalone VM on Qubes 4.0:
https://groups.google.com/d/msgid/qubes-users/53bdab0a-8c70-4fc3-9d4b-224b43849946%40googlegroups.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.