This repository has been archived by the owner. It is now read-only.

Enable 32 bit x86 support - postmarket PC to build postmarket OS #933

Closed
dee-gomma opened this Issue Nov 23, 2017 · 8 comments

Comments

Projects
None yet
3 participants
@dee-gomma

dee-gomma commented Nov 23, 2017

Hi all,

For who like me is still running an old 32 bit PC and wants to play with postmarketOS, I've created a working fork here.
It should work for any x86 architecture (i386 etc., I have i686). Unfortunately it is not yet ready for a pull request (some temporary stuff to be cleaned up, see below), and I have very few spare time lately, so if anybody is interested in completing it that would be great.

Basically, what has been done:

  • Generated needed aports for x86 architecture through "aportgen": binutils, busybox-static, gcc, musl
  • Modified pmb/parse/arch.py adding x86 architecture mappings (possibly to be reviewed)
  • Overwritten aports/cross/quemu-user-static-repack/APKBUILD: original arch x86_64 replaced with x86 and references updated accordingly (quick and dirty temporary solution, we should keep 2 versions for both x86_64 and x86 instead)

I was able to build kernel and pmOS on my i686 PC for a Samsung Galaxy GrandPrime (new potentially supported device, will push it as soon as I clean it up decently)
I could not compile Weston UI unfortunately, looks like a dependency (libunwind-dev) has no x86 support.

Thanks to @drebrez for the guidance and you all for building up this great project!

@ollieparanoid

This comment has been minimized.

Show comment
Hide comment
@ollieparanoid

ollieparanoid Nov 25, 2017

Member

Great work! \o/

The problematic part is the qemu-user-static-repack package, and my favorite solution would be building that for Alpine instead of using the Debian package - then it would work with all architectures. But this is a really good start if someone is in the same situation with an x86 PC!

I don't get what you mean with the libunwind-dev though, since we are cross-compiling you would only need it for the armhf/aarch64 chroot, where it is available. Could you pastebin the end of the log, where you got stuck?

Member

ollieparanoid commented Nov 25, 2017

Great work! \o/

The problematic part is the qemu-user-static-repack package, and my favorite solution would be building that for Alpine instead of using the Debian package - then it would work with all architectures. But this is a really good start if someone is in the same situation with an x86 PC!

I don't get what you mean with the libunwind-dev though, since we are cross-compiling you would only need it for the armhf/aarch64 chroot, where it is available. Could you pastebin the end of the log, where you got stuck?

@dee-gomma

This comment has been minimized.

Show comment
Hide comment
@dee-gomma

dee-gomma Nov 26, 2017

Thank you for the feedback :)

Yes building qemu package independently would be the best solution in fact.

About weston and libunwind-dev dependency, it's a note I've added in my todo list to be investigated. I could not remember exactly the context and what I did to get there, so maybe I am wrong, I truly need to clean-up and try to reproduce it, then I'll ping you back again in case.
Anyway, being that considered, here below few inputs from the current status:

  • on device, in tmp/weston.log, got the following:
    Error loading shared library libweston-3.so.0: No such file or directory (needed by /usr/bin/weston)
  • By running ./pmbootstrap.py build weston, got following error in command line:
    (native) install alpine-base
    (native) install abuild build-base ccache
    ERROR: Could not find package 'libunwind-dev' in the aports folder and could not find it in any APKINDEX 
    
    Extract of log file (note the arch: x86 at the end of 1st line):
    (003919) [23:02:54] Calculate depends of packages ['wayland-protocols', 'libxkbcommon-dev', 'xkeyboard-config', 'libinput-dev', 'libunwind-dev', 'mtdev-dev', 'libxcursor-dev', 'glu-dev', 'pango-dev', 'colord-dev', 'freerdp-dev', 'libwebp-dev', 'libva-dev', 'dbus-dev', 'linux-pam-dev'], arch: x86
    (003919) [23:02:55] ERROR: Could not find package 'libunwind-dev' in the aports folder and could not find it in any APKINDEX
    (003919) [23:02:55] Run 'pmbootstrap log' for details.
    (003919) [23:02:55] See also: <https://postmarketos.org/troubleshooting>
    (003919) [23:02:55] Traceback (most recent call last):
    File "/home/boris/pmbootstrap/pmb/__init__.py", line 58, in main
     getattr(frontend, args.action)(args)
    File "/home/boris/pmbootstrap/pmb/helpers/frontend.py", line 92, in build
     args.buildinfo, args.strict)
    File "/home/boris/pmbootstrap/pmb/build/package.py", line 66, in package
     pmb.chroot.apk.install(args, apkbuild["makedepends"], suffix)
    File "/home/boris/pmbootstrap/pmb/chroot/apk.py", line 197, in install
     strict=True)
    File "/home/boris/pmbootstrap/pmb/parse/depends.py", line 94, in recurse
     in_apkindexes))
    RuntimeError: Could not find package 'libunwind-dev' in the aports folder and could not find it in any APKINDEX
    

dee-gomma commented Nov 26, 2017

Thank you for the feedback :)

Yes building qemu package independently would be the best solution in fact.

About weston and libunwind-dev dependency, it's a note I've added in my todo list to be investigated. I could not remember exactly the context and what I did to get there, so maybe I am wrong, I truly need to clean-up and try to reproduce it, then I'll ping you back again in case.
Anyway, being that considered, here below few inputs from the current status:

  • on device, in tmp/weston.log, got the following:
    Error loading shared library libweston-3.so.0: No such file or directory (needed by /usr/bin/weston)
  • By running ./pmbootstrap.py build weston, got following error in command line:
    (native) install alpine-base
    (native) install abuild build-base ccache
    ERROR: Could not find package 'libunwind-dev' in the aports folder and could not find it in any APKINDEX 
    
    Extract of log file (note the arch: x86 at the end of 1st line):
    (003919) [23:02:54] Calculate depends of packages ['wayland-protocols', 'libxkbcommon-dev', 'xkeyboard-config', 'libinput-dev', 'libunwind-dev', 'mtdev-dev', 'libxcursor-dev', 'glu-dev', 'pango-dev', 'colord-dev', 'freerdp-dev', 'libwebp-dev', 'libva-dev', 'dbus-dev', 'linux-pam-dev'], arch: x86
    (003919) [23:02:55] ERROR: Could not find package 'libunwind-dev' in the aports folder and could not find it in any APKINDEX
    (003919) [23:02:55] Run 'pmbootstrap log' for details.
    (003919) [23:02:55] See also: <https://postmarketos.org/troubleshooting>
    (003919) [23:02:55] Traceback (most recent call last):
    File "/home/boris/pmbootstrap/pmb/__init__.py", line 58, in main
     getattr(frontend, args.action)(args)
    File "/home/boris/pmbootstrap/pmb/helpers/frontend.py", line 92, in build
     args.buildinfo, args.strict)
    File "/home/boris/pmbootstrap/pmb/build/package.py", line 66, in package
     pmb.chroot.apk.install(args, apkbuild["makedepends"], suffix)
    File "/home/boris/pmbootstrap/pmb/chroot/apk.py", line 197, in install
     strict=True)
    File "/home/boris/pmbootstrap/pmb/parse/depends.py", line 94, in recurse
     in_apkindexes))
    RuntimeError: Could not find package 'libunwind-dev' in the aports folder and could not find it in any APKINDEX
    
@dee-gomma

This comment has been minimized.

Show comment
Hide comment
@dee-gomma

dee-gomma Nov 26, 2017

(Nevermind - should be related to the missing --arch=armhf option in pmbootstrap build)

dee-gomma commented Nov 26, 2017

(Nevermind - should be related to the missing --arch=armhf option in pmbootstrap build)

@drebrez

This comment has been minimized.

Show comment
Hide comment
@drebrez

drebrez Jan 17, 2018

Member

@ollieparanoid I tried to make pmbootstrap work on a x86 virtualmachine with the initial work of @dee-gomma, but trying to package the the qemu-user-static-repack properly.
I first started by setting the APKBUILD to "arch=all" and updated internally to package the correct version based on the architecture it is built (abuild has a CARCH env variable for that).
Everything worked fine until I started to try to cross-compile packages for x86_64 and there I discovered some issues/bugs in the various arch to arch functions and hardcoded behavior for -repack suffixes.

Member

drebrez commented Jan 17, 2018

@ollieparanoid I tried to make pmbootstrap work on a x86 virtualmachine with the initial work of @dee-gomma, but trying to package the the qemu-user-static-repack properly.
I first started by setting the APKBUILD to "arch=all" and updated internally to package the correct version based on the architecture it is built (abuild has a CARCH env variable for that).
Everything worked fine until I started to try to cross-compile packages for x86_64 and there I discovered some issues/bugs in the various arch to arch functions and hardcoded behavior for -repack suffixes.

@drebrez

This comment has been minimized.

Show comment
Hide comment
@drebrez

drebrez Jan 17, 2018

Member

Looking again at this commit (dee-gomma@0897321) I'm getting confused.
@ollieparanoid can you correct me if I'm wrong?
From what I understand now, the cross/binutils-x86, cross/busybox-static-x86, cross/gcc-x86 and cross/musl-x86 are only required if I have a host architecture different than x86 and I want to compile a package for x86.
And for example on a x86 if I want to build an x86 package, I don't need those because I can build it in the native chroot.
I've also tried to build an x86_64 package from the x86 host, and that case requires the cross/binutils-x86_64, cross/gcc-x86_64,... packages.
Is it correct?

Now the question is, which host architectures do we want to support for running pmbootstrap?

Since in init.py there is build_device_architectures = ["armhf", "aarch64", "x86_64"], is it correct that we need all the corresponding binutils-*, busybox-static-*,... packages for the cross-compilation?
On the first page the requirement say Linux distribution (x86_64 or aarch64), but probably with aarch64 it will fail if it tries to build the device-qemu-amd64 or device-teclast-x80pro.

Someone has already tried to use pmbootstrap in an armhf device? and cross-compile to aarch64 maybe?

Member

drebrez commented Jan 17, 2018

Looking again at this commit (dee-gomma@0897321) I'm getting confused.
@ollieparanoid can you correct me if I'm wrong?
From what I understand now, the cross/binutils-x86, cross/busybox-static-x86, cross/gcc-x86 and cross/musl-x86 are only required if I have a host architecture different than x86 and I want to compile a package for x86.
And for example on a x86 if I want to build an x86 package, I don't need those because I can build it in the native chroot.
I've also tried to build an x86_64 package from the x86 host, and that case requires the cross/binutils-x86_64, cross/gcc-x86_64,... packages.
Is it correct?

Now the question is, which host architectures do we want to support for running pmbootstrap?

Since in init.py there is build_device_architectures = ["armhf", "aarch64", "x86_64"], is it correct that we need all the corresponding binutils-*, busybox-static-*,... packages for the cross-compilation?
On the first page the requirement say Linux distribution (x86_64 or aarch64), but probably with aarch64 it will fail if it tries to build the device-qemu-amd64 or device-teclast-x80pro.

Someone has already tried to use pmbootstrap in an armhf device? and cross-compile to aarch64 maybe?

@ollieparanoid

This comment has been minimized.

Show comment
Hide comment
@ollieparanoid

ollieparanoid Jan 17, 2018

Member

qemu-user-static-repack

@ollieparanoid I tried to make pmbootstrap work on a x86 virtualmachine with the initial work of @dee-gomma, but trying to package the the qemu-user-static-repack properly.
I first started by setting the APKBUILD to "arch=all" and updated internally to package the correct version based on the architecture it is built (abuild has a CARCH env variable for that).
Everything worked fine until I started to try to cross-compile packages for x86_64 and there I discovered some issues/bugs in the various arch to arch functions and hardcoded behavior for -repack suffixes.

Cool that you're looking into this! I think the only proper way to package qemu-user-static is, to not repack a Debian package, but to extend Alpine's qemu aport to provide qemu-user-static-armhf etc. for various architectures. Then the architecture problem would go away as nice side effect.

AFAIK the only hardcoded behavior for -repack is, that it gets built in the native chroot (the idea was, that if you're only repackaging an archive, it should work in all architectures, and the native one is the fastest one). (This is legacy from days before pmbootstrap was published, where it repackaged LineageOS binary kernels before pmOS was capable of compiling its own kernels. For the Qemu package it does not really make sense, since that will be the native architecture anyway.)

Cross-compiling *-$arch packages

From what I understand now, the cross/binutils-x86, cross/busybox-static-x86, cross/gcc-x86 and cross/musl-x86 are only required if I have a host architecture different than x86 and I want to compile a package for x86.

When compiling for a foreign arch you only need these: musl-$arch, binutils-$arch and gcc-$arch

busybox-static-$arch

The busybox-static-$arch package is a different story: It's only required for isorec kernel devices. We don't use it for compiling there, but put the armhf busybox binary into the initramfs that gets baked into the kernel (so we don't have to use a random busybox binary blob shipped in the kernel's git repo). If there was a isorec aarch64 device (or another reason for having busybox in the built in initramfs in the kernel), then we would need busybox-static-aarch64. But right now it looks like we only need this for armhf. (Oh and and I've just looked at the repo. We have a busybox-static-aarch64 aport in there, but we never use it.).

Example

And for example on a x86 if I want to build an x86 package, I don't need those because I can build it in the native chroot.
I've also tried to build an x86_64 package from the x86 host, and that case requires the cross/binutils-x86_64, cross/gcc-x86_64,... packages.
Is it correct?

Yep!

Which host architectures to support in pmbootstrap?

Now the question is, which host architectures do we want to support for running pmbootstrap?

We can support all architectures, that Alpine runs on, as host architectures, once qemu-user-static is provided by Alpine (for all arch combinations). If that was the case, then the postmarketOS package count does not change when we add more host architectures so I don't see a reason why we should limit ourselves there.

Since in init.py there is build_device_architectures = ["armhf", "aarch64", "x86_64"], is it correct that we need all the corresponding binutils-*, busybox-static-*,... packages for the cross-compilation?
On the first page the requirement say Linux distribution (x86_64 or aarch64), but probably with aarch64 it will fail if it tries to build the device-qemu-amd64 or device-teclast-x80pro.

busybox-static- probably not, see above. The other packages: As of now, no one compiling aarch64 -> x86_64, so it does not matter much that it is missing in practice. But you are right, we should provide them to have real support for non-x86_64 host architectures.

Someone has already tried to use pmbootstrap in an armhf device?

That could be.

and cross-compile to aarch64 maybe?

I doubt that.

PS: I want to point out that we have the following function, to check if a cross-compiler is necessary to compile from one host architecture to a given target architecture:

def cpu_emulation_required(args, arch):
# Obvious case: host arch is target arch
if args.arch_native == arch:
return False
# Other cases: host arch on the left, target archs on the right
not_required = {
"x86_64": ["x86"],
"aarch64": ["armel", "armhf", "armv7"],
}
if args.arch_native in not_required:
if arch in not_required[args.arch_native]:
return False
# No match: then it's required
return True

Member

ollieparanoid commented Jan 17, 2018

qemu-user-static-repack

@ollieparanoid I tried to make pmbootstrap work on a x86 virtualmachine with the initial work of @dee-gomma, but trying to package the the qemu-user-static-repack properly.
I first started by setting the APKBUILD to "arch=all" and updated internally to package the correct version based on the architecture it is built (abuild has a CARCH env variable for that).
Everything worked fine until I started to try to cross-compile packages for x86_64 and there I discovered some issues/bugs in the various arch to arch functions and hardcoded behavior for -repack suffixes.

Cool that you're looking into this! I think the only proper way to package qemu-user-static is, to not repack a Debian package, but to extend Alpine's qemu aport to provide qemu-user-static-armhf etc. for various architectures. Then the architecture problem would go away as nice side effect.

AFAIK the only hardcoded behavior for -repack is, that it gets built in the native chroot (the idea was, that if you're only repackaging an archive, it should work in all architectures, and the native one is the fastest one). (This is legacy from days before pmbootstrap was published, where it repackaged LineageOS binary kernels before pmOS was capable of compiling its own kernels. For the Qemu package it does not really make sense, since that will be the native architecture anyway.)

Cross-compiling *-$arch packages

From what I understand now, the cross/binutils-x86, cross/busybox-static-x86, cross/gcc-x86 and cross/musl-x86 are only required if I have a host architecture different than x86 and I want to compile a package for x86.

When compiling for a foreign arch you only need these: musl-$arch, binutils-$arch and gcc-$arch

busybox-static-$arch

The busybox-static-$arch package is a different story: It's only required for isorec kernel devices. We don't use it for compiling there, but put the armhf busybox binary into the initramfs that gets baked into the kernel (so we don't have to use a random busybox binary blob shipped in the kernel's git repo). If there was a isorec aarch64 device (or another reason for having busybox in the built in initramfs in the kernel), then we would need busybox-static-aarch64. But right now it looks like we only need this for armhf. (Oh and and I've just looked at the repo. We have a busybox-static-aarch64 aport in there, but we never use it.).

Example

And for example on a x86 if I want to build an x86 package, I don't need those because I can build it in the native chroot.
I've also tried to build an x86_64 package from the x86 host, and that case requires the cross/binutils-x86_64, cross/gcc-x86_64,... packages.
Is it correct?

Yep!

Which host architectures to support in pmbootstrap?

Now the question is, which host architectures do we want to support for running pmbootstrap?

We can support all architectures, that Alpine runs on, as host architectures, once qemu-user-static is provided by Alpine (for all arch combinations). If that was the case, then the postmarketOS package count does not change when we add more host architectures so I don't see a reason why we should limit ourselves there.

Since in init.py there is build_device_architectures = ["armhf", "aarch64", "x86_64"], is it correct that we need all the corresponding binutils-*, busybox-static-*,... packages for the cross-compilation?
On the first page the requirement say Linux distribution (x86_64 or aarch64), but probably with aarch64 it will fail if it tries to build the device-qemu-amd64 or device-teclast-x80pro.

busybox-static- probably not, see above. The other packages: As of now, no one compiling aarch64 -> x86_64, so it does not matter much that it is missing in practice. But you are right, we should provide them to have real support for non-x86_64 host architectures.

Someone has already tried to use pmbootstrap in an armhf device?

That could be.

and cross-compile to aarch64 maybe?

I doubt that.

PS: I want to point out that we have the following function, to check if a cross-compiler is necessary to compile from one host architecture to a given target architecture:

def cpu_emulation_required(args, arch):
# Obvious case: host arch is target arch
if args.arch_native == arch:
return False
# Other cases: host arch on the left, target archs on the right
not_required = {
"x86_64": ["x86"],
"aarch64": ["armel", "armhf", "armv7"],
}
if args.arch_native in not_required:
if arch in not_required[args.arch_native]:
return False
# No match: then it's required
return True

@drebrez

This comment has been minimized.

Show comment
Hide comment
@drebrez

drebrez Jan 17, 2018

Member

qemu-user-static-repack

I know that the proper way to package it is to build it by ourself, maybe I'll take a look. The simplest/fastest solution I've done was to download the .deb of both architectures (amd64 and i686) and in the APKBUILD unpack function only extract the $CARCH corresponding architecture. And set the package to arch="all". What you think about adding this solution temporarily until we package it properly?

Cross-compiling *-$arch packages

x86_64 is the only missing right now, it should always reflect what's inside the build_device_architectures variable in pmb/config/init.py.
I will create a PR that adds those 3 packages for x86_64 and add a comment about this beside the variable.

busybox-static-$arch

thanks for remembering me what's the purpose of it, the aarch64 is not needed but can stay there for future uses/needs

Which host architectures to support in pmbootstrap?

I will at least conclude the support for x84 so that @dee-gomma can work on postmarketOS with his old laptop 👍.
In future I would like to try pmbootstrap from a device directly, at least to see if it works with armhf and aarch64 architectures and cross compile from there.

cpu_emulation_required function

I know about that function, but today I found out an issue in another function, will create a PR for it
but I want to check it again first. Seems like a bug in a function that gets casually resolved by another function. I can try to explain it better in chat.

Member

drebrez commented Jan 17, 2018

qemu-user-static-repack

I know that the proper way to package it is to build it by ourself, maybe I'll take a look. The simplest/fastest solution I've done was to download the .deb of both architectures (amd64 and i686) and in the APKBUILD unpack function only extract the $CARCH corresponding architecture. And set the package to arch="all". What you think about adding this solution temporarily until we package it properly?

Cross-compiling *-$arch packages

x86_64 is the only missing right now, it should always reflect what's inside the build_device_architectures variable in pmb/config/init.py.
I will create a PR that adds those 3 packages for x86_64 and add a comment about this beside the variable.

busybox-static-$arch

thanks for remembering me what's the purpose of it, the aarch64 is not needed but can stay there for future uses/needs

Which host architectures to support in pmbootstrap?

I will at least conclude the support for x84 so that @dee-gomma can work on postmarketOS with his old laptop 👍.
In future I would like to try pmbootstrap from a device directly, at least to see if it works with armhf and aarch64 architectures and cross compile from there.

cpu_emulation_required function

I know about that function, but today I found out an issue in another function, will create a PR for it
but I want to check it again first. Seems like a bug in a function that gets casually resolved by another function. I can try to explain it better in chat.

@ollieparanoid

This comment has been minimized.

Show comment
Hide comment
@ollieparanoid

ollieparanoid Jan 17, 2018

Member

I think downloading the deb for both multiple architectures as a quick hack to support another architecture is okay, if it unblocks @dee-gomma.
The rest you're planning sounds good to me! 👍

Member

ollieparanoid commented Jan 17, 2018

I think downloading the deb for both multiple architectures as a quick hack to support another architecture is okay, if it unblocks @dee-gomma.
The rest you're planning sounds good to me! 👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.