Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce cross-arch testing #28

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Commits on Nov 18, 2023

  1. Increase cross-compile compatibility

    Some user distros do not use standard triplets, but instead, the triplet has another name.
    Move the triplet name into a gitignored config file,
    and fix a few trivial error messages breaking a cross-compile build:
    - use headers-generic instead of headers-$arch
    - if ulimit isn't set, don't do it.
    
    Signed-off-by: jpsollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 18, 2023
    Configuration menu
    Copy the full SHA
    5a1ec14 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. Run bcachefs tests on emulated architectures

    Allow the tests to run on a non-ABI-compatible cpu,
    verify code when the architecture isn't physically available.
    in the future, bcachefs-tools should be cross-compiled,
    and the list of CPU types should be expanded
    
    Signed-off-by: jpsollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    b513eff View commit details
    Browse the repository at this point in the history
  2. ktest: upgrade to debian trixie channel

    Due to bcachefs being mainlined,
    it's worth considering a more recent update channel,
    which may include bcachefs features in the future.
    
    Necessary adjustments have been made for the packages,
    but kernel requirements may still need some extra tweaks
    
    Signed-off-by: jpsollie <janpieter.sollie@edpnet.be>
    
    ktest: enable debian sid fallback
    
    RISC-V is currently moving from debian-ports to debian,
    in order to protect from unreleased architectures etc,
    a fallback scenario is needed:
    in case preconfigured settings do not work,
    apply a default sid fallback
    
    Signed-Off-By: jpsollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    3704efe View commit details
    Browse the repository at this point in the history
  3. enable ARMv7 support

    Ktest has now been expanded with cross-binary armv7 (hard float).
    this should allow some testing on more legacy armv7 hardware,
    which may still be present in NAS devices.
    However, due to the nature of the ecosystem, a limit is in place:
    - 3GB ram for 32 bit systems
    - 4 CPUs for 32 bit systems
    I think if you need more, you'd better think of going 64-bit.
    These constraints allow the ARM emulator to run smoothly.
    
    Signed-off-by:  jpsollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    d6aa59b View commit details
    Browse the repository at this point in the history
  4. Enable rust support cross-arch

    Make the rust compiler instsallable via rustup even though it's not in the vm yet.
    This avoids rust compile errors on emulated CPUs,
    rustup installs for host cpu kernel architecture instead of VM architecture.
    
    Signed-off-by: jpsollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    7f4ee0a View commit details
    Browse the repository at this point in the history
  5. Ktest: re-introduce foreign unsupported architectures

    Whereas MIPS and MIPS64 are abandoned my manufacturer,
    SPARC and POWER seem to be still alive,
    and upcoming RISCV also deserves some attention,
    so add those to the list of supported architectures.
    
    Signed-off-by: Janpieter Sollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    721ed7b View commit details
    Browse the repository at this point in the history
  6. fix a number of issues with cross-architecture testing

    This commit fixes a _LOT_ of issues with the previous commits,
    but at least there's one supported big endian architecture now,
    which is capable of booting and running ktests: s390x
    
    For unsupported architectures, a number of root_create features:
    - enabled snapshot lookups for debian archives,
      in case of missing dependencies, it may still be in there
    - enabled possibility to build the package from source,
      this is painful
    - enabled the possibility during root_create to intervene,
      this is only available on unsupported architectures.
    
    So far, it seems the qemu E6500 emulator for powerPC isn't stable,
    and rustup doesn't know about sparc / sparc64,
    so both are currently broken.
    
    Re-organised the cross.conf file as the first one was a bit messy.
    
    Signed-Off-by: Janpieter Sollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    856c6b3 View commit details
    Browse the repository at this point in the history
  7. Fall back to sid rustc if rustup isn't available

    Rustup only has a limited number of architectures,
    when the root_image we are creating isn't supported,
    try to install the debian rustc compiler.
    this will be older, but >95% of time, it should be ok.
    
    Signed-off-by: Janpieter Sollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    81000df View commit details
    Browse the repository at this point in the history
  8. fixup!:

    - fix malfunctioning variables
    - provide sid fallback for missing architectures
    - make the extended installer default for sid
    
    Signed-off-by: Janpieter Sollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    be4a189 View commit details
    Browse the repository at this point in the history
  9. ktest: use overlays to store arch-specific bcachefs-tools build

    when running tests, eg test1 on arch 1, and then on arch 2,
    the whole bcachefs-tools package needs to be rebuilt:
    all binaries of arch 1 need to be erased, and recompiled for arch 2
    when going back to arch 1, it happens again
    
    This patch fixes the issue by applying a arch-specific overlay,
    so bcachefs-tools doesn't need a full rebuild every time.
    The overlay is mounted via fuse-overlay.
    If this isn't available, it will fall back to rsync,
    causing a larger waste of disk space,
    and not-applied-immediately git commits.
    
    Signed-of-by: jpsollie <janpieter.sollie@edpnet.be>
    jpsollie committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    48e153e View commit details
    Browse the repository at this point in the history