enabling ppc64le for Fedora CoreOS

[manojnkumar]

I work in the Power systems development labs at IBM. Currently my responsibilities include OpenShift and OpenStack on Power. When OpenShift pivots from Atomic to CoreOS, I would like to ensure that ppc64le is a primary architecture.

Can contribute some portion of my time to resolving issues identified for Power (ppc64le).

[dustymabe]

Thanks @manojnkumar. Sounds good to me. @sinnykumari has been helped us get ppc64le working in the past for Atomic Host so she should be able to help in this effort as well.

A few things I can think of as things we need to resolve:

• getting coreos-assembler working (see coreos-assembler breaks on ppc64le coreos-assembler#194)
• since we're going to try to target kubernetes/openshift as a build environment it would be nice if the openshift instance in CentOS CI supported multi-arch
• the current plan was to store our coreos-assembler "build container" in quay.io. but we may need to pivot that if quay.io is going to take some time to support multi-arch. I think registry.fedoraproject.org already does today, but I haven't tested it lately.

[sinnykumari]

Yes, can confirm that registry.fedoraproject.org support multiple arches now.
+1 for hosting coreos-assembler images to registry.fedoraproject.org until we have mult-arch support in quay.io .

[bgilbert]

• mantle will need to learn about ppc64le guests. The arch-specific work shouldn't be very involved, but https://github.com/coreos/mantle/issues/943 is probably a prerequisite.
• Ignition can't obtain userdata from QEMU on ppc64le (Support userdata on QEMU PPC64 ignition#666). This isn't a problem for deployment on bare metal, but it makes OS development more difficult and means that kola can't run tests on the architecture.

[ajeddeloh]

General question: how does bootloading work on ppc64le? Do you generally use grub's powerpc-ieee1275 target? Is there a special bootloader?

[sharkcz]

There are 2 cases for modern Power HW

• "pseries" hardware (PowerVM guests/LPARs, KVM/qemu pseries guests) use grub binary (powerpc-ieee1275 aka OpenFirmware)
• "powernv" aka OpenPower bare-metal (and later KVM/qemu powernv guests), they use petitboot in the firmware (== a Linux mini-distro), that interprets other bootloaders' config files (also has a native config)

[jcajka]

IIRC "pseries", requires prepboot partition and "powernv" thanks to the (grub) conf parsing is fine with "intel" like partitioning scheme(without prepboot). @sharkcz correct me if I'm wrong.

[cgwalters]

Does petitboot support BLS?

[sharkcz]

Does petitboot support BLS?

it does, since version 1.8.0. The problem might be that it's part of the firmware, so the version avaiable relies on the HW vendors (like IBM or SuperMicro) releasing new firmware images. On the other hand CoreOS could make "petitboot with BLS" its requirement.

[sharkcz]

for OpenPOWER machines one can build the firmware themselves from the github upstream sources

[ajeddeloh]

@sharkcz so the bootloader is independent of what's on the disk you boot from?

We're using BLS right now but we may want to change that later to support automatic rollback (or do some hybrid approach). It might be that different platforms get different levels of support with regards to automatic rollback.

[sharkcz]

@sharkcz so the bootloader is independent of what's on the disk you boot from?

yes, petitboot (the whole mini-distro) lives in a flash chip and allows booting from any device the underlying Linux kernel (+ related tools like lvm) understands and contains a valid bootloader config file. But I don't know where it stores eg. the last selected/booted kernel or its own configuration (an NVRAM maybe?).

We're using BLS right now but we may want to change that later to support automatic rollback (or do some hybrid approach). It might be that different platforms get different levels of support with regards to automatic rollback.

[cgwalters]

Tentatively closing this issue; I think ppc64le is in a decent state right now, and the remaining work we can track as dedicated issues.

If we feel a need for coordination, I'd like a new issue that looks like #255 as an explicit tracker.

[ildar]

@cgwalters ,

I think ppc64le is in a decent state right now

I still can't find any image. What is the "decent state" you mention?
Would you please keep this issue open until images are available?