Building wheel hangs on ppc64le

[auspicious3000]

When installing praat-parselmouth using pip install praat-parselmouth inside a conda environment on a power9 ppc64le system. The installation always hangs at this step

Building wheels for collected packages: praat-parselmouth
Building wheel for praat-parselmouth (pyproject.toml) ...

[YannickJadoul]

@auspicious3000, I've never built Parselmouth on ppc64le, I think, and I don't know if Praat would actually work/compile there.

It would be good to know exactly where things are hanging, though. Could you run pip install with the -v verbose flag, and post the output?

If I find time over the next holiday, I'll see if I can try building myself into ppc64le (but I will first need to figure out how to do so on my x86-64).

[YannickJadoul]

@auspicious3000, alright, I have managed to build Parselmouth on ppc64le (using qemu on x86-64), which somehow went surprising easily with some Docker-magic.

So, it would seem that it did work for me (installing gcc, cmake, and python3-dev on the ppc64le/ubuntu Docker image on qemu), even though it took a long while because I was emulating the architecture.
There is one test which does not pass, but it is related to some weird floating-point inaccuracy. I haven't figured out why yet, but as the rest passes, it would seem that you can use Parselmouth on pcc64le.

So, I would ask you to try again, but run pip install -v praat-parselmouth. This will give you extra output and give you some sense of progress. The reason might just be that Parselmouth is being built. As I have currently not created and released any ppc64le pre-built binary wheels, it takes a while to compile the whole of Parselmouth, which is why you might be thinking the installation process halts?

[auspicious3000]

@YannickJadoul Thanks for looking into this. You are correct. It is just compiling the package. After waiting for a couple of hours, the compiling crashed at the last step. Due to the long outputs, I am sending it to your email instead.

I actually have successfully installed praat-parselmouth on some pcc64le servers last year. However, the process is kind of fragile, changing a few random things such as the conda version and pip version could make completely different stories. It just worked due to pure luck. This year, the admin probably upgraded the server, which could contribute to the compiling crash.

I have also tried installing version 0.4.0, 0.4.1. They all crashed near the end but with different error messages.

[YannickJadoul]

@auspicious3000, alright, that's already good news, that it's not hanging. Too bad it's so slow, but OK

But I'm still surprised/confused about the error message you sent me by email. The actual compilation seems to work (based on the first few lines, which are the last ones of the compilation), but something seems to be going wrong with scikit-build/setuptools/distutils. Which is slightly weird, because ... well, it worked for me. I'll look further, but perhaps getting the start of the log build log (where these packages get temporarily installed) would be useful.

[YannickJadoul]

Ah, good news (I think, at least?): I managed to reproduce on my computer. For future reference, here is how:

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
$ docker run --rm -it -v ~/Parselmouth_ppc64le:/Parselmouth quay.io/pypa/manylinux2014_ppc64le

And then inside that Docker:

[root@ede090716e30 /]# git clone --recursive --depth 1 --branch v0.4.2 https://github.com/YannickJadoul/Parselmouth.git
[root@da4982ef2c02 /]# cd Parselmouth/
[root@da4982ef2c02 Parselmouth]# export MAKEFLAGS="-j16"
[root@da4982ef2c02 Parselmouth]# /opt/python/cp37-cp37m/bin/python -m pip -v wheel --no-deps .
...

I'm attaching the full log output here as file, but it seems to be the same error, @auspicious3000?
ppc64le_error.txt

[YannickJadoul]

OK, I found the issue. The combination of the pinned version of scikit-build==0.12 and the latest release of setuptools breaks (see CI test run here).

I've committed fb2ecfe which should fix this, and I'll try building again tomorrow.

[YannickJadoul]

@auspicious3000, meanwhile, if you already want to install a cp37 wheel, here is one I built, while figuring out what went wrong (zipped, because GitHub won't let me attach .whl files to a message): praat_parselmouth-0.4.2-cp37-cp37m-linux_ppc64le.zip

[auspicious3000]

@YannickJadoul This works! I didn't expect this could be solved so soon. Thank you very much for your time and effort in digging into this!

[YannickJadoul]

@auspicious3000, thanks for confirming!

I just tested, and pip install git+https://github.com/YannickJadoul/Parselmouth.git@stable now works, but I think I'll just make a new release 0.4.3. I hope soon I'll have 0.5.0 ready (and maybe I can just have ppc64le wheels pre-built), but until then, it seems rather annoying to not have a source distribution that successfully builds.

[YannickJadoul]

So, I've just uploaded (0.4.3)[https://pypi.org/project/praat-parselmouth/0.4.3/]. It still doesn't contain ppc64le wheels, but at least the source distribution now compiles! And hopefully, I can ensure pre-compiled ppc64le wheels in 0.5!

Thanks again for notifying and sending the debug log!