howto cross-compile greenlet for PowerPC on x86_64

[m27315]

Hi,

I am encountering a build issue, while trying to cross-compile greenlet 0.4 for a PowerPC arch on x86_64. I have already cross-compiled Python and used it to serve web-pages with bottle on my PowerPC target. My cross-compile build environment is defined by:

# Undo cross-compile environment
unset ROOT
unset SDKDIR
unset KLIBDIR
unset NFSDIR
unset CONFIG
unset CONFIGURED
unset ARCH
unset OS
unset TOOLCHAIN_BASE
unset TOOLCHAIN_BIN
unset CROSS_COMPILE
unset c
unset KERNEL_DIR
unset AS
unset LD
unset CC
unset AR
unset STRIP
unset SSTRIP
unset OBJCOPY
unset OBJDUMP
unset MAKE
unset CFLAGS

#Make the python interpreter for the host
export PATH="/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/powerpc-none-linux-gnuspe/bin:${PATH}"
export AS=${TOOLCHAIN}as
export LD=${TOOLCHAIN}ld
export CC=${TOOLCHAIN}gcc
export CXX=${TOOLCHAIN}g++
export AR=${TOOLCHAIN}ar
export NM=${TOOLCHAIN}nm
export STRIP=${TOOLCHAIN}strip
export SSTRIP=${TOOLCHAIN}sstrip
export RANLIB=${TOOLCHAIN}ranlib
export READELF=${TOOLCHAIN}readelf
export OBJCOPY=${TOOLCHAIN}objcopy
export OBJDUMP=${TOOLCHAIN}objdump
export BLDSHARED="${TOOLCHAIN}gcc -shared"
export LDSHARED="${TOOLCHAIN}gcc -shared"
export RFS="../../ltib/rootfs"
export CFLAGS="-save-temps -Wall -I${RFS}/usr/include -I${RFS}/include/python2.7 -L${RFS}/usr/lib -L${RFS}/lib"
export LDFLAGS="-mcpu=powerpc -I${RFS}/usr/include -I${RFS}/include/python2.7 -L${RFS}/usr/lib -L${RFS}/lib"
export CROSS_COMPILE=ppc-linux
export CROSS_COMPILE_TARGET=yes
export HOSTARCH=ppc-linux
export BUILDARCH=x86_64-linux-gnu

Building in this environment yields:

$ python ./setup.py build
running build
running build_ext
building 'greenlet' extension
creating build
creating build/temp.linux-x86_64-2.7
/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-gcc -save-temps -Wall -I../../ltib/rootfs/usr/include -I../../ltib/rootfs/include/python2.7 -L../../ltib/rootfs/usr/lib -L../../ltib/rootfs/lib -fPIC -I/usr/include/python2.7 -c greenlet.c -o build/temp.linux-x86_64-2.7/greenlet.o
greenlet.s: Assembler messages:
greenlet.s:832: Error: syntax error; found `(' but expected `,'
greenlet.s:832: Error: junk at end of line: `(31),1'
error: command '/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-gcc' failed with exit status 1

I have studied several blogs and articles on the subject:

• http://stackoverflow.com/questions/11587635/cross-compile-of-pythons-greenlet-and-gevent-on-linux-x86-64-for-powerpc
• http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html
• http://kynesim.blogspot.co.uk/2012/06/cross-compiling-python-for-arm-with.html
• http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/
• http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/

But, I am not getting anywhere.

Can greenlet be cross-compiled? If so, how?

Thanks!

Trevor

[m27315]

By adding the -save-temps flag to the compiler, I was able to find the bad assembly code in greenlet.s:

#APP
 # 52 "platform/switch_ppc_linux.h" 1
    mr 8(31), 1
 # 0 "" 2
#NO_APP
...

Is this a problem in greenlet's macro definition, the cross-compiler, or something else?

Thanks!

[schmir]

Is this a problem in greenlet's macro definition, the cross-compiler, or something else?

yes

[m27315]

:-) Ahh, ok, you got me - bad question. :) ... I have managed to assemble greenlet.o by hacking greenlet.s, changing mr 8(31), 1 to mr 8, 1, but I have no idea if that is correct or usable. I am happy to help debug, but I have no idea of what to try next ... Thanks!

[schmir]

Do you have a native compiler to test on? Does it output the same code?
You can try to run the testsuite.

I could try to install a ppc linux on my old mac mini on the attic, but then there are probably better ways to waste my time.

[schmir]

@mpe and @nacc recently added code for ppc64 linux. This also contains the

 __asm__ ("mr %0, 1" : "=g" (stackref) : );

line. This make me think the problem may be your cross-compiler. But maybe they can give you some hints...

[mpe]

It looks like your compiler has put stackref in memory, despite it being declared as a register variable.

Try changing the constraint on the asm to "=r".

[m27315]

@mpe and @schmir : Thanks so much for your help. Somehow I lost the -O2 compiler flag during my debug. Strangely, adding the optimization flag fixed my problem. O_o Well, at least I can build and install the greenlet module. ... Now, I am going to test it. :) ... Thanks!

@mpe: Your suggestion worked too! Do you have any suggestion as to which I should use, CFLAGS += -O2, setting the asm constraint to "=r", or both? ... Thanks!

[m27315]

@schmir and @mpe: What do you think about changing the "=g" constraint on stackref to "=r" in the PPC switch macro? It seems a painless and beneficial patch, which forces stackref to be assigned to a register, which is critical for the MR opcode on PPC. Should I submit a patch?

[mpe]

@m27315 Yes you should submit a patch to change the constraint to =r. You should fix switch_ppc_linux.h too.

[m27315]

Ok, I submitted a pull request for the patch. If I can do anything else, please let me know. BTW, I also forced the stsizediff requirement to register.

        SLP_SAVE_STATE(stackref, stsizediff);
        __asm__ volatile (
            "mr 11, %0\n"
            "add 1, 1, 11\n"
            "add 30, 30, 11\n"
            : /* no outputs */
            : "r" (stsizediff)
            : "11"
            );
        SLP_RESTORE_STATE();

If that seems incorrect, please let me know.

Thanks to @mpe and @schmir for all the help!

[snaury]

It seems correct. If you look at http://class.ee.iastate.edu/cpre211/labs/quickrefPPC.html you will see that mr instruction indeed operates only on registers, so it should be "r", not "g".

[schmir]

many thanks everyone involved!

[m27315]

Great! Thanks! :)