Compile errors on Linux and macOS

[deeplook]

I'm trying to compile the Unix port on Linux and macOS by doing the following:

git clone https://github.com/micropython/micropython.git
cd micropython
git submodule update --init lib/axtls lib/berkeley-db-1.xx
cd ports/unix
make V=1

But I get this on Linux:

$ make V=1
[...]
Traceback (most recent call last):
  File "../../tools/makemanifest.py", line 252, in <module>
    main()
  File "../../tools/makemanifest.py", line 218, in main
    res, out = system([MPY_CROSS] + mpy_cross_flags.split() + ['-o', outfile, '-s', script, '-O{}'.format(opt), infile])
  File "../../tools/makemanifest.py", line 112, in system
    output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/srv/conda/envs/notebook/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/srv/conda/envs/notebook/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/srv/conda/envs/notebook/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/srv/conda/envs/notebook/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '../../mpy-cross/mpy-cross': '../../mpy-cross/mpy-cross'
../../py/mkrules.mk:103: recipe for target 'build/frozen_content.c' failed
make: *** [build/frozen_content.c] Error 1

$ uname -a
Linux jupyter-binder-2dexamples-2drequirements-2dqzaam6iy 4.14.138+ #1 SMP Tue Sep 3 02:58:08 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux

And this on MacOS 10.14.6:

$ make V=1
[...]
modffi.c:32:10: fatal error: 'ffi.h' file not found
#include <ffi.h>
         ^~~~~~~
1 error generated.
make: *** [build/genhdr/qstr.i.last] Error 1
make: *** Deleting file `build/genhdr/qstr.i.last'

$ uname -a
Darwin ... 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64

I was following the advice from #5207 about cloning the submodules, first... Anything I can do better?

[lsangild]

The MacOS build seems to lack libffi.
Get it by running
brew install libffi

[deeplook]

Seems like I had an up-to-date libffi 3.2.1 already, I've run brew reinstall libffi, but still get the same error.

[rolandvs]

To skip libffi use make MICROPY_PY_FFI=0 or use make minimal. I have the same problem and the extra step was to set the path $export PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig, but now this seems not to work. :-(

[jimmo]

The first error is due to mpy-cross not being available. You need to run make in the mpy-cross directory first.

I'll update #5218 to add a nicer error about this.

[dpgeorge]

I'll update #5218 to add a nicer error about this.

Good idea.

[rolandvs]

@deeplook
What did the trick:

brew install libffi
brew install pkgconfig
export PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig
make 

[deeplook]

Awesome, works for macOS! Thanks! Can we add that to some installation step in some README, please? Let me know where and I could do it... Testing on Linux now...

[deeplook]

Works on Linux, too!

# macOS only prep step
brew install libffi
brew install pkgconfig
export PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig

git clone https://github.com/micropython/micropython.git
cd micropython

git submodule update --init lib/axtls lib/berkeley-db-1.xx

cd mpy-cross
make
cd ..

cd ports/unix/
make 

# smoke test
./micropython -c "print(42)"

[takov751]

I have done the same as in the previous comment which works with x86/x86_64 build,however I have tried to cross compile it ended up with this.

Ubuntu 19.10 
Linux  5.3.0-24-generic #26-Ubuntu SMP Thu Nov 14 01:33:18 UTC 2019 x86_64 GNU/Linux
libffi-dev pkg-config installed
gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf used
target embedded board kernel is 3.x

git clone https://github.com/micropython/micropython.git
cd micropython

git submodule update --init lib/axtls lib/berkeley-db-1.xx

cd mpy-cross
make
cd ..

cd ports/unix/

/micropython/ports/unix$ make V=1 CROSS_COMPILE=arm-linux-gnueabihf- submodules 
Updating submodules: lib/axtls lib/berkeley-db-1.xx lib/libffi
git submodule update --init ../../lib/axtls ../../lib/berkeley-db-1.xx ../../lib/libffi

/micropython/ports/unix$ make CROSS_COMPILE=arm-linux-gnueabihf- MICORPY_PY_FFI=1
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
GEN build/genhdr/qstr.i.last
modffi.c:32:17: fatal error: ffi.h: No such file or directory
 #include <ffi.h>
                 ^
compilation terminated.
make: *** [../../py/mkrules.mk:76: build/genhdr/qstr.i.last] Error 1
make: *** Deleting file 'build/genhdr/qstr.i.last'

[dpgeorge]

however I have tried to cross compile it ended up with this.
...
make CROSS_COMPILE=arm-linux-gnueabihf- MICORPY_PY_FFI=1

Since ffi is failing (because it's not installed for the cross compiler) simply disable it with MICROPY_PY_FFI=0

[takov751]

however I have tried to cross compile it ended up with this.
...
make CROSS_COMPILE=arm-linux-gnueabihf- MICORPY_PY_FFI=1

Since ffi is failing (because it's not installed for the cross compiler) simply disable it with MICROPY_PY_FFI=0

Yes i did. Its works well , however i would like to use modules (like requests, etc) , which does require libffi. Thus this method is not an option. The device i am trying to build for is a Kobo glo hd.To get this level of support i might have to bootstrap a whole system (like debian/alpine linux).
Thank you for your help. I already learnt a lot about cross compiling ,thanks to this project.

[dpgeorge]

You can try building with make MICROPY_STANDALONE=1 MICROPY_PY_FFI=1 which should use the local ffi code in this repo.

[victorromeo]

On Ubuntu 18.04 (x64), I had to install ffi headers,
sudo apt install libffi-dev

After that, I was able to build locally. Cross build support will be my next challenge.

[jonnor]

It seems that solutions have been found for the reported problems. Proposing to close this issue.