Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Building numpy on AIX with gcc and gfortran #6609

Closed
vinhdizzo opened this issue Nov 2, 2015 · 6 comments
Closed

Building numpy on AIX with gcc and gfortran #6609

vinhdizzo opened this issue Nov 2, 2015 · 6 comments
Labels
57 - Close? Issues which may be closable unless discussion continued component: build

Comments

@vinhdizzo
Copy link

Hi,

I have python 2.7.5 installed via binaries provided here. I also installed gcc and gfortran from the same site. I got OpenBlas compiled and Cython compiled as well.

When I go to compile numpy, I get:

CC=gcc python setup.py build --fcompiler=gnu95 -j 12 install --prefix
/my/path
## c and fortran files compile until
ar: adding 5 object files to build/temp.aix-6.1-2.7/libnpysort.a
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'numpy.core._dummy' extension
compiling C sources
C compiler: gcc -g -DNDEBUG -O2

compile options: '-DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.aix-6.1-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/opt/freeware/include/python2.7 -Ibuild/src.aix-6.1-2.7/numpy/core/src/private -Ibuild/src.aix-6.1-2.7/numpy/core/src/private -Ibuild/src.aix-6.1-2.7/numpy/core/src/private -c'
gcc: numpy/core/src/dummymodule.c
./Modules/ld_so_aix xlc_r -ma -I/opt/freeware/include -DAIX_GENUINE_CPLUSCPLUS -Wl,-brtl -bI:./Modules/python.exp -L/opt/freeware/lib -lbsd -lXext build/temp.aix-6.1-2.7/numpy/core/src/dummymodule.o -Lbuild/temp.aix-6.1-2.7 -lm -o build/lib.aix-6.1-2.7/numpy/core/_dummy.so
sh: ./Modules/ld_so_aix:  not found
sh: ./Modules/ld_so_aix:  not found
error: Command "./Modules/ld_so_aix xlc_r -ma -I/opt/freeware/include -DAIX_GENUINE_CPLUSCPLUS -Wl,-brtl -bI:./Modules/python.exp -L/opt/freeware/lib -lbsd -lXext build/temp.aix-6.1-2.7/numpy/core/src/dummymodule.o -Lbuild/temp.aix-6.1-2.7 -lm -o build/lib.aix-6.1-2.7/numpy/core/_dummy.so" failed with exit status 127

Why is it trying to use xlc_r when we've already specified gcc and gfortran?

Second question: how can I link my compiled OpenBLAS for use with numpy?

Thanks.
@juliantaylor
Copy link
Contributor

try also setting LDSHARED to gcc

@juliantaylor
Copy link
Contributor

for openblas, see the site.cfg.example in the source

@vinhdizzo
Copy link
Author

If I add the LDSHARED=gcc and create site.cfg as

[openblas]
libraries = openblas
library_dirs = /sas/outmva/opt/lib
include_dirs = /sas/outmva/opt/include
runtime_library_dirs = /sas/outmva/opt/lib

Re-compiling, I get:

$ LDSHARED=gcc CC=gcc python setup.py build --fcompiler=gnu95 -j 12 install --prefix /sas/outmva/opt
Running from numpy source directory.
Cythonizing sources
numpy/random/mtrand/mtrand.pyx has not changed
/opt/freeware/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in ['/opt/freeware/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

openblas_info:
/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py:635: UserWarning: Specified path  is invalid.
  warnings.warn('Specified path %s is invalid.' % d)
  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/sas/outmva/opt/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/sas/outmva/opt/lib']

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/sas/outmva/opt/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/sas/outmva/opt/lib']

/bin/sh: svnversion:  not found
/bin/sh: svnversion:  not found
F2PY Version 2
lapack_opt_info:
openblas_lapack_info:
C compiler: cc

creating /tmp/tmpOl311o/tmp
creating /tmp/tmpOl311o/tmp/tmpOl311o
compile options: '-c'
cc: /tmp/tmpOl311o/source.c
sh: cc:  not found
sh: cc:  not found
Traceback (most recent call last):
  File "setup.py", line 260, in <module>
    setup_package()
  File "setup.py", line 252, in setup_package
    setup(**metadata)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/core.py", line 135, in setup
    config = configuration()
  File "setup.py", line 156, in configuration
    config.add_subpackage('numpy')
  File "/sas/data04/vinh/numpy-master/numpy/distutils/misc_util.py", line 1001, in add_subpackage
    caller_level = 2)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/misc_util.py", line 970, in get_subpackage
    caller_level = caller_level + 1)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/misc_util.py", line 907, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "numpy/setup.py", line 16, in configuration
    config.add_subpackage('linalg')
  File "/sas/data04/vinh/numpy-master/numpy/distutils/misc_util.py", line 1001, in add_subpackage
    caller_level = 2)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/misc_util.py", line 970, in get_subpackage
    caller_level = caller_level + 1)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/misc_util.py", line 907, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "numpy/linalg/setup.py", line 26, in configuration
    lapack_info = get_info('lapack_opt', 0) # and {}
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 372, in get_info
    return cl().get_info(notfound_action)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 562, in get_info
    self.calc_info()
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 1491, in calc_info
    openblas_info = get_info('openblas_lapack')
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 372, in get_info
    return cl().get_info(notfound_action)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 562, in get_info
    self.calc_info()
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 1706, in calc_info
    if not self.check_embedded_lapack(info):
  File "/sas/data04/vinh/numpy-master/numpy/distutils/system_info.py", line 1735, in check_embedded_lapack
    obj = c.compile([src], output_dir=tmpdir)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/ccompiler.py", line 231, in CCompiler_compile
    single_compile(o)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/ccompiler.py", line 195, in single_compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/sas/data04/vinh/numpy-master/numpy/distutils/unixccompiler.py", line 52, in UnixCCompiler__compile
    raise CompileError(msg)
distutils.errors.CompileError: Command "cc -c /tmp/tmpOl311o/source.c -o /tmp/tmpOl311o/tmp/tmpOl311o/source.o" failed with exit status 127

I believe this is caused by site.cfg. FYI, this was my success message after I compiled OpenBLAS:

OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)

  OS               ... AIX
  Architecture     ... power
  BINARY           ... 64bit
  C compiler       ... GCC  (command line : gcc)
  Fortran compiler ... GFORTRAN  (command line : gfortran)
  Library Name     ... libopenblas_power5p-r0.2.15.a
 (Multi threaded; Max num-threads is 2)

To install the library, you can run "make PREFIX=/path/to/your/installation install".

If I remove site.cfg and re-run, I get

### Warning:  Using unoptimized lapack ###
  adding 'numpy/linalg/umath_linalg.c.src' to sources.
  adding 'numpy/linalg/lapack_lite/python_xerbla.c' to sources.
  adding 'numpy/linalg/lapack_lite/zlapack_lite.c' to sources.
  adding 'numpy/linalg/lapack_lite/dlapack_lite.c' to sources.
  adding 'numpy/linalg/lapack_lite/blas_lite.c' to sources.
  adding 'numpy/linalg/lapack_lite/dlamch.c' to sources.
  adding 'numpy/linalg/lapack_lite/f2c_lite.c' to sources.
building extension "numpy.random.mtrand" sources
building data_files sources
build_src: building npy-pkg config files
running build_py
copying numpy/version.py -> build/lib.aix-6.1-2.7/numpy
copying build/src.aix-6.1-2.7/numpy/__config__.py -> build/lib.aix-6.1-2.7/numpy
copying build/src.aix-6.1-2.7/numpy/distutils/__config__.py -> build/lib.aix-6.1-2.7/numpy/distutils
running build_clib
customize UnixCCompiler
customize UnixCCompiler using build_clib
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'numpy.core._dummy' extension
compiling C sources
C compiler: gcc -g -DNDEBUG -O2

compile options: '-DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.aix-6.1-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/opt/freeware/include/python2.7 -Ibuild/src.aix-6.1-2.7/numpy/core/src/private -Ibuild/src.aix-6.1-2.7/numpy/core/src/private -Ibuild/src.aix-6.1-2.7/numpy/core/src/private -c'
gcc: numpy/core/src/dummymodule.c
gcc build/temp.aix-6.1-2.7/numpy/core/src/dummymodule.o -Lbuild/temp.aix-6.1-2.7 -lm -o build/lib.aix-6.1-2.7/numpy/core/_dummy.so
ld: 0711-317 ERROR: Undefined symbol: .main
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
collect2: error: ld returned 8 exit status
ld: 0711-317 ERROR: Undefined symbol: .main
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
collect2: error: ld returned 8 exit status
error: Command "gcc build/temp.aix-6.1-2.7/numpy/core/src/dummymodule.o -Lbuild/temp.aix-6.1-2.7 -lm -o build/lib.aix-6.1-2.7/numpy/core/_dummy.so" failed with exit status 1

So I think 2 errors going on here. Any thoughts? Thank you.

@rgommers
Copy link
Member

The last report on gh-5474 had:

export 'CC=gcc -lgcc'
export LDSHARED="/opt/freeware/lib/python2.7/config/ld_so_aix gcc -bI:/opt/freeware/lib/python2.7/config/python.exp"
export CFLAGS="-pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -fno-tree-dominator-opts"

doing something sensible with gcc on AIX 6.1

@mattip mattip added the 57 - Close? Issues which may be closable unless discussion continued label Jul 18, 2018
@mattip
Copy link
Member

mattip commented Jul 18, 2018

I think we can close this, we have reports of NumPy building on AIX

@mattip
Copy link
Member

mattip commented Aug 21, 2018

Closing, please open a new issue if new problems with AIX arise.

@mattip mattip closed this as completed Aug 21, 2018
@aixtools aixtools mentioned this issue Mar 22, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
57 - Close? Issues which may be closable unless discussion continued component: build
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@charris @rgommers @vinhdizzo @juliantaylor @mattip