Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Out of tree" build fails on AIX #54865

Closed
pedz mannequin opened this issue Dec 9, 2010 · 10 comments
Closed

"Out of tree" build fails on AIX #54865

pedz mannequin opened this issue Dec 9, 2010 · 10 comments
Labels
3.7 (EOL) end of life build The build process and cross-build

Comments

@pedz
Copy link
Mannequin

pedz mannequin commented Dec 9, 2010

BPO 10656
Nosy @merwok, @haubi, @vadmium
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • aix-fix-out-of-source.diff: Patch to apply on 2.7 branch
  • issue10656-out-of-source-build-on-aix.patch: patch for 3.4
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-11-21.03:31:23.763>
    created_at = <Date 2010-12-09.03:35:03.276>
    labels = ['build', '3.7']
    title = '"Out of tree" build fails on AIX'
    updated_at = <Date 2018-01-02.15:40:35.893>
    user = 'https://bugs.python.org/pedz'

    bugs.python.org fields:

    activity = <Date 2018-01-02.15:40:35.893>
    actor = 'David.Edelsohn'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-11-21.03:31:23.763>
    closer = 'martin.panter'
    components = ['Build']
    creation = <Date 2010-12-09.03:35:03.276>
    creator = 'pedz'
    dependencies = []
    files = ['21489', '35476']
    hgrepos = ['252']
    issue_num = 10656
    keywords = ['patch']
    message_count = 10.0
    messages = ['123668', '128275', '128315', '132649', '219740', '279474', '279475', '279526', '281196', '281307']
    nosy_count = 8.0
    nosy_names = ['sable', 'eric.araujo', 'haubi', 'pedz', 'python-dev', 'Tristan.Carel', 'martin.panter', 'David.Edelsohn']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'compile error'
    url = 'https://bugs.python.org/issue10656'
    versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

    @pedz
    Copy link
    Mannequin Author

    pedz mannequin commented Dec 9, 2010

    I do "out of tree" builds (I think that is the right term). I make an
    empty directory, cd into it, and then do:

    <path to src directory>/configure

    followed by make. This works on most open source pages. It appears to
    be very close to working with Python 2.7.1 but the problem is that it assumes ld_so_aix is in the source tree but it is in the build tree.

    I did an "in tree" build using basically just ./configure && make and so far (I'm still testing) that has worked fine.

    @pedz pedz mannequin added build The build process and cross-build labels Dec 9, 2010
    @sable
    Copy link
    Mannequin

    sable mannequin commented Feb 10, 2011

    There are some unit tests concerning distutils which are currently failing on AIX and may be an illustration of the same problem:

    ======================================================================
    ERROR: test_build_ext (distutils.tests.test_build_ext.BuildExtTestCase)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/unixccompiler.py", line 254, in link
        self.spawn(linker + ld_args)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/ccompiler.py", line 911, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/spawn.py", line 34, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/spawn.py", line 138, in _spawn_posix
        % (cmd[0], exit_status))
    distutils.errors.DistutilsExecError: command './Modules/ld_so_aix' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/tests/test_build_ext.py", line 85, in test_build_ext
        cmd.run()
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/command/build_ext.py", line 347, in run
        self.build_extensions()
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/command/build_ext.py", line 456, in build_extensions
        self.build_extension(ext)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/command/build_ext.py", line 543, in build_extension
        target_lang=language)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/ccompiler.py", line 719, in link_shared_object
        extra_preargs, extra_postargs, build_temp, target_lang)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/unixccompiler.py", line 256, in link
        raise LinkError(msg)
    distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit status 1

    ======================================================================
    ERROR: test_get_outputs (distutils.tests.test_build_ext.BuildExtTestCase)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/unixccompiler.py", line 254, in link
        self.spawn(linker + ld_args)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/ccompiler.py", line 911, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/spawn.py", line 34, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/spawn.py", line 138, in _spawn_posix
        % (cmd[0], exit_status))
    distutils.errors.DistutilsExecError: command './Modules/ld_so_aix' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/tests/test_build_ext.py", line 343, in test_get_outputs
        cmd.run()
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/command/build_ext.py", line 347, in run
        self.build_extensions()
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/command/build_ext.py", line 456, in build_extensions
        self.build_extension(ext)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/command/build_ext.py", line 543, in build_extension
        target_lang=language)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/ccompiler.py", line 719, in link_shared_object
        extra_preargs, extra_postargs, build_temp, target_lang)
      File "/san_cis/home/cis/.buildbot/python-aix6/py3k-aix6-xlc/build/Lib/distutils/unixccompiler.py", line 256, in link
        raise LinkError(msg)
    distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit status 1

    I will take a look.

    @sable
    Copy link
    Mannequin

    sable mannequin commented Feb 10, 2011

    I tried with trunk (python 3.2) on AIX:
    ./build/configure --with-pydebug --without-computed-gotos
    make
    and that worked correctly.

    Not sure with python 2.7.

    @TristanCarel
    Copy link
    Mannequin

    TristanCarel mannequin commented Mar 31, 2011

    I have experienced the same problem under ppc64 aix 6.1.2.0 while compiling Python 2.7.1

    $ cd Python-2.7.1
    $ mkdir __build
    $ cd __build
    $ ../configure --with-gcc=xlc_r --with-cxx-main=xlC_r --with-threads --disable-ipv6
    $ /usr/linux/bin/make
    [...]
    ../Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L.. -L/opt/freeware/lib -L/opt/freeware/lib build/temp.aix-6.1-2.7/data/tcarel/ngbuilder/python/2.7.1/Python-2.7.1/Modules/_struct.o -L/opt/freeware/lib -L/usr/local/lib -o build/lib.aix-6.1-2.7/_struct.so
    unable to execute ../Modules/ld_so_aix: No such file or directory

    There are references to $(srcdir)/Modules/ld_so_aix in both configure.in and Makefile.pre.in whereas ld_so_aix is created in build tree.

    ==================================================

    diff -r f4e6e18cae4b Makefile.pre.in
    --- a/Makefile.pre.in	Tue Mar 29 17:28:25 2011 -0700
    +++ b/Makefile.pre.in	Thu Mar 31 13:05:45 2011 +0200
    @@ -1028,7 +1028,7 @@
     		$(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix	\
     				$(DESTDIR)$(LIBPL)/makexp_aix;		\
     		echo "$(LIBPL)/makexp_aix";			\
    -		$(INSTALL_SCRIPT) $(srcdir)/Modules/ld_so_aix	\
    +		$(INSTALL_SCRIPT) Modules/ld_so_aix	\
     				$(DESTDIR)$(LIBPL)/ld_so_aix;		\
     		echo "$(LIBPL)/ld_so_aix";			\
     		echo; echo "See Misc/AIX-NOTES for details.";	\
    diff -r f4e6e18cae4b configure.in
    --- a/configure.in	Tue Mar 29 17:28:25 2011 -0700
    +++ b/configure.in	Thu Mar 31 13:05:45 2011 +0200
    @@ -1778,7 +1778,7 @@
     then
     	case $ac_sys_system/$ac_sys_release in
     	AIX*)
    -		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
    +		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp -L\$(srcdir)"
     		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
     		;;
     	BeOS*)

    ==================================================

    After fixing these 2 files, Modules/ld_so_aix is found and properly executed by compilation process, but then there is a new issue:

    Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L.. -L/opt/freeware/lib -L/opt/freeware/lib build/temp.aix-6.1-2.7/data/tcarel/Python-2.7.1/Modules/_struct.o -L/opt/freeware/lib -L/usr/local/lib -o build/lib.aix-6.1-2.7/_struct.so
    Modules/ld_so_aix[183]: Modules/makexp_aix: not found.

    This time, makexp_aix is expected to be is the same directory than ld_so_aix whereas makeexp_aix remains in the source tree.

    Here is a fix:

    ==================================================

    diff -r f4e6e18cae4b Modules/ld_so_aix.in
    --- a/Modules/ld_so_aix.in	Tue Mar 29 17:28:25 2011 -0700
    +++ b/Modules/ld_so_aix.in	Thu Mar 31 13:01:57 2011 +0200
    @@ -69,7 +69,10 @@
       echo $usage; exit 2
     fi
     
    -makexp=`dirname $0`/makexp_aix
    +makexp=`dirname $0`/@srcdir@/makexp_aix
    +if test ! -e "$makexp" ; then
    +  makeexp=makexp_aix
    +fi
     
     # Check for existence of compiler.
     CC=$1; shift

    ==================================================

    The first value fix the issue while building Python. The second will use PATH to find makeexp_aix as explained in section `SHARED LIBRARY SUPPORT' in AIX-NOTES (for building shared extension modules).

    You can find a patch for 2.7 branch in attachment.

    Regards

    Tristan Carel
    Exalead

    @haubi haubi mannequin changed the title "Out of tree" build fails on AIX 5.3 "Out of tree" build fails on AIX Jun 4, 2014
    @haubi
    Copy link
    Mannequin

    haubi mannequin commented Jun 4, 2014

    Basically the same as Tristan's patch, with a little improvement to not rely on PATH to find makexp_aix within ld_so_aix.

    Thanks!

    @vadmium
    Copy link
    Member

    vadmium commented Oct 26, 2016

    The idea of the patches makes sense to me.

    Embedding the name of whatever directory Python was built in into the installed version of Modules/ld_so_aix.in does not seem very clean to me. Is there another way of doing this? Perhaps a file copy or symlink?

    @vadmium vadmium added the 3.7 (EOL) end of life label Oct 26, 2016
    @vadmium
    Copy link
    Member

    vadmium commented Oct 26, 2016

    Actually, why do you remove $(srcdir) for Modules/python.exp? This code was added in r88426; see <https://bugs.python.org/issue941346#msg128586\>.

    @haubi
    Copy link
    Mannequin

    haubi mannequin commented Oct 27, 2016

    Modules/python.exp is generated at build-time, thus does not belong to $(srcdir).

    @vadmium
    Copy link
    Member

    vadmium commented Nov 19, 2016

    Okay I will assume that $(srcdir) reference was just due to a mistake or bad cargo-culting then.

    Will commit this soon unless anyone has a good idea to avoid embedding @abs_srcdir@.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 20, 2016

    New changeset 48526666321a by Martin Panter in branch '3.5':
    Issue bpo-10656: Fix out-of-tree building on AIX
    https://hg.python.org/cpython/rev/48526666321a

    New changeset 76d1f8001e27 by Martin Panter in branch '3.6':
    Issue bpo-10656: Merge AIX build fix from 3.5
    https://hg.python.org/cpython/rev/76d1f8001e27

    New changeset 180f046b597e by Martin Panter in branch 'default':
    Issue bpo-10656: Merge AIX build fix from 3.6
    https://hg.python.org/cpython/rev/180f046b597e

    New changeset ca46883fc5cf by Martin Panter in branch '2.7':
    Issue bpo-10656: Fix out-of-tree building on AIX
    https://hg.python.org/cpython/rev/ca46883fc5cf

    @vadmium vadmium closed this as completed Nov 21, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life build The build process and cross-build
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant