Skip to content

Conversation

aixtools
Copy link
Contributor

@aixtools aixtools commented Apr 28, 2020

Make the build utility makexp_aix able to deal with parallel build make (-jNN)
Patch by M. Felt.

https://bugs.python.org/issue40424

@aixtools
Copy link
Contributor Author

p.s. I did not bother with a NEWS blurb - not really related to Python (sic)

@zware zware added the skip news label Jun 1, 2020
@aixtools
Copy link
Contributor Author

aixtools commented Jun 2, 2020

OOps - had not seen skip news today - so added something.

Shall I delete it?

@aixtools
Copy link
Contributor Author

Just thought I'd mention - most of the bot failures, such as: https://buildbot.python.org/all/#builders/227/builds/1150 are caused because makexp_aix was not written (in 1998 and 1999) for parallel processing.

This leads to:

...
./install-sh -c -m 644 ./Tools/gdb/libpython.py python-gdb.py
gcc -pthread -c -Wno-unused-result -Wsign-compare -g -Og -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include    -DPy_BUILD_CORE -o Programs/_testembed.o ./Programs/_testembed.c
sed -e "s,@EXENAME@,/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/target/bin/python3.10d," < ./Misc/python-config.in >python-config.py
LC_ALL=C sed -e 's,\$(\([A-Za-z0-9_]*\)),\$\{\1\},g' < Misc/python-config.sh >python-config
gcc -pthread -c -Wno-unused-result -Wsign-compare -g -Og -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include    -DPy_BUILD_CORE \
      -DGITVERSION="\"`LC_ALL=C git --git-dir ./.git rev-parse --short HEAD`\"" \
      -DGITTAG="\"`LC_ALL=C git --git-dir ./.git describe --all --always --dirty`\"" \
      -DGITBRANCH="\"`LC_ALL=C git --git-dir ./.git name-rev --name-only HEAD`\"" \
      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
rm -f libpython3.10d.a
ar rcs libpython3.10d.a Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/token.o  Parser/pegen/pegen.o Parser/pegen/parse.o Parser/pegen/parse_string.o Parser/pegen/peg_api.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genericaliasobject.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/hamt.o Python/hashtable.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o    Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/_operator.o  Modules/_collectionsmodule.o  Modules/_abc.o  Modules/itertoolsmodule.o  Modules/atexitmodule.o  Modules/signalmodule.o  Modules/_stat.o  Modules/timemodule.o  Modules/_threadmodule.o  Modules/_localemodule.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/faulthandler.o  Modules/_tracemalloc.o  Modules/_peg_parser.o  Modules/symtablemodule.o  Modules/xxsubtype.o Python/frozen.o
./Modules/makexp_aix Modules/python.exp . libpython3.10d.a;  gcc -pthread     -Wl,-bE:Modules/python.exp -lld -o python Programs/python.o libpython3.10d.a -lintl -ldl  -lm   -lm 
./Modules/makexp_aix Modules/python.exp . libpython3.10d.a;  gcc -pthread     -Wl,-bE:Modules/python.exp -lld -o Programs/_testembed Programs/_testembed.o libpython3.10d.a -lintl -ldl  -lm   -lm 
ld: 0711-418 ERROR: Import or export file Modules/python.exp at line 2:
	A symbol name may only be followed by an export attribute
	or an address. The line is being ignored.
ld: 0711-415 WARNING: Symbol PyAST_Check is already exported.
ld: 0711-415 WARNING: Symbol PyAST_Compile is already exported.
...

Specifically, the lines:

./Modules/makexp_aix Modules/python.exp . libpython3.10d.a;  gcc -pthread     -Wl,-bE:Modules/python.exp -lld -o python Programs/python.o libpython3.10d.a -lintl -ldl  -lm   -lm 
./Modules/makexp_aix Modules/python.exp . libpython3.10d.a;  gcc -pthread     -Wl,-bE:Modules/python.exp -lld -o Programs/_testembed Programs/_testembed.o libpython3.10d.a -lintl -ldl  -lm   -lm 

cause the file Modules/python.exp to be created in parallel - and besides frequently causing a build crash - it is also causing about 22k lines of warnings of symbols already exported.

This PR is to minimize, if not eliminate, both issues.

Your consideration for a merge is appreciated.

@kadler
Copy link
Contributor

kadler commented Nov 14, 2020

LGTM but to be fair, I helped write it 😁

@aixtools
Copy link
Contributor Author

Quite right. And a different form is already added, so closing.

@aixtools aixtools closed this Nov 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants