Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

manylinux2014 images #348

Merged
merged 1 commit into from Nov 10, 2019
Merged

manylinux2014 images #348

merged 1 commit into from Nov 10, 2019

Conversation

@mayeut
Copy link
Contributor

mayeut commented Sep 22, 2019

1st attempt at a manylinux2014 image
main differences with the 2010 image:

  • drop python 2.7 (EOL in january 2020) & 3.4 (already EOL)
  • use system provided openssl for python build
  • cmake is not installed anymore (one can use pip install cmake)
  • add aarch64 support
  • i686 uses default gcc 4.8.5 & misses yasm

c.f. #338

Please do comment

@mayeut mayeut mentioned this pull request Sep 22, 2019
@brainwane brainwane mentioned this pull request Sep 22, 2019
17 of 25 tasks complete
@lpsinger lpsinger mentioned this pull request Sep 28, 2019
@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from 507b7c5 to c55f024 Sep 29, 2019
@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Sep 30, 2019

Can 'make' be called with multiple jobs? Would make build faster.

@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Sep 30, 2019

Build on aarch64 fails:

+ for PYTHON in '/opt/python/*/bin/python'
+ /opt/python/cp35-cp35m/bin/python build_scripts/manylinux-check.py
/opt/python/cp35-cp35m/bin/python is NOT manylinux2010 compatible
The command '/bin/sh -c bash build_scripts/build.sh && rm -r build_scripts' returned a non-zero code: 1

Sure, it is manylinux2010 not compatible but only because this architecture did not existed at that time.

docker/build_scripts/build.sh Outdated Show resolved Hide resolved
@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Oct 3, 2019

So, how it will be processed? I am waiting for it to happen and then will discuss with several projects to get them to publish wheel files for AArch64 architecture.

It will allow me to cut build times from 45 minutes to seconds...

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch 3 times, most recently from 2e6fff6 to 79955a0 Oct 3, 2019
@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Oct 7, 2019

Travis CI got aarch64 support. There will be more projects ready to use manylinux2014 image once it get published.

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from 0de2594 to 79955a0 Oct 7, 2019
@auvipy

This comment has been minimized.

Copy link
Contributor

auvipy commented Oct 8, 2019

is it still WIP?

@mayeut

This comment has been minimized.

Copy link
Contributor Author

mayeut commented Oct 8, 2019

@auvipy, yes still a work in progress. Needs at least an auditwheel update (not 2.1.1), README update

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch 2 times, most recently from 9fb92f6 to 8049f83 Oct 8, 2019
@zhaofeng-shu33

This comment has been minimized.

Copy link

zhaofeng-shu33 commented Oct 12, 2019

@auvipy, yes still a work in progress. Needs at least an auditwheel update (not 2.1.1), README update

Can I use this docker image to upload wheel tagged with 2014 to pypi.org ?

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch 2 times, most recently from dcdee0f to 671e936 Oct 14, 2019
@veblush

This comment has been minimized.

Copy link
Contributor

veblush commented Oct 18, 2019

If possible, how about adding i686 to manylinux2014 docker images? It's one of architectures listed in manylinux2014 policy and there is a base CentOS 7 docker image.

@mayeut

This comment has been minimized.

Copy link
Contributor Author

mayeut commented Oct 18, 2019

@veblush, I can't get a recent toolchain on i686, the one you found doesn't exist for centos 7. RHEL provides this for RHEL7 but not centos.
For now, an i686 image would use gcc 4.8.5 (or else someone has to figure out a way to rebuild/store devtoolset-8 from SRPM in a reliable/trusted way).
I've rebuilt devtoolset-8 for centos6-i686 using copr (might do a PR with that) but they do not provide an el7-i686 chroot to do the same for centos7. We could do an image with cross-compilation but https://bugs.centos.org/view.php?id=16181 prevents this.

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch 2 times, most recently from 2079382 to 3d430c5 Oct 18, 2019
@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Oct 20, 2019

Is there any todo list before it will be considered done?

@veblush

This comment has been minimized.

Copy link
Contributor

veblush commented Oct 21, 2019

@mayeut Thank you for the detailed background! Sorry to know i686 doesn't get enough attention from Redhat and community.

@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Oct 21, 2019

@veblush Red Hat decided to stop supporting i686 when RHEL7 development started. There is no new server class hardware with this architecture and users of "legacy" ones were supported with RHEL6.

There is minimal support for i686 just to keep 32-bit applications running on 64-bit hardware.

In CentOS 7 32-bit x86 is supported as one of "alternative" architectures - check http://mirror.centos.org/altarch/7/ for what is provided. And even there is looks like abandoned architecture.

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from 8c02f21 to 5344833 Oct 23, 2019
Copy link
Contributor

hrw left a comment

23:24 - will look at problem tomorrow

docker/build_scripts/build.sh Outdated Show resolved Hide resolved
Copy link
Contributor

hrw left a comment

one more

docker/build_scripts/build.sh Outdated Show resolved Hide resolved
@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from 5344833 to 96f202d Oct 23, 2019
@auvipy
auvipy approved these changes Oct 24, 2019
@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch 2 times, most recently from f26511d to 87a2c5a Oct 24, 2019
@mayeut

This comment has been minimized.

Copy link
Contributor Author

mayeut commented Oct 25, 2019

@hrw, thanks for all your reviews. While the build passes locally, there's an issue with travis-ci (seems docker related) that I can't explain so I'm going to try a few things...

@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Oct 25, 2019

@hrw, thanks for all your reviews.

I work on aarch64 for 7+ years now so I am trying to help wherever I can.

While the build passes locally, there's an issue with travis-ci (seems docker related) that I can't explain so I'm going to try a few things...

No idea why it fails.

@mayeut

This comment has been minimized.

Copy link
Contributor Author

mayeut commented Oct 25, 2019

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch 2 times, most recently from 87a2c5a to 08e534d Oct 26, 2019
@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from d03c44e to 08e534d Nov 4, 2019
@di

This comment has been minimized.

Copy link
Member

di commented Nov 5, 2019

@mayeut Thanks for all your work here. It looks like the Travis issues are resolved, and that this is blocked on a 3.0.0 release of auditwheel (pypa/auditwheel#205), correct?

@mayeut mayeut mentioned this pull request Nov 5, 2019
@mayeut

This comment has been minimized.

Copy link
Contributor Author

mayeut commented Nov 5, 2019

Correct. There's still one PR opened in auditwheel that needs review and be merged.

@hrw

This comment has been minimized.

Copy link
Contributor

hrw commented Nov 6, 2019

And then will be time to go through all python projects and asking them to make more wheels.

@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from aeabd30 to 5880290 Nov 8, 2019
@mayeut mayeut changed the title WIP - manylinux2014 manylinux2014 images Nov 8, 2019
@auvipy
auvipy approved these changes Nov 9, 2019
@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from 5880290 to c90bda2 Nov 9, 2019
README.rst Show resolved Hide resolved
@auvipy
auvipy approved these changes Nov 9, 2019
@mayeut mayeut changed the base branch from master to manylinux2014 Nov 10, 2019
@mayeut mayeut force-pushed the mayeut:manylinux2014-pr branch from c90bda2 to 3206394 Nov 10, 2019
@auvipy auvipy merged commit 03edd41 into pypa:manylinux2014 Nov 10, 2019
1 check was pending
1 check was pending
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
@mayeut mayeut deleted the mayeut:manylinux2014-pr branch Nov 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.