Put libraries in $WHEELNAME.libs #90

lpsinger · 2018-02-07T06:32:07Z

Should fix #89.

lpsinger · 2018-02-07T06:45:02Z

Oh, wow. That actually worked. Now it just needs a unit test.

lpsinger · 2018-02-07T15:14:42Z

@njsmith, this works on all of the existing unit tests, and I can see that the libraries are getting placed where we expected. Do you think it is actually necessary to add a unit test of a wheel with multiple C extensions?

If you have a package that contains multiple C extensions that depend on the same shared libraries, then `delocate-wheel` saves duplicate copies of all of the shared libraries. An example is shown below. This makes for packages that are several times larger than necessary. This patch causes `delocate-wheel` to sweep up all of the dependencies and put them in a single directory, rather than creating duplicates. The convention is based on the proposal in pypa/auditwheel#89 and pypa/auditwheel#90.

If you have a package that contains multiple C extensions that depend on the same shared libraries, then `delocate-wheel` saves duplicate copies of all of the shared libraries. This makes for packages that are several times larger than necessary. This patch causes `delocate-wheel` to sweep up all of the dependencies and put them in a single directory, rather than creating duplicates. The convention is based on the proposal in pypa/auditwheel#89 and pypa/auditwheel#90.

ehashman

I think it is necessary to add a test to ensure the behaviour works right now and in the future when changes are made. Please do add one.

daa · 2018-12-12T19:22:52Z

I can make a pull request with unit test to @lpsinger branch but it has to be merged with pypa/auditwheel master, or I can make such pull request directly to pypa/auditwheel but it must be merged only after this. Which way would be better? @ehashman @lpsinger

lpsinger · 2018-12-13T11:40:19Z

@daa you can actually make a pull request relative to my branch.

daa · 2018-12-13T13:02:49Z

@lpsinger @ehashman please take a look at test when you'll have time for it.

lpsinger · 2018-12-13T16:54:57Z

@lpsinger @ehashman please take a look at test when you'll have time for it.

Will do. I'm a little slammed right now but I should get to it by the weekend. Thank you!

daa · 2018-12-19T19:18:16Z

I haven't noticed at the time that lib/ is ignored and thus required test files were not added to git, will fix it shortly. Sorry.
New pull request with fix: https://github.com/lpsinger/auditwheel/pull/2 please take a look when you'll have some time.

daa · 2018-12-19T19:44:44Z

And one more fix: https://github.com/lpsinger/auditwheel/pull/3

ehashman · 2019-01-23T23:55:47Z

This is a pretty big semantic change so I'm going to hold off until after the 2.0 release to review.

Tests have been added

If you have a package that contains multiple C extensions that depend on the same shared libraries, then `delocate-wheel` saves duplicate copies of all of the shared libraries. This makes for packages that are several times larger than necessary. This patch causes `delocate-wheel` to sweep up all of the dependencies and put them in a single directory, rather than creating duplicates. The convention is based on the proposal in pypa/auditwheel#89 and pypa/auditwheel#90.

codecov · 2019-07-26T06:43:23Z

Codecov Report

Merging #90 into master will decrease coverage by 0.12%.
The diff coverage is 75%.

@@            Coverage Diff             @@
##           master      #90      +/-   ##
==========================================
- Coverage   87.32%   87.19%   -0.13%     
==========================================
  Files          19       19              
  Lines         978      976       -2     
  Branches      215      214       -1     
==========================================
- Hits          854      851       -3     
  Misses         86       86              
- Partials       38       39       +1

Impacted Files	Coverage Δ
auditwheel/repair.py	`83.52% <75%> (-1.53%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0500098...4d9e3df. Read the comment docs.

lpsinger · 2019-07-26T08:22:51Z

I've rebased the patch and it's passing the tests.

auditwheel/repair.py

lpsinger · 2020-01-21T04:53:33Z

I've rebased this PR again. Is there any further feedback, or is this ready to be merged?

auvipy · 2020-01-21T07:54:21Z

can you check te builds?

lpsinger · 2020-01-21T13:47:36Z

can you check te builds?

Yes, one of the jobs failed here:

INFO     test_manylinux:test_manylinux.py:108 docker exec 06c4e504c87a: 'yum install -y atlas atlas-devel'
278Loaded plugins: fastestmirror, ovl
279Determining fastest mirrors
280https://download.sinenomine.net/clefos/7/sclo/s390x/rh/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
281Trying other mirror.
282

Would you please click retry in travis?

lkollar · 2020-01-21T22:20:59Z

I restarted the build but now it's failing because of another issue:

auditwheel repair: error: argument --plat: invalid choice: 'manylinux1' (choose from 'linux_x86_64', 'manylinux1_x86_64', 'manylinux2010_x86_64', 'manylinux2014_x86_64')

For some reason it's passing the wrong platform name.

lpsinger · 2020-01-21T23:30:37Z

Oh, weird. They're all failing now, too.

lpsinger · 2020-01-22T03:41:31Z

@lkollar, I triggered a build on master and it is failing with the same error.

lkollar · 2020-01-22T07:26:08Z

Looks like the pip 20 release is causing the failures because it deprecates tags like cp3-none-any. Opened #219 to fix it.

Fixes #89.

lpsinger · 2020-01-23T14:50:56Z

That worked. Builds are passing now. @auvipy, @lkollar, any more feedback?

lkollar · 2020-01-23T22:26:48Z

Thanks @lpsinger!

lpsinger · 2020-01-23T22:29:34Z

Thanks @lpsinger!

Well, thank you! It'll be great to have this in a release so that we don't have to use my fork any more for building lalsuite.

daa · 2020-01-24T12:52:07Z

Thank you all!

See: - https://github.com/pypa/auditwheel/releases/tag/3.1.0 - pypa/auditwheel#90

…211) See: - https://github.com/pypa/auditwheel/releases/tag/3.1.0 - pypa/auditwheel#90

lpsinger changed the title ~~WIP: Put libraries in $WHEELNAME.libs~~ Put libraries in $WHEELNAME.libs Feb 7, 2018

lpsinger mentioned this pull request Feb 7, 2018

Install one copy of libs used by multiple extension modules matthew-brett/delocate#36

Closed

louhibi mentioned this pull request Sep 26, 2018

fix internal dependencies of libraries #101

Closed

ehashman previously requested changes Nov 18, 2018

View reviewed changes

auvipy approved these changes Jul 26, 2019

View reviewed changes

lkollar reviewed Jul 26, 2019

View reviewed changes

auditwheel/repair.py Outdated Show resolved Hide resolved

Put libraries in $WHEELNAME.libs

4d9e3df

Fixes #89.

lkollar approved these changes Jan 23, 2020

View reviewed changes

lkollar merged commit f0b52b3 into pypa:master Jan 23, 2020

lpsinger deleted the fix-duplicate-libs branch January 23, 2020 22:28

niess mentioned this pull request Mar 30, 2020

RPATH not properly set when using -L option #232

Closed

dhermes pushed a commit to dhermes/bezier that referenced this pull request Apr 8, 2020

Referring to bezier.libs/ instead of bezier/.libs/.

1b79ce7

See: - https://github.com/pypa/auditwheel/releases/tag/3.1.0 - pypa/auditwheel#90

dhermes mentioned this pull request Apr 8, 2020

Update docs/python/binary-extension.rst to refer to bezier.libs. dhermes/bezier#211

Merged

dhermes added a commit to dhermes/bezier that referenced this pull request Apr 8, 2020

Update docs/python/binary-extension.rst to refer to bezier.libs. (#…

e450fbf

…211) See: - https://github.com/pypa/auditwheel/releases/tag/3.1.0 - pypa/auditwheel#90

McSinyx mentioned this pull request May 25, 2020

Handle C extensions not inside a package directory (at the root) matthew-brett/delocate#39

Closed

mattip mentioned this pull request Jun 21, 2020

Native libs at wrong path in Linux wheels numpy/numpy#16655

Closed

minrk mentioned this pull request Aug 31, 2020

Accessing the bundled libzmq.so zeromq/pyzmq#1412

Open

mayeut mentioned this pull request Apr 3, 2021

Auditwheel does not update all library dependencies when a single wheel contains multiple top-level extensions #132

Closed

alecandido mentioned this pull request Sep 22, 2022

Symbols not found #396

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Put libraries in $WHEELNAME.libs #90

Put libraries in $WHEELNAME.libs #90

lpsinger commented Feb 7, 2018

lpsinger commented Feb 7, 2018

lpsinger commented Feb 7, 2018

ehashman left a comment

daa commented Dec 12, 2018

lpsinger commented Dec 13, 2018

daa commented Dec 13, 2018

lpsinger commented Dec 13, 2018

daa commented Dec 19, 2018 •

edited

Loading

daa commented Dec 19, 2018

ehashman commented Jan 23, 2019

codecov bot commented Jul 26, 2019 •

edited

Loading

lpsinger commented Jul 26, 2019

lpsinger commented Jan 21, 2020

auvipy commented Jan 21, 2020

lpsinger commented Jan 21, 2020

lkollar commented Jan 21, 2020

lpsinger commented Jan 21, 2020

lpsinger commented Jan 22, 2020

lkollar commented Jan 22, 2020

lpsinger commented Jan 23, 2020

lkollar commented Jan 23, 2020

lpsinger commented Jan 23, 2020

daa commented Jan 24, 2020

Put libraries in $WHEELNAME.libs #90

Put libraries in $WHEELNAME.libs #90

Conversation

lpsinger commented Feb 7, 2018

lpsinger commented Feb 7, 2018

lpsinger commented Feb 7, 2018

ehashman left a comment

Choose a reason for hiding this comment

daa commented Dec 12, 2018

lpsinger commented Dec 13, 2018

daa commented Dec 13, 2018

lpsinger commented Dec 13, 2018

daa commented Dec 19, 2018 • edited Loading

daa commented Dec 19, 2018

ehashman commented Jan 23, 2019

codecov bot commented Jul 26, 2019 • edited Loading

Codecov Report

lpsinger commented Jul 26, 2019

lpsinger commented Jan 21, 2020

auvipy commented Jan 21, 2020

lpsinger commented Jan 21, 2020

lkollar commented Jan 21, 2020

lpsinger commented Jan 21, 2020

lpsinger commented Jan 22, 2020

lkollar commented Jan 22, 2020

lpsinger commented Jan 23, 2020

lkollar commented Jan 23, 2020

lpsinger commented Jan 23, 2020

daa commented Jan 24, 2020

daa commented Dec 19, 2018 •

edited

Loading

codecov bot commented Jul 26, 2019 •

edited

Loading