feat: add libz.so.1 to whitelisted libraries
#334
Conversation
Codecov Report
@@ Coverage Diff @@
## master #334 +/- ##
==========================================
+ Coverage 89.35% 89.54% +0.18%
==========================================
Files 23 23
Lines 1203 1244 +41
Branches 255 270 +15
==========================================
+ Hits 1075 1114 +39
- Misses 78 79 +1
- Partials 50 51 +1
Continue to review full report at Codecov.
|
mayeut
force-pushed
the
zlib
branch
2 times, most recently
from
3d9e720
to
3845ebc
Compare
This also requires to blacklist some symbols from `libz.so.1`. There was some breakage in ABI compatibility for this library. It's seems unlikely that those symbols are used in the wild.
| section = elf.get_section_by_name(".dynsym") | ||
| if section is not None: | ||
| # look for all undef symbols | ||
| for sym in section.iter_symbols(): | ||
| if sym["st_shndx"] == "SHN_UNDEF": | ||
| undef_symbols.add(sym.name) |
There was a problem hiding this comment.
nit: This could be made into a utility function to find undefined symbols. That would make this function more readable as well.
| @@ -285,12 +293,19 @@ def analyze_wheel_abi(wheel_fn: str) -> WheelAbIInfo: | |||
| sym_tag, | |||
| ucs_tag, | |||
| pyfpe_tag, | |||
| blacklist_tag, | |||
| ) | |||
|
|
|||
|
|
|||
| def update(d, u): | |||
There was a problem hiding this comment.
Unrelated to this PR, but we should refactor this function.
Everything comes from mayeut/pep600_compliance#88 (The same repo was used to define policies for manylinux_2_24 & manylinux_2_27). |
Co-authored-by: László Kiss Kollár <lkollar@users.noreply.github.com>
|
Hi, According to the manylinux policies, libz.so.1 should not be whitelisted. Can we revert this change or at least make it as an opt-in argument so all libraries that use A bit more context on why this is causing problems. Our company runs a minimal Linux runtime to ensure the portability of our code across Linux versions/distros. We have been replying on the manylinux policies to ensure that all pip libraries run on our runtime without missing dependencies. However, this change introduced an external dependency on libz.so.1 (which does not exist in our runtime) into some libraries (huggingface/tokenizers#946, for one example) and prevented us to upgrade to newer versions. |
You're probably talking about the previous PEP 513, PEP 571 & PEP 599. Those 3 PEP have been superseded by the much more generic PEP 600 which does not list allowed / disallowed libraries & which states:
CPython itself relies on zlib for multiple modules (although, if you're using CPython and linking zlib statically, it does not depend on Your company should probably update its minimal runtime package with |
This also requires to blacklist some symbols from
libz.so.1.There was some breakage in ABI compatibility for this library.
It's seems unlikely that those symbols are used in the wild.
Fix #152
Fix #161