Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared lib builder drops package directory #169

Open
jayvdb opened this issue Nov 2, 2019 · 1 comment · May be fixed by #183
Open

Shared lib builder drops package directory #169

jayvdb opened this issue Nov 2, 2019 · 1 comment · May be fixed by #183
Labels
bug Something isn't working

Comments

@jayvdb
Copy link
Contributor

jayvdb commented Nov 2, 2019

Shared libs that are in packages that also contain python code are stored in site-packages/foo/bar.so

PyOxidizer is dropping the foo package name, creating conflicts.

A very prominent example is markupsafe, simplejson and logbook all have a _speedups.so

>>> import simplejson
    import simplejson as json
  File "simplejson", line 115, in <module>
  File "simplejson.encoder", line 15, in <module>
  File "simplejson.encoder", line 12, in _import_speedups
AttributeError: module 'markupsafe._speedups' has no attribute 'encode_basestring_ascii'

tornado and websockets and fastparquet all have a speedups.so

cytoolz and h5py and espressomd all have utils.so, and further nested are zmq/backend/cython/utils.so and yt/analysis_modules/photon_simulator/utils.so

dulwich and h5py have _objects.so

xxhash and lmdb have cpython.so

bsdiff4 and espeak have core.so

astropy and sunpy have _compiler.so

gevent and kivy both have _event.so

PyQt4 and PyQt5 have an almost identical set of built modules under their respective directories.

Cryptodome and Crypto also share a set of similar names.

Third level objects sklearn/manifold/_utils.so and sklearn/tree/_utils.so probably also have this problem, breaking sklearn even if only it is installed.

Third level objects gssapi/raw/misc.so and samba/dcerpc/misc.so likely also conflict.

Third level astropy/stats/_stats.so and scipy/stats/_stats.so

As noted at #167 , brotli and brotlipy both expect from brotli import _brotli to import their .so , but that example is a bit messy as the two libraries conflict anyway.

And that is just what I can find in my site-packages :P Obviously there will be many more conflicts I am unaware of.

@jayvdb jayvdb mentioned this issue Nov 2, 2019
@indygreg indygreg added the bug Something isn't working label Nov 2, 2019
@jayvdb
Copy link
Contributor Author

jayvdb commented Nov 8, 2019

At least one of the problem is they both have symbols PyInit__speedups, and both are written into pyoxidizer/config.c, so probably one is executed twice.

jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 8, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
@jayvdb jayvdb linked a pull request Nov 8, 2019 that will close this issue
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 8, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 9, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 11, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 14, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 15, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 15, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 16, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 19, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 19, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 19, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 19, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 22, 2019
This causes the .obj files to be unparsable by editbin, dumpbin,
nm, objcopy, goblin and object.

Related to indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 22, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 22, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 26, 2019
This causes the .obj files to be unparsable by editbin, dumpbin,
nm, objcopy, goblin and object.

Related to indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 26, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 29, 2019
This causes the .obj files to be unparsable by editbin, dumpbin,
nm, objcopy, goblin and object.

Related to indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 29, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Nov 30, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Dec 23, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Dec 23, 2019
This causes the .obj files to be unparsable by editbin, dumpbin,
nm, objcopy, goblin and object.

Related to indygreg#169
jayvdb added a commit to jayvdb/PyOxidizer that referenced this issue Dec 23, 2019
Built extensions in packages often have common names
like speedups, utils, _objects, cpython, etc. which
reside inside the package namespace.
The compiled extensions each have a PyInit_<module>
which needs to be renamed to PyInit_<pkg>_<module>
to avoid clashes when combined into a static binary.

Fixes indygreg#169
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants