-
Notifications
You must be signed in to change notification settings - Fork 201
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discrepancy between python-tag definitions in Boost.Python and superproject #203
Comments
Indeed, I have modified the |
I suspect, I think, the naming rules for Python-related libraries need to be the same to help downstream projects. As I understand it, the superproject's rule is there for exactly that. If you wish to change the naming scheme, I think it is better to work towards updating the common rule instead of creating a duplicate in Boost.Python. I would urge you to propose your change to the superproject. |
I did propose that. In fact, I asked months ago on the main ML whether anyone had an opinion on the matter (no-one had). But as I said, given that this change requires touching three separate repos, it couldn't be done in one atomic change. |
I don't understand why Boost.MPI needs to change. It has other problems caused by other changes in Boost.Python, see boostorg/mpi#58, but in regard to it using the superproject's |
The point is that moving the |
@aminiussi Could you comment on this? |
Ok, but I'm just out from surgery and not familiar with the Python part. So it will take a couple of days. |
Is this connected to the fact that bjam fails to detect that my python lib and include directory is tagged as python3.6m, not python3.6 ? (I never used it before) |
No. Once that's done, I'll submit a patch upstream to adjust the super-project's Makes sense ? |
Not yet, but it's been 10 years that nothing bjam related made any sense to me :-) Right now, I'm just trying to build boost python against an anaconda 3.6. It uses the Marangozov allocator that the Boost Python build does not handle. But it should start making sense at some point. Thanks for the explanations which should be very useful quite soon I hope. |
@Lastique , to get back to the original report: While I'm happy to contribute my |
Let see if I understand this: we (python related stuff in boost) are moving from a ${major} only library suffix to a ${major}${minor} suffix (which sounds like a good idea) and we need to replicate the work in all python related libs as doing it in the super project would probably break things. Just one question regarding the Marangozov allocator: it seems that Python is moving to the encoding: ${major}${minor}${allocator} where ${allocator} is "" for plain Some distribution links the malloc only distribution to the Marangozov distribution -- but not all of them. The modification discussed here does not seems to address this issue, was it considered and dismissed ? Again, I am not familiar with boost python bindings yet, so maybe this is an trivial issue. Thanks again for the help. |
On 20.05.2018 12:42, Alain Miniussi wrote:
Let see if I understand this: we (python related stuff in boost) are
moving from a ${major} only library suffix to a ${major}${minor}
suffix (which sounds like a good idea) and we need to replicate the
work in all python related libs as doing it in the super project would
probably break things.
Not entirely. Before my recent change we used an ad-hoc convention which
produced `boost_python` (when built with Python 2) and `boost_python3`
(when built with Python 3). Now we are using `boost_python${major}${minor}`.
Just one question regarding the Marangozov allocator: it seems that
Python is moving to the encoding: ${major}${minor}${allocator} where
${allocator} is *""* for plain |malloc| and *m* for Marangozov's
optimized wrapper (it impact the library's name, which moves from
|libpython3.6.(a|so)| to |libpython3.6m.(a|so)| and the include
directory (containing |pyconfig..h|) which move from
|<pythondist>/include/python3.6| to |<pythondist>/include/python3.6m|.
Some distribution links the malloc only distribution to the Marangozov
distribution -- but not all of them.
And I guess they can co-exist (why did they introduce that naming
convention instead of just moving to the Marangozov allocator if it's
better is beyond me).
The modification discussed here does not seems to address this issue,
was it considered and dismissed ?
So IIUC, you are proposing that instead of the `${major}${minor}`
suffix, we use `${major}${minor}${allocator}` ?
The OP requested that we uniformize the `python-tag` use across Boost
libraries (notably `Boost.Python` and `Boost.MPI`, which may in fact be
the only two projects concerned), so what you seem to propose seems to
be (mostly) unrelated to the issue under consideration here. In the
interest of keeping the discussion focussed, would you please submit a
new issue if you are indeed suggesting a suffix change that includes the
allocator ?
(To be honest, I'm not convinced of the value of such a change, as I
have yet to see a platform that has the same Python versions with and
without that allocator. But OK, let's discuss this elsewhere.)
Thanks,
Stefan
…--
...ich hab' noch einen Koffer in Berlin...
|
@stefanseefeld Yes, I'm building Debian packages for my project. The building scripts are based on the official Debian packages and are intended to be compatible. The issue originated from Boost.Python failing to build because its library names have changed compared to 1.66. Indeed, I'm passing Regarding the use of Python, I think you are correct that Boost.Python and Boost.MPI are the only users within Boost. I'm not sure if there are Boost.Build-based users of Python outside Boost, but I wouldn't deny the possibility. There certainly are Boost.Build-based users of Boost out there. |
On 20.05.2018 19:02, Andrey Semashev wrote:
@stefanseefeld <https://github.com/stefanseefeld> Yes, I'm building
Debian packages for my project. The building scripts are based on the
official Debian packages and are intended to be compatible. The issue
originated from Boost.Python failing to build because its library
names have changed compared to 1.66.
Can you elaborate on "failing to build" ? I suppose it's your wrapper
build logic that fails because of it not expecting the new library
names. Right ?
Indeed, I'm passing |--python-buildid| when building Boost.Python and
Boost.MPI, but I don't think I can remove it because this would break
Boost.MPI. Right now I solved the problem locally by reverting
Boost.Python to use the global |python-tag|.
That's sad.
Regarding the use of Python, I think you are correct that Boost.Python
and Boost.MPI are the only users /within Boost/. I'm not sure if there
are Boost.Build-based users of Python /outside Boost/, but I wouldn't
deny the possibility. There certainly are Boost.Build-based users of
Boost out there.
So it seems logical to aim for Boost.MPI also switching to this new
naming convention. Then everyone could simply adjust their expectations
and stop using `--python-buildid`.
But downstream package maintainers reverting my change is rather
unexpected. I had hoped for a more happy embrace. :-)
Stefan
…--
...ich hab' noch einen Koffer in Berlin...
|
Right, exactly. But I don't think the problem is limited to just the packaging. I would expect that the change to also affect any downstream projects that link with Boost.Python - those will need updating, too. |
Yes, of course. That's a feature, not a bug. :-) |
@stefanseefeld I'm all for having whichever boost sonames possible, as long as they are consistent and usable by the reverse dependencies. In Debian & Ubuntu, we used to provide boost-python and boost-python3 sonames, which are symlinks to -py27 & py36 buildid tagged things. The current debian packaging has become incompatible with latest boost releases, simply because well the debian packaging has not bee updated it. The key thing, is in Debian/Ubuntu there is demand for multiple concurrent support of multiple major releases. Currently it's 2.7 & 3.6, soon it will be 2.7/3.6/3.7, and maybe some time later it will become 2.7/3.7. The boost_python3.so symlinks were very convenient as reverse dependencies would link against that to build against the "distro current default python3". I will double check what is the latest sonames advice is for python related sonames, after taking into account everything that was merged so far. Historically, in Debian there have been requests to provide boost_python builds not only for all the currently supported pythons, but also their debug equivalents. But so far I have not even attempted to make that. Historically, I also believe Debian started to provide 2 & 3 builds in parallel, and we somewhat had to make up the names ourselves, which is unfortunate. It would be really nice if the upstream boost build system would compile for python2 and python3 simultaneously and in parallel, with whatever non-conflicting sonames you want to call the libraries as long as they are kind of stable and predictable. There are about 73 reverse binary packages dependencies on boost-python in Debian/Ubuntu, 20 of which use python3. It would be nice if everything or most things would switch to python3 by now. Anyway, time for me to check all the latest greatest things in master & devel branches. |
@xnox, I don't quite understand what you are saying. I'm fully aware that my recent naming changes are not backward compatible. However, I don't see any problem setting up symlinks from In addition, I think an |
@stefanseefeld ack, I'll check to adjust debian packaging to that effect. I think we have been using buildid argument in lieu of python-tag, and should simply stop doing that in debian/ubuntu. |
@stefanseefeld As merged by @aminiussi
Similar config is generated for 3.6 and 3.7 When building above with:
The results are a bit unexpected:
I was expecting instead It's as if something is missing from |
I think i figured it out.... i've ensure that python-tag things are added in the mpi Jamfile.... but also carbon copy&pasted |
Correct ! Make sure your autolink logic is adjusted accordingly. |
In Boost 1.67, Boost.Python uses its own definition of
python-tag
rule inlibs/python/Jamfile
. This rule produces library names likelibboost_python36-py36.so.1.67.0
, notice the 36 afterboost_python
. This is different from Boost 1.66 and from the similar rule defined in the superproject, inboostcpp.jam
.In particular, this results in different naming between Boost.Python and Boost.MPI Python binding libraries. I'm not sure why the duplicate Python version is needed, so this issue requests to port Boost.Python to the superproject definition of the rule.
The text was updated successfully, but these errors were encountered: