Shared libraries need versions

[merkys]

Building OpenMM produces the following public shared libraries without versions in their sonames:

• libOpenMMAmoeba.so
• libOpenMMRPMD.so
• libOpenMM.so
• libOpenMMDrude.so

Without sonames the shared libraries cannot be represented in the shlibs system, and if linked by binaries its interface cannot safely change. Therefore no backward-compatible way exists to migrate programs linked against them to a new binary interface.

I have found this issue while packaging OpenMM for Debian.

[jchodera]

@peastman : This seems like a good idea.

[samuelflores]

@peastman : Yes please! I would love to be able to distribute the full stack including molmodel and MMB.

https://github.com/peastman

[mbanck]

Building OpenMM produces the following public shared libraries without versions in their sonames:

* libOpenMMAmoeba.so
* libOpenMMRPMD.so
* libOpenMMDrude.so

Those three might be plugins (they seem to link with -ldl?) which are not supposed to be linked to by third party applications? If that is indeed the case, another option would be to install them in a private path, like /usr/lib/openmm/.

That would still mean libOpenMM.so should get a library version, unfortunately CMake does not make that very straight-forward.

[peastman]

All the libraries at the top level of lib are ones that other programs can link to directly. The plugins, which get loaded with dlopen, are in the plugins subdirectory.

[samuelflores]

Hi Peter, Thanks -- I am forwarding your response to Andrius and Andreas. Sam

…

On 29 Mar 2019, at 19:49, peastman ***@***.***> wrote: All the libraries at the top level of lib are ones that other programs can link to directly. The plugins, which get loaded with dlopen, are in the plugins subdirectory. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2281 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGHOPVq_EUa28xAzHIhZOcdSL2kyczIPks5vbmAcgaJpZM4boJ5->.

[jchodera]

All the libraries at the top level of lib are ones that other programs can link to directly. The plugins, which get loaded with dlopen, are in the plugins subdirectory.

@peastman : What happens if one wants to install two different versions of OpenMM at the system level? How will plugin libraries from different versions avoid clobbering each other?

[peastman]

If you want to install two different versions, you install them in different locations. Since conda is our preferred installation mechanism these days, that happens automatically. Each conda environment gets its own plugin directory.

[jchodera]

We're talking about packaging for Debian in this thread:

I have found this issue while packaging OpenMM for Debian.

conda doesn't care what they're named---thinks will get modified to work. But if there's a chance to make things also easily work with Linux package managers, that sounds like an easy win.

[peastman]

I feel like I've missed part of the conversation. Why are we talking about it? We've standardized on conda as our recommended installation mechanism. Are there situations where that doesn't work? If so, what are they? What exactly are the problems we're trying to solve? We need to understand that, then consider all possible solutions. Are they problems that only occur on Debian? If not, then a Debian-specific solution won't meet our requirements.

Adding another supported installation mechanism isn't something to do lightly, since it adds a major ongoing support burden.

[samuelflores]

including andrius and andreas.. they are trying to package for debian. no idea if conda is compatible with that.

…

Sent from my iPhone

On Apr 1, 2019, at 18:08, peastman ***@***.***> wrote: I feel like I've missed part of the conversation. Why are we talking about it? We've standardized on conda as our recommended installation mechanism. Are there situations where that doesn't work? If so, what are they? What exactly are the problems we're trying to solve? We need to understand that, then consider all possible solutions. Are they problems that only occur on Debian? If not, then a Debian-specific solution won't meet our requirements. Adding another supported installation mechanism isn't something to do lightly, since it adds a major ongoing support burden. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[merkys]

soname is UNIX-standard (not limited to Debian, but including all Linux and Mac systems) parameter of a shared library, denoting its name and ABI version. These are required by the software that links against the shared library to memorize what ABI version was used during the linking. Once an ABI of the shared library changes, the linked software has no way to check that. More explicit description of the system is given in this Stack Overflow thread.

[jchodera]

I feel like I've missed part of the conversation. Why are we talking about it?

@peastman: Some very nice folks have offered to package up OpenMM for the debian packaging system. On Linux, as you well know, packaging systems (like yum and dpkg) are the primary means that many sysadmins (and users with sysadmin rights or friends) bring software onto their systems, so it represents a major route by which software is installed and upgraded. Other MD simulation packages like gromacs, for example are distributed this way as one route to installation.

These very nice folks have observed that, while the default CMake installation scheme installs OpenMM in system library paths, it doesn't follow standard conventions regarding what ABI version was installed.

With some very simple changes to how CMake names these libraries, we can follow these standard conventions and allow these very nice people to package OpenMM for us, allowing Linux users who don't use conda to easily install OpenMM.

We've standardized on conda as our recommended installation mechanism. Are there situations where that doesn't work? If so, what are they?

Linux users who don't use conda would benefit from this scheme.

What exactly are the problems we're trying to solve?

Right now, we're trying to solve the problem that OpenMM doesn't conform to standard practices for how libraries should be named.

We need to understand that, then consider all possible solutions. Are they problems that only occur on Debian? If not, then a Debian-specific solution won't meet our requirements.
Adding another supported installation mechanism isn't something to do lightly, since it adds a major ongoing support burden.

Is there a significant support burden to complying with UNIX standard practices in library naming? It would seem to be straightforward and simple, and enable these very nice folks (and others!) to easily package OpenMM for Linux package managers.

[peastman]

Is there a significant support burden to complying with UNIX standard practices in library naming? It would seem to be straightforward and simple, and enable these very nice folks (and others!) to easily package OpenMM for Linux package managers.

I'm not talking about library naming. I'm talking about the major support burden that comes from having these nice people create Debian packages for OpenMM. Because they'll create the packages, but I'm the one who will get emailed every time someone encounters an error trying to install them! Then a year from now, the people who created the packages will move on and no one will be creating new packages, but the old ones will still be there. And five years from now I'll still be getting bug reports from people who installed OpenMM with the Debian tools and got an ancient version.

[jchodera]

It sounds like there are two distinct issues here:

1. We may or may not want anyone to create Debian packages for OpenMM
2. We don't conform to the soname standard

I think (1) could be addressed if there are folks willing to assume the support responsibility for creating the Debian packages.

(2) is something that we should address irrespective of how (1) is addressed.

[samuelflores]

Hi Peter and John, Regarding issue 1, I personally would not care if the OpenMM package in Debian were not updated regularly. The pieces of OpenMM I use are pretty ancient, mostly I just want the neighborlisting function which is super fast. Maybe the burden of updating could be left to some future interested party. Sam

…

On 2 Apr 2019, at 20:30, John Chodera ***@***.***> wrote: It sounds like there are two distinct issues here: We may or may not want anyone to create Debian packages for OpenMM We don't conform to the soname standard <https://en.wikipedia.org/wiki/Soname> I think (1) could be addressed if there are folks willing to assume the support responsibility for creating the Debian packages. (2) is something that we should address irrespective of how (1) is addressed. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2281 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGHOPdVzNfCo33t1qJctrul85zeJ53uJks5vc6HRgaJpZM4boJ5->.

[peastman]

(2) is something that we should address irrespective of how (1) is addressed.

Feel free to submit a PR.

Maybe the burden of updating could be left to some future interested party.

"Some future interested party" means me. If Debian packages get created and there's no long term commitment for someone else to support them, that automatically means that either I'll have to maintain them, or else I'll have to bear the burden of lots of people running archaic versions of OpenMM.

[merkys]

@jchodera correctly identified two separate issues in this discussion. I'll try to answer questions regarding Debian (from my personal experience):

• Sustainability. Most of the packages in Debian are maintained by so-called teams, thus even if individuals get "missing in action", teams keep updating the packages and fixing bugs in them.

• Debian-specific bugs. Any bugs originating from Debian packages should be forwarded to Debian to be fixed there.

• Alternative ways of distributing the software. Should Debian packages happen to be deemed buggy or archaic at any point in time, there is always an option of conda. So Debian packages must not be viewed as a replacement of it.

I am interested of packaging OpenMM as it is a dependency of MacroMolecule Builder.

[peastman]

We decided against versioned libraries. They're incompatible with PyPI packaging, we never used them for anything, and they were never fully implemented. I removed them in #4498.