Foreign library versioning #4074

abooij · 2016-11-01T23:50:10Z

FOREIGNLIBFIELDS
add senity checks for this: for now only supported on linux
ADDSONAME
INSTALLSONAME
document new field and code
write a test
implement new ideas in Specify version (soname) for foreign-library #4052

mention-bot · 2016-11-01T23:50:11Z

@abooij, thanks for your PR! By analyzing the history of the files in this pull request, we identified @edsko, @dcoutts and @phadej to be potential reviewers.

abooij · 2016-11-04T16:35:27Z

@ezyang The code is now ready for review. What's missing is documentation: I'll write something in Cabal/doc/developing-packages.rst. Anywhere else? Is the code documented clearly?

After review I can clean up the history a little bit. Or just make it one big commit?

ezyang · 2016-11-05T05:42:17Z

You're not done until there's documentation ;) I think something nestled with the existing foreign library docs would make sense.

ezyang

After having finally read up on how libtool versioning works, I'm a bit torn about adopting it wholesale for Cabal.

The good thing is that it is "standard".

The bad things... well, it's hella confusing. Here's a quip from https://autotools.io/libtool/version.html

To set the version of the library, libtool provides the -version-info parameter, which accepts three numbers, separated by colons, that are called respectively, current, revision and age. Both their name and their behaviour, nowadays, have to be considered fully arbitrary, as the explanation provided in the official documentation is confusing to say the least, and can be, in some cases, considered completely wrong.

It continues:

The main reason for which libtool uses this scheme for version information, is that it allows to have multiple version of a given library installed, with both link editor and dynamic linker choosing the latest available at build and run time. With modern distributions packaging standards, this situation should not be happening anyway.

TBH, I'm not even sure how libtool style version numbers help in this case.

More fun:

Here is a C library maintainer getting confused by libtool's versioning scheme, and doing something very silly to fix it: https://sourceforge.net/p/dar/bugs/152/ (and some fallout on Debian https://lists.debian.org/debian-devel/2012/05/msg00378.html )
Here is a mailing list post about how all of Ports is misusing libtool versioning (and getting X+Y major versions): https://lists.freebsd.org/pipermail/freebsd-ports/2014-May/092082.html
Here's a project that accidentally bumped age rather than revision when they did updates: http://markmail.org/message/oo6gudzuleotubyx
Here's an SO answer, where libtool's behavior on other non-Linux platforms is.... just not right at all! http://stackoverflow.com/questions/15215898/why-is-libtools-current-used-as-soversion-on-bsd-rather-than-major If libtool's special versioning scheme is meant as an abstraction layer to make it work on other platforms... they are doing a really bad job. (BTW, OS X behavior seems to coincide with Linux: http://lists.apple.com/archives/darwin-dev/2005/Apr/msg00063.html )

So.... I am seriously tempted to ditch libtool numbers, and just use the good, old, simple, "here is the version number that will get put in the SONAME", and tell users to update the major if they break ABI-compatibility, and minor if they add a new interface. (Honestly, we should probably have an option to compute the patchlevel automatically from the Cabal version.) As a comparison, CMake parses out major.minor from the soname and uses that on Windows: https://cmake.org/cmake/help/cmake2.6docs.html#prop_tgt:SOVERSION

But I am open to being convinced.

ezyang · 2016-11-05T05:44:58Z

Cabal/Distribution/Simple/GHC.hs

+      when (isJust (foreignLibELFVersion flib)) $ do
+          let (Platform _ os) = hostPlatform lbi
+          when (os /= Linux) $ die
+            "Can't install foreign-library symlink on non-Linux OS"


Is it true that it should be impossible to get here (the checks during configure should catch it?)

Yes. I can add a comment to that effect.

ezyang · 2016-11-05T05:46:49Z

Cabal/Distribution/Simple/GHC.hs

+#ifndef mingw32_HOST_OS
+          -- createSymbolicLink file1 file2 creates a symbolic link
+          -- named file2 which points to the file file1.
+          createSymbolicLink name (dstDir </> flibBuildName lbi flib)


NB: it's name, not dst, because the symlink will be relative to the directory it's created in, i.e. distDir. Tricky!

I will add a comment to that effect.

ezyang · 2016-11-05T05:53:49Z

Cabal/Distribution/Simple/GHC.hs

+    (Linux, ForeignLibNativeShared)
+  = case foreignLibELFVersion flib of
+      Nothing -> "lib" ++ nm <.> "so"
+      Just v  -> "lib" ++ nm <.> "so" <.> show (head (versionNumbers v))


I am very glad you got rid of this 'head' subsequently ;)

While I agree, it couldn't have resulted in errors, since a user can only set v to a non-null version.

ezyang · 2016-11-05T05:57:01Z

Cabal/Distribution/Types/ForeignLib.hs

@@ -42,16 +53,81 @@ data ForeignLib = ForeignLib {
    }
    deriving (Generic, Show, Read, Eq, Typeable, Data)

+data LibVersionInfo = LibVersionInfo Int Int Int deriving (Data, Eq, Generic, Typeable)


This needs a comment (esp https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html). And although in the standard format the parameters are positional, it would be far better if we had some accessors here to make it clear.)

I will refer to libtool more clearly here.
I can also add these accessors. There is also libVersionInfoCRA which gives you a tuple of the three numbers, but I can see how that is not sufficient and/or clear enough.

ezyang · 2016-11-05T05:59:43Z

Cabal/Distribution/Simple/GHC.hs

@@ -878,20 +878,20 @@ flibTargetName lbi flib =
    platformOS :: Platform -> OS
    platformOS (Platform _arch os) = os

-    -- If a foreign lib foo has elf-version 3.2.1, it should be built
+    -- If a foreign lib foo has lib-version-info 5:1:2, it should be built
    -- as libfoo.so.3.2.1


(This comment is pretty confusing if you're not fluent in libtool's versioning scheme)

I will add [an appropriate reference to] an explanation.

abooij · 2016-11-05T16:47:24Z

@ezyang Thanks for your review.

So.... I am seriously tempted to ditch libtool numbers, and just use the good, old, simple, "here is the version number that will get put in the SONAME", and tell users to update the major if they break ABI-compatibility, and minor if they add a new interface. [...] But I am open to being convinced.

Well, the advantage of version-info over version numbers is that it is more likely we will be able to parse version-info into the data required for non-Linux OSes. For example, we can't reliably deduce an age from major.minor.build (as people will just be randomly bumping version numbers). But it is important that we know both "current" and "current-age" on OSX. So while I agree with your worries about version-info, if there is any potential future wish for OSX support, this is the more general solution.

I can add documentation on how to set these values, if that'd help (basically repeating the three golden rules of version-info).

Honestly, we should probably have an option to compute the patchlevel automatically from the Cabal version.

No! ABI versions are NOT equal to package versions, and this is already an important distinction made in C. This does cause problems. And besides, in my use case this is absolutely undesirable.

ezyang · 2016-11-05T18:04:25Z

No! ABI versions are NOT equal to package versions, and this is already an important distinction made in C. This does cause problems. And besides, in my use case this is absolutely undesirable.

Even the revision level? According to the libtool guidance, you are supposed to bump revision whenever there is a source code change. I guess, maybe not if the C library didn't change (even though the hs library did)?

abooij · 2016-11-05T18:57:47Z

(Also as discussed on IRC:)
For example: there's no reason why the ABI changes with every release of the package. So then there's no reason to even bump the "revision" field. And one might want to make a major change to the haskell code without modifying the ABI (in which case you would update the package version but not the ABI version), or vice versa (in which case you would increase the major version of the ABI, but only make an incremental version bump for the Haskell package). ABI versions are, to some extent, independent of release versions.

We need to build the library under a different name to set the SONAME, and can't set it directly, since GHC overrides -soname linker options.

When a foreign-library has a version, during installation, we need to install appropriate symlinks. This is very platform-dependent.

Libtool sets library version data by the -version-info flag, which takes current[:revision[:age]] data. This is then parsed into a major.minor.build version number. We copy this approach so that library versioning may be generalised to other operating systems than Linux.

ezyang · 2016-11-26T07:43:54Z

This looks reasonable.

abooij · 2016-11-26T10:15:54Z

I wrote some documentation. So this work now seems ready. Final feedback welcome.

ezyang · 2016-11-26T18:14:50Z

Ship it!

abooij force-pushed the foreign-soname branch 12 times, most recently from 5841300 to 2fad4d2 Compare November 4, 2016 13:58

ezyang reviewed Nov 5, 2016

View reviewed changes

abooij mentioned this pull request Nov 5, 2016

Specify version (soname) for foreign-library #4052

Closed

abooij added 5 commits November 23, 2016 22:59

Introduce elf-version field for foreign libraries

ea37978

Make new elf-version field Linux-only

db38c03

During linking, set SONAME for foreign libraries

594f35d

We need to build the library under a different name to set the SONAME, and can't set it directly, since GHC overrides -soname linker options.

Install symlinks for versioned foreign-libraries

b1a4a81

When a foreign-library has a version, during installation, we need to install appropriate symlinks. This is very platform-dependent.

Test that foreign libraries are built with the right version data

6862fcf

abooij force-pushed the foreign-soname branch from 2fad4d2 to 06a36ea Compare November 23, 2016 23:00

abooij force-pushed the foreign-soname branch from 06a36ea to c2a20f4 Compare November 24, 2016 08:36

Reintroduce field to set version of foreign library directly

3ad9421

Document version fields for foreign libraries [ci skip]

bdbd834

abooij force-pushed the foreign-soname branch from 4b071dc to bdbd834 Compare November 26, 2016 10:14

abooij changed the title ~~[WIP] Foreign library versioning~~ Foreign library versioning Nov 26, 2016

abooij merged commit 8744e30 into haskell:master Nov 26, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Foreign library versioning #4074

Foreign library versioning #4074

abooij commented Nov 1, 2016 •

edited

Loading

mention-bot commented Nov 1, 2016

abooij commented Nov 4, 2016

ezyang commented Nov 5, 2016

ezyang left a comment

ezyang Nov 5, 2016

abooij Nov 5, 2016

ezyang Nov 5, 2016

abooij Nov 5, 2016

ezyang Nov 5, 2016

abooij Nov 5, 2016

ezyang Nov 5, 2016

abooij Nov 5, 2016

ezyang Nov 5, 2016

abooij Nov 5, 2016

abooij commented Nov 5, 2016

ezyang commented Nov 5, 2016

abooij commented Nov 5, 2016

ezyang commented Nov 26, 2016

abooij commented Nov 26, 2016

ezyang commented Nov 26, 2016

Foreign library versioning #4074

Foreign library versioning #4074

Conversation

abooij commented Nov 1, 2016 • edited Loading

mention-bot commented Nov 1, 2016

abooij commented Nov 4, 2016

ezyang commented Nov 5, 2016

ezyang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abooij commented Nov 5, 2016

ezyang commented Nov 5, 2016

abooij commented Nov 5, 2016

ezyang commented Nov 26, 2016

abooij commented Nov 26, 2016

ezyang commented Nov 26, 2016

abooij commented Nov 1, 2016 •

edited

Loading