Define conventions for shared / static builds

[davydden]

subj. Roughly that +shared enables shared libs, whereas ~shared implies that static-only libs will be built.

Most likely we cant really enforce +shared not to have static libs, as some packages may build them anyway.

from @alalazo

What all the other packages do is to build both shared and static libraries when +shared and just static when ~shared.

While here, also document +pic variant for static libs only and that +pic+shared should throw an error.

[citibeth]

See: https://groups.google.com/forum/#!topic/spack/gNAp1CmhxF0

[citibeth]

@adamjstewart

This is a continuation of #2478... I think we're getting ahead of ourselves here. Let's go back to use cases, of which I see two:

1. You work in a shared library environment. You want to build everything with +shared. Things that can't do shared libs, you want to do +pic (since you intend to link them with shared libs).

2. You work on a Cray and need to build things static. Now you want to build everything with +static, and you don't care about PIC stuff.

Case (1) should be the default. But case (2) should be as easy as possible for Cray users to turn on, and to use consistently.

Are there any other use cases here? If not, we need only one variant, which I would call +static (since shared libs are the default these days). If you turn on +static, you will get libraries suitable for use in a static linking environment. If you don't turn it on, you will get libraries suitable for use in a shared linking environment. You might get extra libraries in either case, which you won't really care about.

With this in place, there should be a way for Cray users to put +static in their packages.yaml to turn it on everywhere. Packages that can only produce shared libs would become an error on Cray.

[Many users THINK they want static linking because it's easier to mv the binary around, between machines, etc. A better approach is to make a mvexe script that uses ldd to identify the shared libraries that also need to be moved; collect them all together in a single directory; and then rewrite RPATHs accordingly.]

[davydden]

If not, we need only one variant, which I would call +static (since shared libs are the default these days). If you turn on +static, you will get libraries suitable for use in a static linking environment. If you don't turn it on, you will get libraries suitable for use in a shared linking environment.

what about pic, i presume it's on by default on all static libs, right? I does not really hurt much so i would say we don't have to cary around extra variant.

p.s. I don't have a strong opinion on how to call a single variant to control shared/static, namely shared with +shared default, or static with ~static default.

[alalazo]

@citibeth If I understand your proposal correctly, i.e.:

• there's only one variant that controls shared/static
• if in shared mode pic is assumed if you can only build static libraries

then I fully agree with it.

I don't see why we may want to use:

variant('static', default=False)

instead of

variant('shared', default=True)

as there are 32 package that now adopt the opposite convention vs. 0 that adopt this one, but if people like the renaming I won't oppose.

If you think of an additional pic variant on top of the one above, then I don't agree because of either +shared~pic or ~static~pic.

@davydden

what about pic, i presume it's on by default on all static libs, right?

Right now I think it doesn't work like that. Slightly ot: does anybody know any architecture which enters in "Spack range" and don't permit pic code?

[citibeth]

No, pic is off for static. You only need it if you plan to link with dynamic libs.

…

On Dec 6, 2016 5:00 AM, "Massimiliano Culpo" ***@***.***> wrote: @citibeth <https://github.com/citibeth> If I understand your proposal correctly, i.e.: - there's only one variant that controls shared/static - if in shared mode pic is assumed if you can only build static libraries then I fully agree with it. I don't see why we may want to use: variant('static', default=False) instead of variant('shared', default=True) as there are 32 package that now adopt the opposite convention vs. 0 that adopt this one, but if people like the renaming I won't oppose. If you think of an additional pic variant on top of the one above, then I don't agree because of either +shared~pic or ~static~pic. @davydden <https://github.com/davydden> what about pic, i presume it's on by default on all static libs, right? Right now I think it doesn't work like that. Slightly ot: does anybody know any architecture which enters in "Spack range" and don't permit pic code? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2380 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB1cd-87u7RHalk3CDLXthslehw_YzH7ks5rFTI4gaJpZM4K5gYF> .

[alalazo]

No, pic is off for static. You only need it if you plan to link with dynamic libs.

Sorry, I don't get to what part of the comment you are replying...

Anyhow, I am fine as long as we keep only one variant. If we want to go with static then:

• +static means build libraries that can be linked statically
• ~static means build libraries that can be linked dynamically (so either shared libraries or static with pic)

If we want to maintain the current convention just substitute +static with ~shared above and vice-versa.

[citibeth]

h

I don't see why we may want to use: variant('static', default=False) instead of variant('shared', default=True)

In 1990, static libs were standard and people were just beginning to add shared libs to their builds. '+shared' made sense from that point of view. Today shared libs are standard, so it makes sense to acknowledge that by using '+static' for the nonstandard build.

Right now I think it doesn't work like that.

That is right. Under my scheme, pic would only be used if you don't ask for '+static' and the build is unable to make a shared lib. So instead it will make a static lib that can be linked with shared. It codifies and simplifies the way '+pic' is currently used.

[davydden]

It looks like we all agree to use a single variant, so it only a matter of taste how to name it: shared or static. @tgamblin ping

[citibeth]

Sorry, I disagree after a night's sleep; see #2492

…

On Tue, Dec 6, 2016 at 6:41 AM, Denis Davydov ***@***.***> wrote: It looks like we all agree to use a single variant, so it only a matter of taste how to name it: shared or static. @tgamblin <https://github.com/tgamblin> ping — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2380 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB1cd-ddty8EuHCTq7RfQdVoCJ0UTKFjks5rFUnfgaJpZM4K5gYF> .

[citibeth]

So... we've had a lot of talk about Universal Variants (#2492) and gneralized Variant Forwarding (#2594), and recurring issues with R.

But putting all that aside... I think #2644 will get us what we need here in a simple, understandable way. If we can also get #2386 merged, then the final proposal looks like this:

1. Package authors implement an exclusive multi-valued variant link-type=[shared, static] to control whether the library is intended to be linked in shared or static builds. It should default to link-type=shared in the package.

2. With run with link-type=static, the package must produce static (.a) libraries. It might or might not produce shared librares as well. (If it does, they are vestigal).

3. When run with link-type=shared, the package must produce either (a)shared libraries, or (b) static libraries built with the PIC flag. If (a), it might also produce static libraries (with or without PIC); as in (2) above, these will be vestigal.

4. Users building executables with shared linkage must build with link-type=shared. Users building executables with static linkage must build with link-type=static. Users building both must build the required packages twice. (NOTE: In the case of packages that can build shared + static simultaneouly, this would result in a bit of redundant building. But it greatly simplifies the logic for everyone involved).

5. In order to specify whether they need shared or static builds, users will typically set link-type under the all section in packages.yaml (see Allow setting default variants #2644). Spack will ship with a packages.yaml that sets link-type=shared by default, and link-type=static by default on Cray.

6. Package authors should never try to forward the link-type variant using either top-down or bottom-up forwarding. Setting it globally in packages.yaml will be the normal way to do things.

7. If a package builds only shared libraries, it can optionally omit the link-type variant. On a machine that doesn't support shared libraries, this could make it silently fail if one tries to build with all: link-type=static. But by definition, such a build was never going to succeed anyway. Better would be to include the link-type variant, but only allow it to be set to shared; this will cause illegal builds to "fail fast" during concretization.

8. Packages that don't produce shared or static libraries (eg, most Python/R packages) can ignore all this.

Thoughts? Once #2386 is merged, this is eminently doable. Or should we go ahead without #2386 (@tgamblin)? I've heard that Cray requires static linking; are there any Cray users that could review this proposal?

One thing this proposal doesn't really support well is people who want to use static linking for part of their build and shared for other parts; for example, build common stuff shared, but app-specific stuff static. Is this something people want to do, that we should support?

[citibeth]

This discussion is now obsolete, closing.