Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openssh: fix static build #100906

Open
wants to merge 2 commits into
base: staging
from
Open

openssh: fix static build #100906

wants to merge 2 commits into from

Conversation

@KAction
Copy link
Contributor

@KAction KAction commented Oct 18, 2020

This pull requests fixes static build of openssh and its dependency,
keyutils:

$ nix-build -A pkgsStatic.openssh

It seems that this is symptom of more general problem: if library "foo"
depends on library "bar", then to link dynamically program that uses
library "foo" it is enough to pass "-lfoo" to compiler, but it is
necessary to pass "-lfoo -lbar" (in that order) to compiler to link it
statically.

Maybe proper fix would be to add -l{bar} into NIX_LDFLAGS for every
{bar} in build closure when static build is requested
(stdenv.hostPlatform.isStatic)?

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@KAction KAction changed the title Openssh openssh: fix static build Oct 18, 2020
@ofborg ofborg bot requested review from edolstra and aneeshusa Oct 18, 2020
#
# ~kaction
+ optionalString stdenv.hostPlatform.isStatic ''
export NIX_LDFLAGS="$NIX_LDFLAGS -lncurses"

This comment has been minimized.

@nh2

nh2 Oct 18, 2020
Contributor

Fixing static builds is great, but this doesn't seem like the right way to do it.

  • NIX_LDFLAGS is a hack that should only be used as last resort. It sneaks flags past the package's build system, thus often making downstream packages not work as expected.
  • pkg-config should be able to figure that out, if used:
    % nix-shell -A openssh
    
    $ pkg-config --libs libedit
    -L/nix/store/sbfj9lhpfi5v9vwrdf9my8l1m7cwbi37-libedit-20191231-3.1/lib -ledit
    
    $ pkg-config --libs libedit --static
    -L/nix/store/sbfj9lhpfi5v9vwrdf9my8l1m7cwbi37-libedit-20191231-3.1/lib -ledit -lncurses

Notice how when --static is passed to pkg-config, it actually emits the required flags of the recursive dependencies.

The way that works is this:

$ echo $PKG_CONFIG_PATH | tr ':' '\n' | grep libedit | head -n1
/nix/store/wx8xws36cz66gabl9sapgaz9p70h7x1m-libedit-20191231-3.1-dev/lib/pkgconfig

$ cat /nix/store/wx8xws36cz66gabl9sapgaz9p70h7x1m-libedit-20191231-3.1-dev/lib/pkgconfig/libedit.pc
...
Libs: -L${libdir} -ledit
Libs.private: -lncurses 
...

Here we can see that the .pc file for libedit "remembers" its non-dynamic dependencies in Libs.private, so that when asked to be linked in statically by a downstream package, it can emit them.

This is pkg-config's direct way to address your observation:

It seems that this is symptom of more general problem: if library "foo"
depends on library "bar", then to link dynamically program that uses
library "foo" it is enough to pass "-lfoo" to compiler, but it is
necessary to pass "-lfoo -lbar" (in that order) to compiler to link it
statically.

So, the right way that this should work is that you tell openssh's build system to link itself statically, based on that it should invoke pkg-config with the --static flag, and then this manual addition of flags should not be necessary.

Clearly somewhere that process breaks down, and the best way forward is to investigate where.

This comment has been minimized.

@aneeshusa

aneeshusa Oct 18, 2020
Contributor

That's a really great explanation, I learned something new today about how pkg-config works. As openssh maintainer I would agree the current state of the PR doesn't look like the right way to solve things but no idea what the right approach is so thanks for stepping in and posting @nh2!

This comment has been minimized.

@KAction

KAction Oct 19, 2020
Author Contributor

@nh2 Thank you for review.

Unfortunately, openssh build system does not understand concept of static build, so I had to fall back on sed. Libedit was quite simple, since it used pkg-config, so I just replaced all calls to pkg-config with pkg-config --static.

Kerberos was harder. Despite library providing pkg-config files, build system uses either own shell code to figure out compiler flags. I added "kerberos.dev" into build inputs, so configure script control flow into the branch which is simpler to patch.

This comment has been minimized.

@nh2

nh2 Oct 19, 2020
Contributor

@KAction Yes, that looks good, great!

Please add what you described here as a comment on top of your seds in the nix file though, so that we keep the rationale next to the code.

Your change here might actually help me with nh2/static-haskell-nix#68 which is nice.

@nh2
Copy link
Contributor

@nh2 nh2 commented Oct 19, 2020

@KAction You'll have to base the PR against staging because it's a mass rebuild (you can do it by rebasing with git and then using Github's Edit button at the top).

@KAction KAction force-pushed the KAction:openssh branch from 8203acd to d57850d Oct 20, 2020
@FRidh FRidh added this to WIP in Staging via automation Oct 20, 2020
@FRidh FRidh moved this from WIP to Needs review in Staging Oct 20, 2020
@FRidh FRidh changed the base branch from master to staging Oct 20, 2020
@KAction KAction force-pushed the KAction:openssh branch from d57850d to 4879ea9 Oct 20, 2020
@KAction
Copy link
Contributor Author

@KAction KAction commented Oct 20, 2020

@nh2 Rebased on staging, @FRidh changes target branch for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Needs review
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.