Make Linux package universal.

[TravisEz13]

PR Summary

Make Linux package universal.

• Add packaging build for Linux
• Consolidate to two universal packages one for RedHat and one for deb
• Use post install and removal scripts to deal with symbolic links

NON-GOAL: Improve libmi support.

PR Context

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • None
  • OR
  • This is breaking but cannot be experimental.
  • Experimental feature(s) needed
    • Experimental feature name(s):
• User-facing changes
  • Not Applicable
  • OR
  • Documentation needed
    • Issue filed: Document use of universal packages MicrosoftDocs/PowerShell-Docs#7387
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
    • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
      (which runs in a different PS Host).
      • Issue filed:
    • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
      • Issue filed:
    • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
      • Issue filed:
    • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
      • Issue filed:

[ghost]

🎉v7.2.0-preview.5 has been released which incorporates this pull request.:tada:

Handy links:

• Release Notes

[DHowett]

As a package maintainer, I'd like to lodge my dissent with making files (or symlinks, or permission changes) in postinst scripts. They can't be properly removed, the package manager doesn't know about them, and generally they circumvent the OS's mechanism for handling packaged data. 😄

[iSazonov]

Perhaps @heaths has thoughts how improve this.

[TravisEz13]

@DHowett Please open an issue. This objection needs to be reviewed. As these symlinks are for a feature that eventually will no longer work, my proposal would be to just remove the links, but that breaks that feature for everyone.

Another alternative, to maintain compatibility, is to make a second package which this package has as an optional dependency on.

[heaths]

Is this question in response to tools/packaging/packaging.strings.psd1? As for build scripts, I see no problems with that and changes should be isolated (i.e. don't mess with the host outside of the build directory), but if in relation to the aforementioned file I really don't know. All I can say is that I don't see symlinking other packages' files by scripts that don't own them. Seems wrong to mess with another system like that - like in the old days before SFC on Windows when third-parties would replace Windows' bits.

[TravisEz13]

@heaths libmi, provided by another team, has a hard coded path to load the libraries from. That team has decided not to support PowerShell anymore. To keep things functioning (until it completely brakes), this is our work around. symlinks would be our solution, if through the package manager or the script. I'm honestly happy to pull the symlinks from the package all together.

I don't think this is the same as replacing windows bits. We are symlinking other packages INTO our folder, not the other way around.

[jborean93]

provided by another team, has a hard coded path to load the libraries from.

That's not true, the "universal" Linux build they have provided you are linked to the following OpenSSL libs

• libssl.so.1.0.0
• libcrypto.so.1.0.0

These are not absolute paths but library names that the loader checks in the various library paths on the host. You can see that in the following snippet, note I don't have OpenSSL 1.0.x installed and deleted the symlinks to show you what actual linked name so not found is expected.

[root@713b4cbcd41d /]# ldd /opt/microsoft/powershell/7/libmi.so
ldd: warning: you do not have execution permission for `/opt/microsoft/powershell/7/libmi.so'
        linux-vdso.so.1 (0x00007ffeb393a000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb157362000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fb15715e000)
        libpam.so.0 => /lib64/libpam.so.0 (0x00007fb156f4e000)
        libssl.so.1.0.0 => not found
        libcrypto.so.1.0.0 => not found
        libc.so.6 => /lib64/libc.so.6 (0x00007fb156b8b000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb157582000)
        libaudit.so.1 => /lib64/libaudit.so.1 (0x00007fb156961000)
        libcap-ng.so.0 => /lib64/libcap-ng.so.0 (0x00007fb15675b000)

Typically these libraries are just picked up in the normal load path like /usr/lib, or the one specified by LD_LIBRARY_PATH. The libmi.so binary also has an RPATH set to $ORIGIN which allows it to also check the origin path (same dir it is placed in) which is what allows you to use the symlinks

[root@2cd102a3a066 /]# objdump -x /opt/microsoft/powershell/7/libmi.so | grep RPATH
  RPATH                $ORIGIN

But with the exception of EL7, which I go into below, you don't need the symlinks at all as lib[crypto|ssl].so.1.0.0 is in the normal system paths for distributions that ship with OpenSSL 1.0.x. On Ubuntu 16.04 I deleted the symlinks and you can see that it picked up the system locations just fine.

root@9d42597878f5:/# ldd /opt/microsoft/powershell/7/libmi.so
        linux-vdso.so.1 =>  (0x00007ffc80db9000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0741cf0000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0741aec000)
        libpam.so.0 => /lib/x86_64-linux-gnu/libpam.so.0 (0x00007f07418de000)
        libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f0741676000)
        libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f0741231000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0740e67000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0741f0d000)
        libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007f0740c40000)

In a normal situation you should just be picking up these libraries as they are installed, this actually works just fine if you have OpenSSL 1.0.x installed with only 2 caveats

• Enterprise Linux 7 (CentOS 7, RHEL 7, etc) uses .10 as the suffix for the OpenSSL libs in /usr/lib64 so they won't be picked up
  • This is one case where you probably need the symlinks or to patch the ELF headers to point to the distro specific path for that distribution only
  • Other distributions that still package OpenSSL 1.0.x use the lib[ssl|crypto].so.1.0.0 file so they just work
• macOS does have a hardcoded path to /usr/local/opt/openssl/lib/lib[ssl|crypto].1.0.0.dylib
  • This is a different distribution package and IIRC it doesn't contain symlinks so I consider it a separate problem

The other issue is that the PowerShell package for newer distributions have a symlink to point lib[ssl|crypto].1.0.0 to the OpenSSL 1.1.x paths which is completely wrong. This causes a seg fault in the process when libmi tries to use any OpenSSL APIs.

So ultimately what I think should happen is

• Patch the libmi.so library to change the suffix of libssl and libcrypto to .10 for the RPM/Enterprise Linux package
• Do nothing, no symlinks on the other Linux distributions

I also think you should be consuming the newer releases from the omi repo where that team has actually provided an official "universal" OpenSSL 1.1.x build which you can use on newer distributions that don't ship OpenSSL 1.0.x (read mostly all of them). It also means that patched libmi.so library is only shipped on EL7, everything else shouldn't require symlinks.

macOS is another problem and there's not much you can do there without recompling your own version of omi on OpenSSL 1.1.x which is understandable why you don't want to do.

[TravisEz13]

@jborean93 This issue is not to improve libmi behavior. Please open a new issue for making libmi behavior better.

As for resolving this, I'll open a new issue for resolving creating the symlinks in the postscript. Thinking about it, I think the best approach is separate packages with the symlink.

[jborean93]

I would love to resolve the libmi but I know the team doesn’t want to look into it. My posts are talking about the issues with how the current libmi is packaged. The package has symlinks which is not needed all the time (only EL7). The package symlinks point to OpenSSL 1.1.x which causes seg faults in PowerShell whenever you try to use any of the TLS components in libmi. How is that not related to the packaging components and thus this PR.

there are comments asking why things are done the way they are, I’m explaining the problem and why the existing packaging setup is wrong and giving possible fixes for it. I can’t really raise the PR myself because there are a lot of internal moving parts in the packaging process so my comments are a way of trying to stress what is wrong and hopefully illustrate what could be done to solve it.

[daxian-dbw]

@jborean93 How about making the PSWSMan module the only source of libmi and libpsrpclient binaries for PowerShell remoting over WSMan on Linux and macOS? Namely, powershell will not ship those libs, but instead, tell users to leverage the PSWSMan module to enable WSMan remoting on Linux and macOS.

Please see the discussion topic #15310, and let's discuss the feasibility there.