psrp is unable to load libmi from nuget cache

[adityapatwardhan]

psrp nuget package has libpsrpclient.so which depends on libmi.so.

When an netcore app adds reference to psrp nuget package in its csproj and executes dotnet publish, both the nuget packages are downloaded to nuget cache.

libpsrp.so expects the libmi.so to be present in the same folder, but libmi.so is located in the nuget cache under it's package folder.

Example:

Location of psrp in nuget cache.
/home/user/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native/libpsrpclient.so

Location of libmi in nuget cache.
/home/user/.nuget/packages/libmi/1.4.101000/runtimes/linux-x64/native/libmi.so

psrp expects libmi.so to be at /home/user/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native/

Thanks @ianByrne for finding the issue.

Steps to reproduce

Sample located at: #3417 (comment)

Expected behavior

Application runs as expected.

Actual behavior

Error:

This parameter set requires WSMan, and no supported WSMan client library was found. WSMan is either not installed or unavailable for this system.: Unable to load DLL 'libpsrpclient': The specified module or one of its dependencies could not be found

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.0.1
PSEdition                      Core
GitCommitId                    v6.0.1
OS                             Linux 4.4.0-112-generic #135 Ubuntu 
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[dantraMSFT]

For those hitting this issue, such as PowerShell hosts, the work around is to build targetting a specific platform. This will place the binaries from the psrp and mi packages directly in the output directory with the app itself.

[ianByrne]

Hi @dantraMSFT - I am targeting linux-x64 in both my publish command, and my csproj file:
dotnet publish -c Release --runtime linux-x64

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <RuntimeIdentifiers>linux-x64</RuntimeIdentifiers>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.PowerShell.SDK" Version="6.0.1.1" /> 
    <PackageReference Include="Microsoft.PowerShell.Commands.Diagnostics" Version="6.0.1.1" />
    <PackageReference Include="Microsoft.WSMan.Management" Version="6.0.1.1" />
    <PackageReference Include="psrp" Version="1.4.1" />
  </ItemGroup>

</Project>

The files do indeed get placed into the same output directory as the app itself, however when the app is run I still receive the above error.

[dantraMSFT]

@ianByrne: I'm still working on a viable solution to this.
For now...
I suspect psrp is getting loaded from the nuget cache. You can confirm this by manually copying libmi.so to the location of psrp in the nuget cache. Let me know if that solves the problem.

[ianByrne]

Thanks for the tip @dantraMSFT , however I haven't had any luck.

I am testing this using a docker image (built on a Linux machine):

FROM microsoft/dotnet:2.0-sdk

WORKDIR /app
RUN mkdir /app/src
COPY . ./src
RUN dotnet publish ./src/testapp/testapp.csproj -c Release -o /app/ --runtime linux-x64

RUN cp ./libmi.so /root/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native/libmi.so
WORKDIR /root/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native
RUN ls

WORKDIR /app
ENTRYPOINT ["dotnet", "testapp.dll"]

The build output reveals that the two files are indeed in the /root/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native dir

2018-03-03T13:37:06.4932950Z Step 8/10 : RUN ls
2018-03-03T13:37:06.6504763Z ---> Running in b8261c8465bf
2018-03-03T13:37:06.9260673Z libmi.so
2018-03-03T13:37:06.9273397Z libpsrpclient.so
2018-03-03T13:37:07.3225190Z ---> 2cc54749e797
2018-03-03T13:37:07.3419046Z Removing intermediate container b8261c8465bf

However, on running the app, same error.

[ianByrne]

@dantraMSFT do you have any other suggestions for a workaround, or an ETA of when it may get resolved? Thanks

[Qowy]

How is this even supposed to work?
If I am on a Windows Machine and deploy the project (no matter if framework dependent or independent) libpsrpclient.so and libmi.so are never published. They sit in my .nuget folder, but what good is that if I publish to another Machine without SDK?

And what kind of nuget package is psrp anyway? Since when are we using packages that are only applicable to certain OS? Why isn't it just part of the Microsoft.WSMan.Management? Native libraries for certain OS is done all the time isn't it?

Running the project on a Linux Machine with the SDK also does not work. The files are present in my .nuget folder but I still get Exception: Unable to load DLL 'libpsrpclient': The specified module or one of its dependencies could not be found. (using dotnet run)

[Qowy]

By the way:

Copying
/home/user/.nuget/packages/libmi/1.4.101000/runtimes/linux-x64/native/libmi.so
to
/home/user/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native/
is currently a workaround.

However this requires the the app to be run from the SDK

Looking at your Dockerfile, you are both exporting the app to run without a dotnet Runtime by specifying --runtime and starting it from the SDK anyway.
Maybe try one or the other.

[dantraMSFT]

I'll be releasing a nuget package by EOW that solves the problem on Linux by incorporating libmi into the psrp package. Building with and without --runtime or --output works as expected.

On macOs, the results are more limited. Currently, it will only work when building for a specific runtime.

FYI: the reason we're hitting this problem is due to RPATH usage in psrp and mi. Both are configured to work with PowerShell Core which builds to a specific runtime and third party binaries are restored to the bin directory at build time.

I'll post again when I've published the packages.

[dantraMSFT]

Release 1.4.2-2 (https://github.com/PowerShell/psl-omi-provider/releases/tag/v1.4.2-2) has been published.

[ianByrne]

Thanks @dantraMSFT and @adityapatwardhan - I can see that the psrp package was updated on the 20th, although I still seem to be struggling with this... Can you see where I'm going wrong?

NuGet.conf

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" /> 
    <add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
    <add key="powershell-core" value="https://powershell.myget.org/F/powershell-core/api/v3/index.json" />
  </packageSources>
</configuration>

PSTest.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <RuntimeIdentifiers>linux-x64</RuntimeIdentifiers>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.PowerShell.Commands.Diagnostics" Version="6.0.2" />
    <PackageReference Include="Microsoft.PowerShell.SDK" Version="6.0.2" />
    <PackageReference Include="Microsoft.WSMan.Management" Version="6.0.2" />
    <PackageReference Include="psrp" Version="1.4.2" />
  </ItemGroup>

</Project>

Dockerfile

FROM microsoft/dotnet:2.0-sdk

WORKDIR /app
RUN mkdir /app/src
COPY . ./src
RUN dotnet publish ./src/PSTest.csproj -c Release -o /app/ --runtime linux-x64
# Have also tried without --runtime linux-x64

# Have also tried copying the file directly:
# RUN mkdir -p /root/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native
# RUN cp ./libmi.so /root/.nuget/packages/psrp/1.4.1/runtimes/linux-x64/native/libmi.so

WORKDIR /app

ENTRYPOINT ["dotnet", "PSTest.dll"]

Program.cs

using System;
using System.Management.Automation;
using System.Management.Automation.Runspaces;

namespace PSTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Creating PowerShell");
            PowerShell powerShell = PowerShell.Create();

            Console.WriteLine("Creating runspace");
            powerShell.Runspace = RunspaceFactory.CreateRunspace(new WSManConnectionInfo()); // Throws exception

            Console.WriteLine("We didn't make it this far");
        }
    }
}

Output

Creating PowerShell
Creating runspace
Unhandled Exception: System.Management.Automation.Remoting.PSRemotingTransportException: This parameter set requires WSMan, and no supported WSMan client library was found. WSMan is either not installed or unavailable for this system. ---> System.DllNotFoundException: Unable to load DLL 'libpsrpclient': The specified module or one of its dependencies could not be found.

The Docker image is being built on a VSTS Hosted Linux Preview agent, and is being run on an AWS Fargate ECS cluster.

[dantraMSFT]

The manual workaround of copying libmi to a 1.4.1 directory is not likely to solve the problem.

From your build steps, you should expect to see the update libpsrpclient.so and libmi.so binaries in the --output directory (-o app) and running your app from that directory should work.
I'm speculating that the binaries are not, in fact, being pulled down and suggest you explictly delete the psrp and mi directories from the nuget cache. Your build steps should then indicate that 1.4.2 was pulled down as part of the build.

[ianByrne]

The manual workaround of copying libmi to a 1.4.1 directory is not likely to solve the problem.

Good point 😁 I forgot to update that when I changed to 1.4.2...

I'll add some steps to peek into the /app/ dir to ensure that the files are indeed there, and will also try to explicitly delete from the nuget cache. Might not get around to this for another couple days so will report back then.