Test runs on Ubuntu are showing useless stack on failure

[krwq]

Running test is performed with: Tools\msbuild.sh /t:BuildAndTest project.csproj

example stack trace on Ubuntu 16.04 (obviously big chunk of stack is missing)

  xUnit.net console test runner (64-bit .NET Core)
  Copyright (C) 2014 Outercurve Foundation.
  
  Discovering: System.Security.Cryptography.Xml.Tests
  Discovered:  System.Security.Cryptography.Xml.Tests
  Starting:    System.Security.Cryptography.Xml.Tests
     System.Security.Cryptography.Xml.Tests.EndToEnd.SignedXmlHasVerifiableSignature [FAIL]
        System.PlatformNotSupportedException : Operation is not supported on this platform.
        Stack Trace:
           .../corefx/bin/obj/Unix.AnyCPU.Debug/System.Security.Cryptography.Csp/netcoreapp/System.Security.Cryptography.Csp.notsupported.cs(208,0): at System.Security.Cryptography.SHA1CryptoServiceProvider..ctor()
  Finished:    System.Security.Cryptography.Xml.Tests
  
  === TEST EXECUTION SUMMARY ===

on the first look it looks like somewhere in the code there is something like:

try {
  runtest();
}
catch (Exception e) {
  throw e;
}

replacing throw e; with simply throw; should fix it. I did not investigate where that code is or if that is the problem.

[stephentoub]

@krwq, do you have a link to a CI test log as an example?

(I'm also interested in the specific example you chose: System.Security.Cryptography.Csp is Windows-only.)

[krwq]

@stephentoub - I have this happening on my Ubuntu box at home. I'm seeing this also on CI run in here:
https://ci.dot.net/job/dotnet_corefx/job/master/job/centos7.1_debug_prtest/3898/consoleText (search for SignedXmlHasVerifiableSignature). Look at the PR: dotnet/corefx#16545 - it is really hard to tell where the issue is looking at this stack trace

[stephentoub]

I agree there's something amiss about the stack trace, but I would hold off on that until we understand why Csp is being used on Unix. Are you using something from the Csp library explicitly? It looks like the failure is coming from SHA1.Create(), which is in turn creating a SHA1CryptoServiceProvider, which throws PNSE in its ctor on Unix. But I would expect SHA1.Create() to create something usable on Unix. Have I misread things? Any idea where a SHA1CryptoServiceProvider is being created? If we're actually using a Windows assembly here, I wouldn't be surprised if other things are amiss and that's contributing to bad stacks.

[krwq]

@stephentoub this is happening in CryptoConfig.cs (search for http://www.w3.org/2000/09/xmldsig#sha1). The type is being created by reflection. I'm currently looking into that

[stephentoub]

Thanks. Related, @bartonjs, we didn't make RSACryptoServiceProvider, SHA1CryptoServiceProvider, etc. mostly work as shims on top of the corresponding Unix functionality?

[bartonjs]

@stephentoub I don't think that work ever got done, no. (@steveharter: If it was done it would have been by you...)

[stephentoub]

Thanks. My gut is that's going to be a real issue for having existing code be portable with .NET Standard 2.0. cc: @danmosemsft, @karelz

[krwq]

@bartonjs, @stephentoub is it reasonable to always return SHA1.Create() instead of SHA1CSP? Sounds slightly scary from compat perspective. Perhaps maybe create a fake SHA1CSP class (and similar for all algorithms) which would simply use SHA1? (non windows only or perhaps always do that)

Other option is I can handle it on crypto-xml side

[karelz]

@steveharter @bartonjs was it something we planned to do as part of .NET Standard 2.0? Or did we intentionally left it for future?

[danmoseley]

How would I see this omission in the code -- PNSE? Area owners all signed off on all their instances of "throw new PNSE()" (but I regret we did not ask to make sure there was an explanatory comment on each)

[bartonjs]

I vaguely thought it was part of the plan, and tracked somewhere... but I didn't pay a lot of attention to the ns20 project.

@danmosemsft: https://github.com/dotnet/corefx/search?utf8=%E2%9C%93&q=GeneratePlatformNotSupportedAssembly; or, specifically https://github.com/dotnet/corefx/blob/master/src/System.Security.Cryptography.Csp/src/System.Security.Cryptography.Csp.csproj#L9

[krwq]

@danmosemsft - what's the plan now? Should I do some workarounds in crypto-xml for now or do we want to port CSP?

[danmoseley]

I'm not qualified to answer that question. Can we bring over a useful compatible implementation here that works on Unix by using only API that's available on Unix?

@karelz I didn't know about GeneratePlatformNotSupportedAssembly. There are about 40 assemblies that have it and al or almost all have it specifically for Unix. We need to check what API those guys have that is in NS2.0. We have a couple of examples here -- are there others. It should not be hard to diff and we can take that offline and figure out what to do.

[krwq]

@danmosemsft, I believe it should be possible - I just made MSDN crypto-xml sample work on Ubuntu. @bartonjs - is it ok if I leave my wrapper for CryptoConfig and place workarounds for this over there?

[karelz]

It seems we missed this in NS2.0 work - @steveharter can you please look into it? Please file a bug (this one has a weird title), get cost and then let's decide who/when does it.

@danmosemsft let's do the larger sweep offline.

@krwq do we need an issue tracking the original title "Useless stacks on Linux"?