Failing tests on Mono 4.8 and Mono latest

[dsyme]

Some tests have started to fail on Linux Mono latest:

Succeeds: https://travis-ci.org/fsharp/FSharp.Compiler.Service/builds/213030630

Failed: https://travis-ci.org/fsharp/FSharp.Compiler.Service/builds/213819074

These have identical mono versions

Mono JIT compiler version 4.8.0 (Stable 4.8.0.520/8f6d0f6 Wed Mar 15 16:18:35 UTC 2017)

But the F# package being used has changed:

Unpacking libfsharp-core4.3-cil (from .../libfsharp-core4.3-cil_4.0.0.3-0xamarin1_all.deb) ...

to

Unpacking libfsharp-core4.3-cil (from .../libfsharp-core4.3-cil_4.0.1.20-0xamarin3_all.deb) ...

With this new package the assembly FSharp.Core 4.3.0.0 is no longer being resolved. Also the compiler used seems to have changed:

				Tool /usr/lib/cli/fsharp/fsc.exe execution started with arguments: ...
				Tool /usr/lib/mono/fsharp/fsc.exe execution started with arguments: ...

[dsyme]

Here are the failures

Errors and Failures:
1) Test Failure : FSharp.Compiler.Service.Tests.ExprTests.Check use of type provider that provides calls to F# code
     Should not be errors, but: [|startup (1,1)-(1,1) typecheck error internal error: BuildFrameworkTcImports: no resolution of 'FSharp.Core.dll';
  unknown (1,1)-(1,1) typecheck error Unable to find the file 'FSharp.Core.dll' in any of
 /usr/lib/mono/4.5-api
 /home/travis/build/fsharp/FSharp.Compiler.Service/tests/service/data/TestProject
 /home/travis/build/fsharp/FSharp.Compiler.Service/bin/v4.5/|]
  Expected is <System.Object[0]>, actual is <Microsoft.FSharp.Compiler.FSharpErrorInfo[2]>
  Values differ at index [0]
  Extra:    < <startup (1,1)-(1,1) typecheck error internal error: BuildFrameworkTcImports: no resolution of 'FSharp.Core.dll'>, <unknown (1,1)-(1,1) typecheck error Unable to find the file 'FSharp.Core.dll' in any of
 /usr/lib/mono/4.5-api
 /home/travis/build/fsharp/FSharp.Compiler.Service/tests/service/data/TestProject
 /home/travis/build/fsharp/FSharp.Compiler.Service/bin/v4.5/> >
at FSharp.Compiler.Service.Tests.ExprTests.Check use of type provider that provides calls to F# code () [0x0008b] in <58d91b2685c5f7fea7450383261bd958>:0

2) Test Failure : FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- 2nd level references
     Expected: "Found 'FSharp.Core.dll'"
Actual: "Failed to find 'FSharp.Core.dll'"
  Expected string length 23 but was 32. Strings differ at index 1.
  Expected: "Found 'FSharp.Core.dll'"
  But was:  "Failed to find 'FSharp.Core.dll'"
  ------------^

at FsUnit.shouldEqual[a] (a x, a y) [0x0002f] in <58d91b2685c5f7fea7450383261bd958>:0
at FSharp.Compiler.Service.Tests.ProjectOptionsTests.checkOption (System.String[] opts, System.String s) [0x0003f] in <58d91b2685c5f7fea7450383261bd958>:0
at FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- 2nd level references () [0x00029] in <58d91b2685c5f7fea7450383261bd958>:0

3) Test Failure : FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- references
     Expected: "Found 'FSharp.Core.dll'"
Actual: "Failed to find 'FSharp.Core.dll'"
  Expected string length 23 but was 32. Strings differ at index 1.
  Expected: "Found 'FSharp.Core.dll'"
  But was:  "Failed to find 'FSharp.Core.dll'"
  ------------^

at FsUnit.shouldEqual[a] (a x, a y) [0x0002f] in <58d91b2685c5f7fea7450383261bd958>:0
at FSharp.Compiler.Service.Tests.ProjectOptionsTests.checkOption (System.String[] opts, System.String s) [0x0003f] in <58d91b2685c5f7fea7450383261bd958>:0
at FSharp.Compiler.Service.Tests.ProjectOptionsTests.Project file parsing -- references () [0x00029] in <58d91b2685c5f7fea7450383261bd958>:0

4) Test Failure : Tests.Service.MultiProjectAnalysisTests.Type provider project references should not throw exceptions
     Expected: false
Actual: true
  Expected: False
  But was:  True

at FsUnit.shouldEqual[a] (a x, a y) [0x0002f] in <58d91b2685c5f7fea7450383261bd958>:0
at Tests.Service.MultiProjectAnalysisTests.Type provider project references should not throw exceptions () [0x00085] in <58d91b2685c5f7fea7450383261bd958>:0

5) Test Failure : Tests.Service.ProjectAnalysisTests.Test Project25 whole project errors
     Expected: 0
Actual: 6
  Expected: 0
  But was:  6

at FsUnit.shouldEqual[a] (a x, a y) [0x0002f] in <58d91b2685c5f7fea7450383261bd958>:0
at Tests.Service.ProjectAnalysisTests.Test Project25 whole project errors () [0x00064] in <58d91b2685c5f7fea7450383261bd958>:0

6) Test Failure : Tests.Service.ProjectAnalysisTests.Test symbol uses of type-provided members
     Expected: [|("FSharp.Data.XmlProvider", "file1", ((4, 15), (4, 26)),
   ["class"; "provided"; "erased"]);
  ("FSharp.Data.XmlProvider", "file1", ((4, 15), (4, 26)),
   ["class"; "provided"; "erased"]);
  ("FSharp.Data.XmlProvider", "file1", ((4, 15), (4, 26)),
   ["class"; "provided"; "erased"]);
  ("FSharp.Data.XmlProvider", "file1", ((4, 15), (4, 26)),
   ["class"; "provided"; "erased"]);
  ("TypeProviderTests.Project", "file1", ((4, 5), (4, 12)), ["abbrev"]);
  ("TypeProviderTests.Project", "file1", ((5, 8), (5, 15)), ["abbrev"]);
  ("FSharp.Data.XmlProvider<...>.GetSample", "file1", ((5, 8), (5, 25)),
   ["member"]);
  ("Microsoft.FSharp.Core.int", "file1", ((7, 23), (7, 26)), ["abbrev"]);
  ("Microsoft.FSharp.Core.int", "file1", ((7, 23), (7, 26)), ["abbrev"]);
  ("TypeProviderTests.Record.Field", "file1", ((7, 16), (7, 21)), ["field"]);
  ("TypeProviderTests.Record", "file1", ((7, 5), (7, 11)), ["record"]);
  ("TypeProviderTests.Record", "file1", ((8, 10), (8, 16)), ["record"]);
  ("TypeProviderTests.Record.Field", "file1", ((8, 17), (8, 22)), ["field"]);
  ("TypeProviderTests.r", "file1", ((8, 4), (8, 5)), ["val"]);
  ("FSharp.Data.XmlProvider", "file1", ((10, 8), (10, 19)),
   ["class"; "provided"; "erased"]);
  ("FSharp.Data.XmlProvider<...>", "file1", ((10, 8), (10, 68)),
   ["class"; "provided"; "staticinst"; "erased"]);
  ("FSharp.Data.XmlProvider<...>.GetSample", "file1", ((10, 8), (10, 78)),
   ["member"]); ("TypeProviderTests", "file1", ((2, 7), (2, 24)), ["module"])|]
Actual: [|("TypeProviderTests.Project", "file1", ((4, 5), (4, 12)), ["abbrev"]);
  ("Microsoft.FSharp.Core.int", "file1", ((7, 23), (7, 26)), ["abbrev"]);
  ("Microsoft.FSharp.Core.int", "file1", ((7, 23), (7, 26)), ["abbrev"]);
  ("TypeProviderTests.Record.Field", "file1", ((7, 16), (7, 21)), ["field"]);
  ("TypeProviderTests.Record", "file1", ((7, 5), (7, 11)), ["record"]);
  ("TypeProviderTests.Record", "file1", ((8, 10), (8, 16)), ["record"]);
  ("TypeProviderTests.Record.Field", "file1", ((8, 17), (8, 22)), ["field"]);
  ("TypeProviderTests.r", "file1", ((8, 4), (8, 5)), ["val"]);
  ("TypeProviderTests", "file1", ((2, 7), (2, 24)), ["module"])|]
  Expected is <System.Tuple`4[System.String,System.String,System.Tuple`2[System.Tuple`2[System.Int32,System.Int32],System.Tuple`2[System.Int32,System.Int32]],Microsoft.FSharp.Collections.FSharpList`1[System.String]][18]>, actual is <System.Tuple`4[System.String,System.String,System.Tuple`2[System.Tuple`2[System.Int32,System.Int32],System.Tuple`2[System.Int32,System.Int32]],Microsoft.FSharp.Collections.FSharpList`1[System.String]][9]>
  Values differ at index [0]
  Expected: <(FSharp.Data.XmlProvider, file1, ((4, 15), (4, 26)), [class; provided; erased])>
  But was:  <(TypeProviderTests.Project, file1, ((4, 5), (4, 12)), [abbrev])>

at FsUnit.shouldEqual[a] (a x, a y) [0x0002f] in <58d91b2685c5f7fea7450383261bd958>:0
at Tests.Service.ProjectAnalysisTests.Test symbol uses of type-provided members () [0x00541] in <58d91b2685c5f7fea7450383261bd958>:0

7) Test Error : Tests.Service.ProjectAnalysisTests.Test symbol uses of type-provided types
   System.NullReferenceException : Object reference not set to an instance of an object
  at Tests.Service.ProjectAnalysisTests.Test symbol uses of type-provided types () [0x0006b] in <58d91b2685c5f7fea7450383261bd958>:0 
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in <f712f98eb8e445c8918edaf595bbe465>:0 

[dsyme]

Also this on macOS:

  /Users/travis/build/fsharp/FSharp.Compiler.Service/tests/service/FSharp.Compiler.Service.Tests.fsproj.metaproj : 
    error MSB4025: The project file could not be loaded. Could not find file 
  
  "/Users/travis/build/fsharp/FSharp.Compiler.Service/tests/service/FSharp.Compiler.Service.Tests.fsproj.metaproj".

[dsyme]

@nosami Can you build this project on Mono 5.0 macOS ? Do you know what this might be?
"/Users/travis/build/fsharp/FSharp.Compiler.Service/tests/service/FSharp.Compiler.Service.Tests.fsproj.metaproj".

[nosami]

I get the same error on Mono 5.0. I never heard of a metaproj until just now :)

I created an msbuild diagnostic log here

[dsyme]

Trying to fix it here: 7a60b9c

[dsyme]

That gets a bit further, but now we have resources problems:

     Should not be errors, but: [|startup (1,1)-(1,1) parameter error Problem reading assembly '/Users/travis/build/fsharp/FSharp.Compiler.Service/tests/service/data/TestTP/../../../../bin/v4.5/TestTP.dll': Could not find any resources appropriate for the specified culture or the neutral culture.  Make sure "fsstrings.resources" was correctly embedded or linked into assembly "FSharp.Compiler.Service" at compile time, or that all the satellite assemblies required are loadable and fully signed.|]

[dsyme]

@nosami we need a new FSharp.Compiler.Tools after fsharp/fsharp#734 which includes the target fix you made

[dsyme]

Fixed at last