regression in SimpleConstructors test inside System.Xml.Linq.TreeManipulation.Tests

[wfurt]

I can reproduce it locally - the caveat is that this is outer loop test so I guess it got missed by CI.
I was suspecting #44414 but I can reproduce it even when syncing prior to that.

I started to fail 11/10 .e.g. today.
Binary search points to #44300

cc: @krwq and @stephentoub who touch the area recently.

There is not much aside from:

   Discovering: System.Xml.Linq.TreeManipulation.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Xml.Linq.TreeManipulation.Tests (found 74 test cases)
    Starting:    System.Xml.Linq.TreeManipulation.Tests (parallel test collections = on, max threads = 16)
  TestCase:XNode.ReplaceWith -
  Pass:10, Fail:0, Skip:0
  TestCase:XNode.ReplaceWith with Events -
  Pass:1, Fail:0, Skip:0
  TestCase:Constructors with params - XElement - node + array -
  Pass:13, Fail:0, Skip:0
  TestCase:Simple constructors -
  	DTD - all nulls
  Expected:  ()
  Actual  :  (System.String)

  Microsoft.Test.ModuleCore.TestFailedException: dtd.InternalSubset, data[3]
     at Microsoft.Test.ModuleCore.TestLog.Compare(Object actual, Object expected, String message) in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/XDocument.Test.ModuleCore/testlog.cs:line 207
     at XLinqTests.SimpleObjectsCreation.DTDConstruct() in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/SimpleObjectsCreation.cs:line 936
     at Microsoft.Test.ModuleCore.TestVariation.Execute() in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/XDocument.Test.ModuleCore/testvariation.cs:line 64
     at Microsoft.Test.ModuleCore.TestCase.Execute() in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/XDocument.Test.ModuleCore/testcase.cs:line 84
  	 FAILED
  Pass:73, Fail:1, Skip:0
  TestCase:XNode.Remove with Events -
      XLinqTests.TreeManipulationTests.SimpleConstructors [FAIL]
        Assert.Equal() Failure
        Expected: 0
        Actual:   1
        Stack Trace:
          /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/TreeManipulationTests.cs(432,0): at XLinqTests.TreeManipulationTests.RunTestCase(TestItem testCase)
          /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/TreeManipulationTests.cs(92,0): at XLinqTests.TreeManipulationTests.SimpleConstructors()
  Pass:10, Fail:0, Skip:0

[ghost]

Tagging subscribers to this area: @buyaa-n, @krwq, @jeffhandley
See info in area-owners.md if you want to be subscribed.

---

Issue meta data

Issue content:	I can reproduce it locally - the caveat is that this is outer loop test so I guess it got missed by CI. I was suspecting #44414 but I can reproduce it even when syncing prior to that. I started to fail 11/10 .e.g. today. Binary search points to #44300 cc: @krwq and @stephentoub who touch the area recently. There is not much aside from: Discovering: System.Xml.Linq.TreeManipulation.Tests (method display = ClassAndMethod, method display options = None) Discovered: System.Xml.Linq.TreeManipulation.Tests (found 74 test cases) Starting: System.Xml.Linq.TreeManipulation.Tests (parallel test collections = on, max threads = 16) TestCase:XNode.ReplaceWith - Pass:10, Fail:0, Skip:0 TestCase:XNode.ReplaceWith with Events - Pass:1, Fail:0, Skip:0 TestCase:Constructors with params - XElement - node + array - Pass:13, Fail:0, Skip:0 TestCase:Simple constructors - DTD - all nulls Expected: () Actual : (System.String) Microsoft.Test.ModuleCore.TestFailedException: dtd.InternalSubset, data[3] at Microsoft.Test.ModuleCore.TestLog.Compare(Object actual, Object expected, String message) in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/XDocument.Test.ModuleCore/testlog.cs:line 207 at XLinqTests.SimpleObjectsCreation.DTDConstruct() in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/SimpleObjectsCreation.cs:line 936 at Microsoft.Test.ModuleCore.TestVariation.Execute() in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/XDocument.Test.ModuleCore/testvariation.cs:line 64 at Microsoft.Test.ModuleCore.TestCase.Execute() in /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/XDocument.Test.ModuleCore/testcase.cs:line 84 FAILED Pass:73, Fail:1, Skip:0 TestCase:XNode.Remove with Events - XLinqTests.TreeManipulationTests.SimpleConstructors [FAIL] Assert.Equal() Failure Expected: 0 Actual: 1 Stack Trace: /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/TreeManipulationTests.cs(432,0): at XLinqTests.TreeManipulationTests.RunTestCase(TestItem testCase) /Users/furt/github/wfurt-ssl/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/TreeManipulationTests.cs(92,0): at XLinqTests.TreeManipulationTests.SimpleConstructors() Pass:10, Fail:0, Skip:0
Issue author:	wfurt
Assignees:	krwq
Milestone:	[object Object]

[stephentoub]

This was introduced here:
https://github.com/dotnet/runtime/pull/44300/files#diff-6dcf882a73b2db05168fe9308ac11aca79bad7100caaabdefdbbedec9f3e25f0R23
causing this test to now be out-of-date:

runtime/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/SimpleObjectsCreation.cs

Lines 926 to 927 in 6072e4d

	//[Variation(Priority = 1, Desc = "DTD - all nulls", Params = new object[] { new string[] { "root", null, null, null }, "<!DOCTYPE root >" })]
	public void DTDConstruct()

[v-haren]

failed again in job: runtime-libraries-coreclr outerloop 20201117.1

failed test: XLinqTests.TreeManipulationTests.SimpleConstructors

net6.0-Linux-Release-arm64-CoreCLR_release-(Alpine.312.Arm64.Open)ubuntu.1804.armarch.open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.12-helix-arm64v8-20200602002604-25f8a3e

Error message

Assert.Equal() Failure
Expected: 0
Actual:   1


Stack trace
   at XLinqTests.TreeManipulationTests.RunTestCase(TestItem testCase) in /_/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/TreeManipulationTests.cs:line 432
   at XLinqTests.TreeManipulationTests.SimpleConstructors() in /_/src/libraries/System.Private.Xml.Linq/tests/TreeManipulation/TreeManipulationTests.cs:line 92

[krwq]

There is actually a bit regression as serialized output differs a bit (prints extra [] in the doc type where previously it didn't - it has no semantic meaning though). I have explained it more in: #44866 - I'll revert the behavior change I introduced before and we can re-think what the correct behavior should be in the other issue. The XmlDocumentType actually behaves differently than it's documented but consistent with XDocumentType (before my change) but nullability of XmlDocumentType is correct in the doc (it's nullable)

[wfurt]

BTW the test seems pretty fast and stable. Is there reason to keep it as Outerloop?

[krwq]

@wfurt IIRC I initially measured the tests before marking them outerloop and I put some threshold of some number of milliseconds after which we started being too slow overall for XML, initially the guidelines for inner loop were much stricter I believe so we can reconsider that now. Let's split this investigation from the fix though