Inherited Test SetUp, TearDown, etc. are not executed in .NET Core if they are not public

Hello All,

In trying to port some NUnit tests to .NET Core I've found that methods with [OneTimeSetUp], [SetUp], [TearDown], and [OneTimeTearDown] attributes from a base class are not being called on [Test]s in derived classes. This is only occurring when running against .NET Core, for all versions of NUnit I've tried (3.7.1 and up).

Apologies if this has been documented elsewhere (I believe I checked thoroughly...) or if I'm doing something silly here!

To reproduce:
csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>

  <!--<PropertyGroup>
    <TargetFramework>net46</TargetFramework>
  </PropertyGroup>
  <PropertyGroup Condition="'$(TargetFramework)' == 'net46'">
    <TargetFrameworkIdentifier>.NETFramework</TargetFrameworkIdentifier>
  </PropertyGroup>-->

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
    <PackageReference Include="NUnit" Version="3.8.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.8.0" />
  </ItemGroup>

</Project>

Test:

public class BaseClass
    {
        // These methods all get called when running the "Inheritance" test in .NET framework,
        // but not in .NET Core
        [OneTimeSetUp]
        protected void OneTimeSetUp()
        {
            Console.WriteLine("OneTimeSetUp");
        }

        [SetUp]
        protected void SetUp()
        {
            Console.WriteLine("SetUp");
        }

        [TearDown]
        protected void TearDown()
        {
            Console.WriteLine("TearDown");
        }

        [OneTimeTearDown]
        protected void OneTimeTearDown()
        {
            Console.WriteLine("OneTimeTearDown");
        }

        public class DerivedClass : BaseClass
        {
            [Test]
            public void Inheritance()
            {
                Console.WriteLine("Test");
            }
        }
    }

EDIT: use XML, not C#, formatting for csproj

[rprouse]

Thanks, I think I've seen this issue in another repository, but it does belong here. Your code looks good, so I am going to mark this as a bug immediately. The reflection APIs have subtle differences between .NET Core and .NET Framework, so it is likely caused by that. We will look into it. Thanks.

[mikkelbu]

Hi @DeclanCodes. Can you check wether making the methods public solves the problem (see #1657 for a similar issue).

Changing those methods to public makes them called as expected - thanks for the workaround, @mikkelbu

Sounds like this is the same issue, but #1657 is muddied by the fact that it references the dotnet-test-nunit repo in the title. I'll leave the decision of whether to close this one up to the maintainers

[CharliePoole]

If non-public setup and teardown are working in other builds, I would say there's some kind of bug here. At a minimum it should be documented.

[rprouse]

I have updated the title to highlight the problem. I also think this issue is more clearly documented than #1657, so I am going to close the other issue as a duplicate.

[mikkelbu]

Just looking at the code, I think the problem lies in TypeExtensions.GetAllMethods which looks as follows:

        static IList<MethodInfo> GetAllMethods(this Type type, bool includeBaseStatic = false)
        {
            List<MethodInfo> methods = type.GetTypeInfo().DeclaredMethods.ToList();
            type = type.GetTypeInfo().BaseType;
            if (type != null)
            {
                var baseMethods = type.GetAllMethods(includeBaseStatic)
                    .Where(b => b.IsPublic && (includeBaseStatic || !b.IsStatic) && !methods.Any(m => m.GetRuntimeBaseDefinition() == b));
                methods.AddRange(baseMethods);
            }

            return methods;
        }

We only look at the methods of the base-class which have IsPublic == true. That seems wrong.

I'll try to add some tests and supply a PR.

[rprouse]

Good catch, I suspected it was in our extensions, but didn't know which one.

[mikkelbu]

I think that there is a similar problem for properties (the code in ApplyBindingFlags wrt. BindingFlags.NonPublic is quite similar). I'll create a new issue for this.

[ChrisMaddock]

@rprouse - I haven't milestoned this, as I'm not sure what you'll be doing for 3.8.2. (With hindsight - maybe I shouldn't have merged either - sorry!)

Will leave it with you.