Make TestMethod.Arguments public or otherwise accessible (e.g. TestContext)

[bartoszkp]

NUnit 3.6.1 (from NuGet)

I need a property in test's base class that returns a different value depending on test method's parameters (details in this SO question: http://stackoverflow.com/questions/42763574/how-to-get-current-parameter-value-from-testcontext - I don't feel they are strictly relevant here though).

I've created a ITestAction attribute and in the BeforeTest call I have access to TestMethod instance describing currently executed test.

If I could read the Arguments property I could react and accordingly set some properties in the fixture's base. Unfortunately, it's internal. This doesn't seem justified - a TestMethod description should naturally present method call arguments to the clients.

As a side note, I've tried also to use properties for this scenario, but even though I added them to my TestCaseSource results, they are not accessible from test method's level (probably related to issues #1358, #548 or #796).

[CharliePoole]

Unfortunately, the only "clients" for whom we intended TestMethod are other components in the framework - hence the use of internal. If we thought the arguments were generally needed, we would probably add them to the ITest interface. Alternatively (or additionally) we could add the arguments to the TestContext.CurrentContext.CurrentTest.

The ITest interface is intended for use by extenders. TestContext is intended for use by general users writing tests with NUnit, although extenders can use it as well.

But...

We need a good use case to justify either addition. I've read your description in this issue several times, as well as the SO post, and I'm still not clear why you need this. The argument is passed directly to your test in the first place. I'm missing the reason you need to go to so much trouble to retrieve it. On SO, somebody asked what you want to use the value for. You didn't appear to answer so I'm re-asking the question: where do you need this value and why? I add "where" because it clearly can't be in the test method, which already has it.

IF you have a use case that makes sense to us, we can then evaluate how widespread that need might be and decide whether to implement this and at what priority.

BTW, the SO example uses Given/When/Then syntax but doesn't indicate what product you are using. That may have some bearing on why I'm not understanding exactly what you want to do.

[bartoszkp]

Thank you for your prompt answer. Extending the TestContext class as you explain would be great!

First of all, as a side note, the Given/When/Then syntax are ordinary C# methods designed in such a way, to make the tests readable - no special framework was used for this, just some "fluent magic".

Back to the point:

Unfortunately I doubt that the use case will convince you - the explanation will take long, and I doubt it will be interesting to anyone. However of course, I will provide it, since you ask :)

Before that, I think the better argument for this is that conceptually, the test method call arguments are observable property of a TestMethod, and taking into account your idea about the TestContext - test method call arguments conceptually are a part of a test context. So my main argument is just conceptual completeness.

Another justification, although as I've mentioned I understand it may be seen as minor, objectively speaking, is the scenario you ask about.

In my case I have tests that validate reading data - series of values with timestamps from a data provider service. Naturally, almost all test cases are some variations of this basic form:

Given(Data.ForRange(FromTimestamp, ToTimestamp(granularity)).WithValue(42));

When(ReadingData.ForRange(FromTimestamp, ToTimestamp(granularity)));

ThenResultEquals(Data.ForRange(FromTimestamp, ToTimestamp(granularity)).WithValue(42));

There are many ways of reading the data, including some interpolations, approximations, etc. etc. - thus there are more or less 8k tests of this form. And what is the most important: the data can have various granularities: samples can be stored for each second, or only each month etc.

Now, these tests are all organized into fixtures, naturally, with one common base:

[TestFixture]
public class FirstRangeInterpolationTests : TestsBase
{
[Test]
[TestCaseSource(typeof(AllGranularities))] // yields Second, Minute, Hour, ..., Month, Year
public void GivenSomeData_WhenReadingForWiderRange_ReturnsInterpolatedData(Granularity granularity)
{
//...
}
}

The timestamps from the first snippet (FromTimestamp and ToTimestamp) are defined in the base class:

public class TestsBase
{
    // This timestamp happens to be a beginning of a Year, Month, Week, Day, Hour, Minute and Second
    // so is perfect for testing all granularities universally
    public DateTime FromTimestamp { get { return new DateTime(2018, 1 1); } }

    // current solution:
    public DateTime ToTimestamp(Granularity granularity)
    {
        return FromTimestamp.AddSteps(granularity, 5);
    }

    // desired solution:
    public DateTime ToTimestamp 
    {
        get 
        {
            return FromTimestamp.AddSteps(granularity, 5); // need granularity for current test
        }
    }
}

An example data timestamp range for testing is always [FromTimestamp - FromTimestamp + 5 steps) - where a step depends on granularity.

So, the "current solution" is of course a trivial solution to this "problem" (AddSteps is our extension method).

However, I want to avoid repeating the same thing in each of 8k tests: ToTimestamp(granularity), because the expression for creating given and expected data are already quite complicated. The tests would look much clearer if it was just FromTimestamp and ToTimestamp - as these timestamps, being admittedly important, are not the most important issue in each test. Also, for our developers it is obvious that a "step" is directly implied by the granularity context, so making this implicit is not a problem. The problem is just too much noise in these tests.

I've omitted some details regarding irrelevant aspects of our complicated code base for clarity here, but I will show an exact, yet still one of the simplest examples, from our tests :

Given(A.Signal().WithGranularity(granularity).With(
    Data<DataType>.ForRange(FromTimestamp, ToTimestamp(granularity), granularity)
        .WithValue(Value(15))
        .WithQuality(Quality.Good)
        .StartingWith(Value(10))
        .EndingWith(Value(Value(20))));

removing one granularity makes (arguably) quite a big difference here:

Given(A.Signal().WithGranularity(granularity).With(
    Data<DataType>.ForRange(FromTimestamp, ToTimestamp, granularity)
        .WithValue(Value(15))
        .WithQuality(Quality.Good)
        .StartingWith(Value(10))
        .EndingWith(Value(Value(20))));

(side note: I'm also working on removing the last ForRange argument).

So, if in TestsBase I could access current granularity for a test (i.e. the argument passed from TestCaseSource to the test method), the solution would be trivial. Otherwise it looks as I've shown in the aforementioned SO post and I don't need to be reminded that it's ugly and unsafe ;) Having the values in TestContext would also work great, as I've mentioned.

[CharliePoole]

OK, just to be sure I understand, let me restate this with drastic over-simplification:

1. The argument is available to the test method in the derived test class.
2. You would like to have it available to other methods, without those methods taking an argument. The methods in question happen to be in a base class, but even if they were not the same issues arise.
3. You would also like to avoid having every test store the argument in a protected member, since that would add ~8000 lines to your tests.

Right?

Question: Do all the tests in a given concrete fixture use the same granularity? Or could you re-organize them to make that so? If so, I suggest making granularity an argument of the fixture constructor rather than the test. You can save it in the constructor for use by all tests and not have to wait for a new NUnit release.

It does seem to me that TestContext is the best place to put this for you, since you are only creating an extension because no other approach is available. If you agree, I'll re-title this.

[bartoszkp]

Yes, that's about right. Sorry, I've edited the title before I saw your response - of course feel free to change it as you see fit - extending TestContext works great for me, and in general looks much cleaner than exposing TestMethod's members.

Your idea with the constructor is clever, however it won't work because of other circumstances in my scenario - 1) Minor inconvenience: TestCaseSource cannot be applied to fixture, so I'd have to enumerate all granularities for all fixtures 2) Major inconvenience: our fixtures are generic and already parametrized by all possible data types (the tests concern integers, decimals and doubles and even bools and strings for some cases). This would lead to a combinatoric explosion of TestFixture attributes.

Thank you very much for considering this issue as a possible improvement of your framework.

[CharliePoole]

I'll do that. I'm not unwilling to do the other, but I would do it through the ITest interface, which is what we really give you for an ActionAttribute - even though you can always cast it to TestMethod.

I can't help you with your second point, but for the first one see TestFixtureSourceAttribute.

[Suremaker]

Hello,
I have a similar need in order to implement parameterized test method formatting in LightBDD framework (LightBDD/LightBDD#22).

I have an attribute implementing the ITestAction interface where I capture the context of the test: https://github.com/LightBDD/LightBDD/blob/master/src/LightBDD.NUnit3/FeatureFixtureAttribute.cs

With the ITest.Arguments property exposed I could properly capture the passed parameters and include them in formatted scenario name.

[CharliePoole]

This seems reasonable to me for adding to our backlog. If anyone wants to do a PR, please step up. 😄