Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor difficulty adjustment mod combinations test #15868

Merged
merged 2 commits into from
Dec 1, 2021

Conversation

smoogipoo
Copy link
Contributor

I wanted to check whether this handled the MultiMod case correctly (see TestMultiModFlattening) and found that it was very hard to read.
This refactors the tests to be explicit about expected types in each mod combination.

Comment on lines 145 to 154
foreach (Type[] expected in expectedCombinations)
{
Type[] actualTypes = ModUtils.FlattenMods(actualCombinations).Select(m => m.GetType()).ToArray();

Assert.Multiple(() =>
{
foreach (var expectedType in expected)
Assert.Contains(expectedType, actualTypes);
});
}
Copy link
Collaborator

@bdach bdach Nov 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it's just me, but the way this is written is confusing me, for two reasons. First off, actualTypes is computed in the inner loop, even though its value actually is the same in every iteration of the loop. Worse yet, the flattening of all actualCombinations and the containment check weakens the test.

Consider TestIncompatibleWithSameInstanceViaMultiMod(): the expected combinations are:

expectedCombinations = {{ModNoMod}, {ModA}, {ModA, ModB}}

But the FlattenMods operation returns a flat list instead, yielding

actualTypes = {ModNoMod, ModA, ModA, ModB}

And because the containment assert is checking each individual mod from the expected combination at once rather than the entire set, you can get false positives. As in, imagine that the last element in actualCombinations was {ModB}, rather than {ModA, ModB}; actualTypes would still contain ModA, even though the behaviour would be erroneous. And conversely, if you change {ModA, ModB} in expectedCombinations to just {ModB}, the test passes erroneously for the same reason.

If I were to write this method myself, I would write:

private void assertCombinations(Type[][] expectedCombinations, Mod[] actualCombinations)
{
    Assert.AreEqual(expectedCombinations.Length, actualCombinations.Length);

    Assert.Multiple(() =>
    {
        for (int i = 0; i < expectedCombinations.Length; ++i)
        {
            Type[] expectedTypes = expectedCombinations[i];
            Type[] actualTypes = ModUtils.FlattenMod(actualCombinations[i]).Select(m => m.GetType()).ToArray();

            Assert.That(expectedTypes, Is.EquivalentTo(actualTypes));
        }
    });
}

which would correctly fail in the cases I gave, because entire sets of mods are being checked, rather than only individual parts of these sets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I've applied your fix.

@bdach
Copy link
Collaborator

bdach commented Nov 30, 2021

To approach this from another angle, I think that the big part of why the original test was confusing was that multi mods were checked out of order rather than along with everything else. As in, I feel like something like below would be an improvement over master:

[Test]
public void TestDoubleIncompatibleMods()
{
    var combinations = new TestLegacyDifficultyCalculator(new ModA(), new ModB(), new ModIncompatibleWithA(), new ModIncompatibleWithAAndB()).CreateDifficultyAdjustmentModCombinations();

    Assert.AreEqual(8, combinations.Length);

    Assert.IsTrue(combinations[0] is ModNoMod);

    Assert.IsTrue(combinations[1] is ModA);

    Assert.IsTrue(combinations[2] is MultiMod);
    Assert.IsTrue(((MultiMod)combinations[2]).Mods[0] is ModA);
    Assert.IsTrue(((MultiMod)combinations[2]).Mods[1] is ModB);

    Assert.IsTrue(combinations[3] is ModB);

    Assert.IsTrue(combinations[4] is MultiMod);
    Assert.IsTrue(((MultiMod)combinations[4]).Mods[0] is ModB);
    Assert.IsTrue(((MultiMod)combinations[4]).Mods[1] is ModIncompatibleWithA);

    Assert.IsTrue(combinations[5] is ModIncompatibleWithA);

    Assert.IsTrue(combinations[6] is MultiMod);
    Assert.IsTrue(((MultiMod)combinations[6]).Mods[0] is ModIncompatibleWithA);
    Assert.IsTrue(((MultiMod)combinations[6]).Mods[1] is ModIncompatibleWithAAndB);

    Assert.IsTrue(combinations[7] is ModIncompatibleWithAAndB);
}

That said, if the logic error pointed out in my review above is fixed, I don't really have a strong personal preference for either. But the fact that that logic was wrong may indicate that it would be better to just keep it simple.

Copy link
Sponsor Member

@peppy peppy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reads saner.

@peppy peppy merged commit a89e18d into ppy:master Dec 1, 2021
@smoogipoo smoogipoo deleted the refactor-test branch September 11, 2023 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants