Add preserveTestName CLI flag to remove partition and browser

[tasha-urbancic]

Closes: #129

I came across the same issue as #129 . Specifically, when I increased our parallelization and added --random we lost CI insights as the output changes. To increase consistency in the output I have added an optional flag --preserve-test-name that removes the dynamic segments from the output. This is a first attempt at solving the issue, so perhaps it isn't the best way.

Because this is is my first time contributing, I am not sure if I am missing something important here. I would love feedback. Thanks in advance!

[tniezurawski]

This is a real issue. We have insights enabled on CircleCI and because of splitting and parallelizing tests, we can't benefit from it to the fullest. Here's an example of flaky tests. We know they fail more often but the name of tests are not consistent so the counting doesn't work as it should:

That of course affects all analyses, like the slowest tests. We see clear duplicates there, the difference comes only from the prepended info about partition and browser... which to be honest, doesn't give us much 🤷‍♂️

How can I help make it happen? Who's approving and releasing this project? @rwjblue? @Turbo87? @SergeAstapov?

[SergeAstapov]

@tniezurawski can't help with pushing this through. but changes looks good to me! thanks for adding tests, this really helps.

[tniezurawski]

Small typo:

Suggested change

	However, if you change the amount of parallelization, or randomize accross partitions, the output will change for the same test, which may be an issue if you are tracking test insights over time.
	However, if you change the amount of parallelization, or randomize across partitions, the output will change for the same test, which may be an issue if you are tracking test insights over time.

[tasha-urbancic]

Thanks for catching this! I have fixed the typo.

[tniezurawski]

Thanks @SergeAstapov 🙌 and thanks @tasha-urbancic for the change 😍

@tasha-urbancic Maybe consider adding "Closes: #129" to the description so it should close the issue automatically after this change is merged.

[tniezurawski]

Just tested on my CI and see the proper result in xunit report (running on 4 containers x 3 browsers each):

<testcase classname="Chrome 106.0" name="Acceptance | 404 page: should redirect user to 404 page when using wrong URL" time="0.261"/>

🎉

[tniezurawski]

By looking at the releases page, I think @Turbo87 and @rwjblue are maintainers of this package. What can we do to make this change happen?

[ef4]

Hi @tasha-urbancic and @tniezurawski, would either or both of you be willing to be added as maintainers to this package?

[tniezurawski]

@ef4 Yes, I'd like to 👍

[tasha-urbancic]

@ef4 same! 👍