Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Performance Optimization #1463

Merged
merged 25 commits into from May 15, 2020
Merged

Test Performance Optimization #1463

merged 25 commits into from May 15, 2020

Conversation

jeremydmiller
Copy link
Member

  • Introduced some new testing base classes to centralize integration testing
  • Trying to reuse a DocumentStore over most of the integration tests with IntegrationContext
  • Separated the Schema tests that tend to be slow and destructive of the database structure to Marten.Schema.Testing
  • Running the main test library with xUnit parallelization turned on
  • Selectively turns on Storyteller retry for some of the not perfectly reliable async daemon tests

I'm a little disappointed by the progress here, but I see a little better than a 25% reduction in execution time for the unit tests.

@@ -10,6 +12,9 @@ internal class MartenBuild
{
private const string BUILD_VERSION = "3.11.0";

private const string DockerConnectionString =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONCERN: That's minor, but wouldn't it be better to use the same config as in the pipeline tests? https://github.com/JasperFx/marten/pull/1463/files#diff-fec826feae04e51c0d94076385408bdcR36 It might be also good to use different database than default postgres as it might interfere with other user configuration.

UseDefaultSchema();
_store.Advanced.Clean.CompletelyRemoveAll();

#if NET461
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONCERN: Why is it needed?


public void Dispose()
{

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FIX: Redundant empty line.

Suggested change

@@ -78,9 +89,42 @@ private static void Main(string[] args)
Target("pack", DependsOn("compile"), ForEach("./src/Marten", "./src/Marten.CommandLine", "./src/Marten.NodaTime"), project =>
Run("dotnet", $"pack {project} -o ./../../artifacts --configuration Release"));

Target("init-db", () =>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I like that idea 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use that for something similar at work, but w/ Sql Server

Copy link
Collaborator

@oskardudycz oskardudycz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd lie if I said that I went through all of those files 🙃 I went through the base classes and some randomly choosen files. I added few comments but I like the idea 👍 I'll try to upgrade to .NET Core 3.1 now.

}

[Collection("integration")]
public class IntegrationContext : StoreContext<DefaultStoreFixture>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONCERN: Why do we need few IntegrationContexts? Can't we reuse them? If they have different purposes then it would be good if they have different naming.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a different integration context in the Schema project and yet another in the Noda, but they behave differently than the one, single context here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename them to eg. HarnessIntegrationContext etc.?


namespace Marten.NodaTime.Testing
{
public abstract class IntegrationContext : IDisposable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONCERN: Cannot we reuse the IntegrationContext class from the main project? Maybe we could create a separate base project with base classes to reuse them between different test projects?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't playing nicely here, and I was lazy. I don't see it doing any harm

@oskardudycz
Copy link
Collaborator

@jeremydmiller it would be also good to made the base classes abstract and their constructor protected ;)

@jeremydmiller
Copy link
Member Author

I think the base classes are all protected, and I'm saying no to making the constructor protected. That's super annoying 'cause R# tries to make the subclass constructors protected. Can we just get this thing in if the tests pass?

@jeremydmiller jeremydmiller merged commit 44f4e3c into master May 15, 2020
@jeremydmiller jeremydmiller deleted the tests branch May 15, 2020 11:36
@oskardudycz oskardudycz added the CI label May 19, 2020
@oskardudycz oskardudycz added this to the 3.12.0 milestone May 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants