Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide reusable base implementation of test discovery #1739

Closed
marcphilipp opened this issue Jan 20, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@marcphilipp
Copy link
Member

commented Jan 20, 2019

All class-based test engines should handle the following selectors during test discovery:

  • ClasspathResourceSelector
  • ClasspathRootSelector
  • ModuleSelector
  • PackageSelector
  • UniqueIdSelector
  • ClassSelector
  • MethodSelector

The code required to handle the first four of these is straightforward but repetitive and easy to forget. Moreover, resolving UniqueIdSelectors is non-trivial but the implementations of the Jupiter and Vintage engine provide examples of how to handle them correctly. Both use a similar yet slightly different approach. If we implemented the base algorithm once in an extensible way, it could be reused by other test engines. Doing so would reduce the burden on test engine authors and increase consistency across test engine implementations.

Deliverables

  • Extract an extendable base algorithm for resolving selectors from existing implementations
  • Provide extensible configuration mechanism
  • Provide sensible defaults for class-based test engines
  • Use base algorithm in Jupiter and Vintage
  • Reach out to authors of third-party test engines to get feedback and help them adopt the new base algorithm
@gopinath-langote

This comment has been minimized.

Copy link
Contributor

commented Feb 25, 2019

@marcphilipp anyone working on this?

@sbrannen

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

Yes, as you can see above to the right, it's assigned to @marcphilipp and labeled as "in progress".

@gopinath-langote

This comment has been minimized.

Copy link
Contributor

commented Feb 25, 2019

@sbrannen 👍

marcphilipp added a commit that referenced this issue Mar 3, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent and its designated children can be specified using
`DiscoverySelectors`. For example, in the case of the Jupiter engine,
when resolving a nested test class, there's a resolver that specifies
the parent using the `ClassSelector` of the enclosing class and the
children as the set of `MethodSelectors` of the testable methods it
contains joined with the set of `ClassSelectors` of nested classes it
contains.

The algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves #1739.

marcphilipp added a commit that referenced this issue Mar 3, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent and its designated children can be specified using
`DiscoverySelectors`. For example, in the case of the Jupiter engine,
when resolving a nested test class, there's a resolver that specifies
the parent using the `ClassSelector` of the enclosing class and the
children as the set of `MethodSelectors` of the testable methods it
contains joined with the set of `ClassSelectors` of nested classes it
contains.

Overall, the algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves #1739.

marcphilipp added a commit that referenced this issue Mar 3, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent of a `TestDescriptor` and its designated children can
be specified using `DiscoverySelectors`. For example, in the case of the
Jupiter engine, when resolving a nested test class, there's a resolver
that specifies the parent using the `ClassSelector` of the enclosing
class and the children as the set of `MethodSelectors` of the testable
methods it contains joined with the set of `ClassSelectors` of nested
classes it contains.

Overall, the algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves #1739.

marcphilipp added a commit that referenced this issue Mar 3, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent of a `TestDescriptor` and its designated children can
be specified using `DiscoverySelectors`. For example, in the case of the
Jupiter engine, when resolving a nested test class, there's a resolver
that specifies the parent using the `ClassSelector` of the enclosing
class and the children as the set of `MethodSelectors` of the testable
methods it contains joined with the set of `ClassSelectors` of nested
classes it contains.

Overall, the algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves #1739.

marcphilipp added a commit that referenced this issue Mar 3, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent of a `TestDescriptor` and its designated children can
be specified using `DiscoverySelectors`. For example, in the case of the
Jupiter engine, when resolving a nested test class, there's a resolver
that specifies the parent using the `ClassSelector` of the enclosing
class and the children as the set of `MethodSelectors` of the testable
methods it contains joined with the set of `ClassSelectors` of nested
classes it contains.

Overall, the algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves #1739.

@marcphilipp marcphilipp referenced this issue Mar 3, 2019

Merged

Introduce configurable discovery algorithm #1798

7 of 7 tasks complete

@ghost ghost added status: reviewing and removed status: in progress labels Mar 3, 2019

marcphilipp added a commit that referenced this issue Mar 7, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent of a `TestDescriptor` and its designated children can
be specified using `DiscoverySelectors`. For example, in the case of the
Jupiter engine, when resolving a nested test class, there's a resolver
that specifies the parent using the `ClassSelector` of the enclosing
class and the children as the set of `MethodSelectors` of the testable
methods it contains joined with the set of `ClassSelectors` of nested
classes it contains.

Overall, the algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves #1739.

@ghost ghost removed the status: reviewing label Mar 7, 2019

alixwar added a commit to alixwar/junit5 that referenced this issue Mar 10, 2019

Introduce configurable discovery algorithm
The `EngineDiscoveryRequestResolver` class implements a test discovery
in a way that can be reused by different test engines. The Jupiter and
Vintage test engines are changed to use it.

Main entry point is `EngineDiscoveryRequestResolver.builder()` which
allows to add resolvers (via a new `SelectorResolvers` interface) and
visitors (reusing `TestDescriptor.Visitor`) that will be called to
resolve `DiscoverySelectors` in a given `EngineDiscoveryRequest`.

In order to support class based test engines, the predefined
`ClassContainerSelectorResolver` may be added via the builder as well.
It resolves `ClasspathRootSelectors`, `ModuleSelectors`, and
`PackageSelectors` into `ClassSelectors` by scanning for classes in the
corresponding class container.

The fundamental principle behind `SelectorResolver` is that both the
designated parent of a `TestDescriptor` and its designated children can
be specified using `DiscoverySelectors`. For example, in the case of the
Jupiter engine, when resolving a nested test class, there's a resolver
that specifies the parent using the `ClassSelector` of the enclosing
class and the children as the set of `MethodSelectors` of the testable
methods it contains joined with the set of `ClassSelectors` of nested
classes it contains.

Overall, the algorithm works as follows:

1. Enqueue all selectors in the supplied request to be resolved.
2. While there are selectors to be resolved, get the next one.
   Otherwise, the resolution is finished.
   a. Iterate over all registered `SelectorResolvers` in the order they
      were registered in and find the first one that returns a non-empty
      resolution.
   b. If such a resolution exists, enqueue the contained selectors.
   c. For each exact match in the resolution, expand its children and
      enqueue them as well.
   d. Iterate over all registered visitors and let the engine descriptor
      accept them.

Descriptors with the same unique ID are only resolved once, even if
specified using multiple selectors.

Resolves junit-team#1739.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.