Focus option for top level tests

[sksamuel]

Prefix a test with f: (top level only) and that test (or group of tests) will be the only ones executed.

[bbaldino]

This is awesome!

[sksamuel]

It's in 3.1.1 if you want to use it :)

[dave08]

@sksamuel Does this work in all test types?

[sksamuel]

Should work for all, but only top level tests. @dave08

[dave08]

It would be great if it could work for groupings and top level tests, since in KotlinTest there's no IDE arrows like in Spek and JUnit besides each test, so one has to run the whole class or disable all the other tests just to concentrate on the one test he/she's developing... @sksamuel Or is there another way to do that? Also, you mean that test("f:Test one") { } works? Or does one have to provide it through options? Also, the ! is a convenient shortcut even for other test types, does the user absolutely need the options() function to provide it?

[sksamuel]

If you put f: in front of your test name then only that test (and nested tests under it) will be executed @dave08
Eg,

See

https://github.com/kotlintest/kotlintest/blob/master/kotlintest-tests/kotlintest-tests-core/src/test/kotlin/com/sksamuel/kotlintest/FocusTest.kt

I'm not sure what you mean by your second comment about ! and options.

[dave08]

Say:

class FooTest : DescribeSpec({
	describe("A foo") {
		context("!Planned feature (not yet working on this)") {
			// ... All the tests are skipped/disabled under here because of the ! prefix
		}

		context("Feature already finished") {
			// ... These tests should run when there are no `f:`s
		}

		context("f:Current feature") {
			// ... Only these tests should run as long as there's `f:` here
		}
	}
})

@sksamuel

[sksamuel]

I see, no you can't do that. The reason is, inner tests are not known until runtime. So it would break the way the test runners work.

You can put the f: on the describe block.

[dave08]

So how does providing the options() work in it(), one could disable the test there... if when adding the test you check the label string to make the same decision it would be possible no? At least for !... I don't think I saw a parameter for f: in the options(), maybe that's what you mean is impossible? I was actually wondering why there's no focus param there... @sksamuel

I suppose the multi-level f: or focus on option is too hard to currently implement, so I shouldn't open a seperate issue about this feature (this is probably the wrong issue to be discussing this know that I understand the title better...)?

I feel this feature is pretty important given that there aren't any arrows to use to run individual cases in the IDE. Maybe this could be implemented under the hood using one test per class with an internal marking that other tests should not be run?

[sksamuel]

In order to determine which test to run, it has to check all of them in the same scope.

If you do:

  test("test 1") { }
  test("f: test 2") { }

Then test 1 needs to be skipped because you've focused test 2. So it needs to scan ahead. We can't do this in nested scopes as the test closures are executed lazily, so that variable initialisation etc, works properly.

[dave08]

So maybe have a f2: on each level scope to focus on the second (or whatever) item on the level under it? A bit harder to use, but may be easier to know at run time what to run @sksamuel.

Sorry for insisting, its fine if you just say its not possible and thats it, I'm just trying to see if there's a workable solution, I think its a nice addition to a dev's workflow.

[sksamuel]

That's not the problem.
The problem is that you have to look ahead. If you mark test 20 with f: or f2, then the engine has to skip the first 19. It can only know to do this if it has already scanned the complete list of test names.

[dave08]

I mean f2 would be on the parent to indicate the second test or group under it is focused so you know before running what's under it

[sksamuel]

You mean f2 would indicate the 2nd test was focused, and f3 the third test and so on?

[dave08]

Yes, the second test under the indicated scope, but a clearer naming might be chosen to indicate not focusing on the item with the f but its child maybe f-2: (like the freespec group indicator - ) or 2> or >2 ?

It might get complicated in parameterized tests, I don't know if it would be possible to run the whole parameter set (not just the 4th one for eg.), which makes I think more sense in the devs workflow.

[TAGC]

I see, no you can't do that. The reason is, inner tests are not known until runtime. So it would break the way the test runners work.

Yeah, this is pretty annoying. Using DescribeSpec, I need to literally comment out every test but the one I want to focus on in order to isolate it.

[LeoColman]

I believe that with the intellij plugin, this will be somewhat easier, right?

…

On Wed., Feb. 20, 2019, 09:03 David ***@***.*** wrote: I see, no you can't do that. The reason is, inner tests are not known until runtime. So it would break the way the test runners work. Yeah, this is pretty annoying. Using DescribeSpec, I need to literally comment out every test but the one I want to focus on in order to isolate it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#329 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABgRIxGx0Qs4oQCuLWujc-4tovW8WRYnks5vPTmdgaJpZM4T9oLO> .

[sksamuel]

Yes, you can run in the inner test with the plugin. It will actually make focus a bit redundant. On Wed, 20 Feb 2019 at 12:05, Leonardo Colman Lopes < notifications@github.com> wrote:

…

I believe that with the intellij plugin, this will be somewhat easier, right? On Wed., Feb. 20, 2019, 09:03 David ***@***.*** wrote: > I see, no you can't do that. The reason is, inner tests are not known > until runtime. So it would break the way the test runners work. > > Yeah, this is pretty annoying. Using DescribeSpec, I need to literally > comment out every test but the one I want to focus on in order to isolate > it. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > < #329 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ABgRIxGx0Qs4oQCuLWujc-4tovW8WRYnks5vPTmdgaJpZM4T9oLO > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#329 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGtrRbhFs8MPrId_ad6dJ30Xc93HQks5vPTnygaJpZM4T9oLO> .

[TAGC]

Oh, I didn't realise there was a plugin, that's neat.

I just installed it and now notice there's arrows in the IDE beside each individual test case (within a DescribeScope). However, clicking any of them just runs all the test cases, the same as if I click the arrow beside the top-level "describe(...)". Should the plugin support running individual nested test cases?

This is what I mean:

[LeoColman]

It will actually make focus a bit redundant.

Not redundant if you consider people still use eclipse

[sksamuel]

Do they? In 2019 :) Yes fair point.

…

On Wed, 20 Feb 2019 at 12:34, Leonardo Colman Lopes < ***@***.***> wrote: It will actually make focus a bit redundant. Not redundant if you consider people still use eclipse — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#329 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGqh_MmVWL2cMz7DBCargTvgBkVxSks5vPUDZgaJpZM4T9oLO> .

[sksamuel]

@TAGC You're the second person to say this. Although the "run" tool tip is wrong (I don't know how to fix that currently), for me it does only run the correct single test. Can you try it, and take a screenshot of the run configuration page.

[TAGC]

Sure, the run configuration (this is used regardless of which green arrow I click):

And here's the test output pane:

Here's the relevant logging output:

40 [main] DEBUG KotlinTestEngine - configurationParameters=LauncherConfigurationParameters []
40 [main] DEBUG KotlinTestEngine - uniqueId=[engine:kotlintest]
60 [main] DEBUG KotlinTestEngine - LauncherDiscoveryRequest [
classpathRootSelectors=[]
classpathResourceSelectors=[]
classSelectors=[com.mycompany.testmaster.domain.nodes.CompositeNodeTests]
methodSelectors=[]
directorySelectors=[]
fileSelectors=[]
moduleSelectors=[]
packageSelectors=[]
uniqueIdSelectors=[]
uriSelectors=[]
engineFilters=[]
postDiscoveryFilters=[]
classnameFilters=[]
packageNameFilters=[]
]
120 [main] DEBUG TestDiscovery - Loaded 1 classes from classnames...
395 [main] DEBUG TestDiscovery - ...which has filtered to 1 non abstract classes
395 [main] DEBUG TestDiscovery - 1 classes after applying discovery extensions405 [main] DEBUG KotlinTestEngine - configurationParameters=LauncherConfigurationParameters []
405 [main] DEBUG KotlinTestEngine - uniqueId=[engine:kotlintest]
405 [main] DEBUG KotlinTestEngine - LauncherDiscoveryRequest [
classpathRootSelectors=[]
classpathResourceSelectors=[]
classSelectors=[com.mycompany.testmaster.domain.nodes.CompositeNodeTests]
methodSelectors=[]
directorySelectors=[]
fileSelectors=[]
moduleSelectors=[]
packageSelectors=[]
uniqueIdSelectors=[]
uriSelectors=[]
engineFilters=[]
postDiscoveryFilters=[]
classnameFilters=[]
packageNameFilters=[]
]
415 [main] DEBUG KotlinTestEngine - JUnit execution request [configurationParameters=LauncherConfigurationParameters []; rootTestDescriptor=KotlinTestEngineDescriptor: [engine:kotlintest]]
415 [main] DEBUG JUnitTestRunnerListener - Engine started; classes=[[class com.mycompany.testmaster.domain.nodes.CompositeNodeTests]]
415 [main] DEBUG TestEngine - Submitting 1 specs
415 [main] DEBUG TestEngine - Waiting for spec execution service to terminate
437 [kotlintest-engine-0] DEBUG JUnitTestRunnerListener - prepareSpec [Description(parents=[], name=CompositeNodeTests)]447 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Creating test case descriptor Description(parents=[CompositeNodeTests], name=Describe: Composite node)/Container
457 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Notifying junit of start event [engine:kotlintest]/[spec:CompositeNodeTests]/[test:Describe%3A Composite node]457 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Creating test case descriptor Description(parents=[CompositeNodeTests, Describe: Composite node], name=Scenario: immediately executes once and terminates if it has no inputs and no outputs)/Test
457 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Notifying junit of start event [engine:kotlintest]/[spec:CompositeNodeTests]/[test:Describe%3A Composite node]/[test:Scenario%3A immediately executes once and terminates if it has no inputs and no outputs]
634 [kotlintest-engine-0] DEBUG JUnitTestRunnerListener - Notifying junit of test case completion [engine:kotlintest]/[spec:CompositeNodeTests]/[test:Describe%3A Composite node]=TestResult(status=Failure, error=java.lang.AssertionError: Expected exception java.lang.IllegalStateException but no exception was thrown, reason=null, metaData={})

[sksamuel]

Thanks. It's not actually created it as a kotlintest run but a standard junit 5. Is there an option to create a KT test manually if you click the plus icon ?

…

On Wed, 20 Feb 2019 at 12:43, David ***@***.***> wrote: Sure, the run configuration: [image: idea64_2019-02-20_12-39-08] <https://user-images.githubusercontent.com/1588951/53092489-d58a4880-350c-11e9-8db5-4a2663eea7f1.png> And here's the test output pane: [image: idea64_2019-02-20_12-40-40] <https://user-images.githubusercontent.com/1588951/53092498-d7540c00-350c-11e9-9d5c-a75756df262f.png> Here's the relevant logging output: 40 [main] DEBUG KotlinTestEngine - configurationParameters=LauncherConfigurationParameters [] 40 [main] DEBUG KotlinTestEngine - uniqueId=[engine:kotlintest] 60 [main] DEBUG KotlinTestEngine - LauncherDiscoveryRequest [ classpathRootSelectors=[] classpathResourceSelectors=[] classSelectors=[com.mycompany.testmaster.domain.nodes.CompositeNodeTests] methodSelectors=[] directorySelectors=[] fileSelectors=[] moduleSelectors=[] packageSelectors=[] uniqueIdSelectors=[] uriSelectors=[] engineFilters=[] postDiscoveryFilters=[] classnameFilters=[] packageNameFilters=[] ] 120 [main] DEBUG TestDiscovery - Loaded 1 classes from classnames... 395 [main] DEBUG TestDiscovery - ...which has filtered to 1 non abstract classes 395 [main] DEBUG TestDiscovery - 1 classes after applying discovery extensions405 [main] DEBUG KotlinTestEngine - configurationParameters=LauncherConfigurationParameters [] 405 [main] DEBUG KotlinTestEngine - uniqueId=[engine:kotlintest] 405 [main] DEBUG KotlinTestEngine - LauncherDiscoveryRequest [ classpathRootSelectors=[] classpathResourceSelectors=[] classSelectors=[com.mycompany.testmaster.domain.nodes.CompositeNodeTests] methodSelectors=[] directorySelectors=[] fileSelectors=[] moduleSelectors=[] packageSelectors=[] uniqueIdSelectors=[] uriSelectors=[] engineFilters=[] postDiscoveryFilters=[] classnameFilters=[] packageNameFilters=[] ] 415 [main] DEBUG KotlinTestEngine - JUnit execution request [configurationParameters=LauncherConfigurationParameters []; rootTestDescriptor=KotlinTestEngineDescriptor: [engine:kotlintest]] 415 [main] DEBUG JUnitTestRunnerListener - Engine started; classes=[[class com.mycompany.testmaster.domain.nodes.CompositeNodeTests]] 415 [main] DEBUG TestEngine - Submitting 1 specs 415 [main] DEBUG TestEngine - Waiting for spec execution service to terminate 437 [kotlintest-engine-0] DEBUG JUnitTestRunnerListener - prepareSpec [Description(parents=[], name=CompositeNodeTests)]447 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Creating test case descriptor Description(parents=[CompositeNodeTests], name=Describe: Composite node)/Container 457 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Notifying junit of start event [engine:kotlintest]/[spec:CompositeNodeTests]/[test:Describe%3A Composite node]457 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Creating test case descriptor Description(parents=[CompositeNodeTests, Describe: Composite node], name=Scenario: immediately executes once and terminates if it has no inputs and no outputs)/Test 457 [kotlintest-test-executor-0] DEBUG JUnitTestRunnerListener - Notifying junit of start event [engine:kotlintest]/[spec:CompositeNodeTests]/[test:Describe%3A Composite node]/[test:Scenario%3A immediately executes once and terminates if it has no inputs and no outputs] 634 [kotlintest-engine-0] DEBUG JUnitTestRunnerListener - Notifying junit of test case completion [engine:kotlintest]/[spec:CompositeNodeTests]/[test:Describe%3A Composite node]=TestResult(status=Failure, error=java.lang.AssertionError: Expected exception java.lang.IllegalStateException but no exception was thrown, reason=null, metaData={}) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#329 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGtoHHAQ92hpilLQJoZNcz2_A18Z-ks5vPULwgaJpZM4T9oLO> .