Create an IntelliJ plug-in #2

helmbold · 2017-06-12T09:50:19Z

I think the single greatest step possible for KotlinTest would be adding a plug-in for IntelliJ. For example it is not possible to run a single test case from the open spec file.

If we decide to develop it, the development should probably happen in another Git repo. This issue is here just as a starting point.

ghost · 2017-07-19T07:30:51Z

What is the status of this issue?
I'd really love to use KotlinTest but as long as I am not able to execute single tests from IntelliJ I have to stick to JUnit.

elect86 · 2017-07-19T09:02:05Z

What do you mean exactly?

Because I am able to run single tests by pressing the circular arrow on the left side of the class test..

ghost · 2017-07-19T09:07:21Z

This runs all tests in the test class. But I cannot run one individual test.

KotlinTest	JUnit

elect86 · 2017-07-19T09:35:55Z

Ah sorry, I misunderstood single test with single class

sksamuel · 2017-07-19T13:47:07Z

I think in all honesty this plugin is unlikely to happen as it would be a hard task and I can't see intellij doing it when they have spek.

helmbold · 2017-07-20T19:58:13Z

@sksamuel Yes, I don't expect contributions from JetBreains, too - at least not before KotlinTests is used by a lot of JetBrains customers ... However maybe we can at least outline what have to be done to write such a plug-in and see then whether it would be possible or not.

elect86 · 2017-11-29T22:08:32Z

The best I came with at the moment is this

class obj : StringSpec() {

    val path = models + "/OBJ/"

    init {
        box(path + "box.obj")
        spider(path + "spider.obj")
        ...
    }
}

comment/uncomment as wished

LindsayPeters · 2018-04-15T01:31:19Z

I'm reverting back to JUnit. When you already have a lot of tests in the class, it just takes too long to debug the new one you are working on as all the others run too.

sksamuel · 2018-04-15T04:05:46Z

You can run the tests in a single class by clicking the green play icon by the classname. There's no need to run all the tests.

sksamuel · 2018-04-15T04:06:27Z

Although that won't help if you have 200 tests in a single file.

LindsayPeters · 2018-04-15T04:27:16Z

Exactly! Not 200 tests but enough that they may take a minute or more to run. The "workaround" is to put the new test at the top of the test class, then kill the test execution once that new test has run

kastork · 2018-04-15T13:48:04Z

Another workaround is to write all your tests with a config. Tests you're currently developing either don't get the config or get a different config until they graduate to being done. When the whole class is ready, swap the flag.

val INCLUDE_ALL = false  // or true if not working on a single test

init {
  test("developing test") {
    // ...
  }

  test("old test") {
    // ...
  }.config(enabled=INCLUDE_ALL)

  test("another old test") {
    // ...
  }.config(enabled=INCLUDE_ALL)
}

sksamuel · 2018-04-15T21:43:15Z

We could also add a feature like spec, whereby if you start the test name with "!" then that test is skipped. So you can just quickly prefix your tests with "!" except the one being worked on.

bbaldino · 2018-04-15T22:07:59Z

I was thinking something similar, but the opposite. Add something in front of the test name to only run that test.

sksamuel · 2018-04-15T22:19:06Z

we can do both! So "!" means do not run this test (ie, shorthand for config(enabled=false)) and "somethingelse" means run only this test.

What symbol / short text should be used for this.

bbaldino · 2018-04-15T22:36:41Z

~~Karma~~ Jasmine (JavaScript) was what made me think of it...it uses a prefix of "f" (I think for "focus"?) Which can be placed at any context to run only things within it.

EDIT: jasmine actually uses a different function for this (fdescribe vs describe), not 'f' in the description itself...which makes sense since that would probably conflict a lot...so something more along what you described (! for exclude, maybe @ or something for 'only this') would probably be better.

sksamuel · 2018-04-15T23:36:40Z

I've been thinking and the "test only" might be hard to do in kotlintest, as you have nested scopes, so it would need to execute all scopes to find the names before it knows if one is annotated with the 'test only'

LindsayPeters · 2018-04-16T12:10:03Z

Good comments and suggestions. However the problem with changing your test code to get a single test to run (like ddescribe or whatever), apart from the inconvenience, is that you run the risk of checking it in like that - then you have to think about commit-hooks and so it goes on...

helmbold · 2018-04-24T07:53:39Z

What is if the shortcut (!) is not in accordance with the explicit config (config(enabled = ...) )?

sksamuel · 2018-04-24T07:59:36Z

Good point. Throw a config exception?

…

On Tue, 24 Apr 2018, 08:53 Christian Helmbold, ***@***.***> wrote: What is if the shortcut (!) is not in accordance with the explicit config (config(enabled = ...))? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/kotlintest/kotlintest/issues/184#issuecomment-383838231>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGtf1W1zOWYOFu2ecTxB27Tdp4snGks5trtoDgaJpZM4N24pp> .

helmbold · 2018-04-26T09:31:46Z

I'm not sure if I like two ways to configure the same thing (though I have to admit that the ! is pretty handy). But if both options exist and are used inconsistently together, then, yes, I would throw an exception (UserTooStupidException obviously ;-) ).

sksamuel · 2018-04-26T13:57:14Z

Two ways is not ideal, but it's a nice timesaver. I suppose it's not as useful as the focus option, although I don't see how we could support the focus option.

bbaldino · 2018-04-26T17:39:45Z

Two ways is not ideal, but it's a nice timesaver. I suppose it's not as useful as the focus option, although I don't see how we could support the focus option.

Yeah that's a bummer as I agree the focus option seems the more useful of the two. Aren't all tests discovered first in the discovery phase? What if it was applied to the function call itself as opposed to the string description (i.e. fshould("do something...") instead of should("@do something..."). Though then I guess it couldn't be used on the string blocks.

sksamuel · 2018-04-26T17:46:51Z

Nested tests aren't discovered until the outer test is executed.

…

On Thu, 26 Apr 2018, 18:39 bbaldino, ***@***.***> wrote: Two ways is not ideal, but it's a nice timesaver. I suppose it's not as useful as the focus option, although I don't see how we could support the focus option. Yeah that's a bummer as the focus option seems the more useful of the two. Aren't all tests discovered first in the discovery phase? What if it was applied to the function call itself as opposed to the string description (i.e. fshould("do something...") instead of ***@***.*** something..."). Though then I guess it couldn't be used on the string blocks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/kotlintest/kotlintest/issues/184#issuecomment-384726735>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGntDF4iUHHVNv2URkXCBCHYrFAldks5tsgZigaJpZM4N24pp> .

kintomiko · 2018-07-04T10:16:16Z

our team use meet the same problem and
we worked around it by adding the helper method for the FeatureSpec

    fun qscenario(name: String, test: () -> Unit): TestCase{
        return scenario("[QuickRun] $name", test).tag(QuickRun)
    }

while QuickRun is a tag which is added as includeTags to Intellij Junit default config
The method return a TestCase to support the same config style at the end of the scenario.

All the above is done in 2.x.x version but >3.x.x version, you just can NOT have the extension qscenario return a TestCase and then apply config to it which make our workaround invalid anymore...

IceMimosa · 2018-07-28T07:10:26Z

How is the kotlintest-intellij plugin going? Thanks !!! 😜

sksamuel · 2018-07-28T10:01:42Z

slowly ;)

sksamuel · 2018-07-28T10:05:20Z

@kintomiko something like this should work.

    fun qscenario(name: String, test: TestContext.() -> Unit) = {
      context.registerTestCase(name, this@AbstractFeatureSpec, test, TestCaseConfig(tags = setOf(QuickTag)), TestType.Test)
    }

You might need to copy the AbstractFeatureSpec class and extend it to add this function.

ghostbuster91 · 2018-08-11T17:25:06Z

@sksamuel Hi some time ago I created an intellij idea plugin for other test framework, which does something similar I belive. I don't know if I will have time to develop another one for kotlintest but I think that you might find it helpful.
https://github.com/ghostbuster91/nspek-intellij-plugin
Let me know if you will have any questions

sksamuel · 2019-02-05T22:04:22Z

@blackmichael did you get chance to have a look ?

sksamuel · 2019-02-05T22:28:01Z

I've published an update for the plugin. It's pending jetbrains approval.

FeatureSpec support has now been added, and the double test output has been fixed.
@Kerooker

bbaldino · 2019-02-06T18:43:05Z

@sksamuel is there a list of which specs are supported by the plugin at this point?

sksamuel · 2019-02-06T18:47:17Z

Word string should behaviour feature fun

…

On Wed, 6 Feb 2019, 18:43 bbaldino ***@***.*** wrote: @sksamuel <https://github.com/sksamuel> is there a list of which specs are supported by the plugin at this point? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/kotlintest/kotlintest/issues/184#issuecomment-461138636>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGvv8Ycsfb0i3kAFqqPkkSA5tCblEks5vKyI5gaJpZM4N24pp> .

bbaldino · 2019-02-06T18:49:34Z

Thanks, I'm trying ShouldSpec but not seeing much difference, but I just noticed in the 3.3 release bug that 3.3 is required for the plugin to work?

sksamuel · 2019-02-06T19:18:52Z

Yes grab 3.3 snapshot from maven central snapshots

…

On Wed, 6 Feb 2019, 18:49 bbaldino ***@***.*** wrote: Thanks, I'm trying ShouldSpec but not seeing much difference, but I just noticed in the 3.3 release bug that 3.3 is required for the plugin to work? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/kotlintest/kotlintest/issues/184#issuecomment-461140926>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGqNMfVPEmAwng-MKP3A3g0kNV3pkks5vKyO_gaJpZM4N24pp> .

bbaldino · 2019-02-06T19:33:39Z

I did manage to get 3.3.0.1785-SNAPSHOT downloaded but still don't notice a difference in intellij. Should I be seeing the 'run' icons next to all the should blocks?

sksamuel · 2019-02-12T13:30:03Z

Yes, if you're running the 3.3.0.2 plugin, you should be able to see run icons for all tests in should spec, string spec, fun spec, behavior spec and word spec. Are you not ?

sksamuel · 2019-02-12T13:31:50Z

bbaldino · 2019-02-12T17:03:39Z

I'm using 3.3.0.2 plugin and kotlintest 3.3.0.1785-SNAPSHOT, but don't see the run icons for individual tests in ShouldSpec:

sksamuel · 2019-02-12T17:15:03Z

There's a bug I see where the outer context's don't work but the inner ones do for me.

Just to confirm whether it's a bug in the spec, or the entire plugin, can you create a StringSpec test just to see ?

bbaldino · 2019-02-12T17:18:32Z

StringSpec looks ok

sksamuel · 2019-02-12T17:29:48Z

Maybe 3.3.0.2 didn't work with should and I'm daydreaming. 3.3.0.3 is pending approval at the moment which is what I just used.

sksamuel · 2019-02-13T14:08:21Z

@bbaldino 3.3.0.4 has been approved do you want to try that ?

bbaldino · 2019-02-13T17:10:59Z

Just updated the plugin, I still don't see them for my existing tests. Weirdly, though, if I create a new test file they do show up (I tried closing and re-opening the pane for my existing test file, but they still don't show up there, and I had restarted IntelliJ after updating the plugin).

But, even with the new test file I notice that when I use the run button next to one test, all the tests are still run.

Clicking the play button on line 32, for example, runs all the tests.

sksamuel · 2019-02-13T17:26:27Z

Is there some output that says something like "xxxxx startsWith xxxxxx"

…

On Wed, 13 Feb 2019 at 17:11, bbaldino ***@***.***> wrote: Just updated the plugin, I still don't see them for my existing tests. Weirdly, though, if I create a new test file they do show up (I tried closing and re-opening the pane for my existing test file, but they still don't show up there, and I had restarted IntelliJ after updating the plugin). But, even with the new test file I notice that when I use the run button next to one test, all the tests are still run. [image: image] <https://user-images.githubusercontent.com/4016469/52729997-347d1a00-2f6f-11e9-9de5-8d78411ca650.png> Clicking the play button on line 32, for example, runs all the tests. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/kotlintest/kotlintest/issues/184#issuecomment-463284036>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGk95r0GjIzmpB7YiHuvoC1jymLO8ks5vNEcjgaJpZM4N24pp> .

sksamuel · 2019-02-13T17:27:02Z

As for your existing test - try adding a blank line somewhere to force reparse.

…

On Wed, 13 Feb 2019 at 17:26, Stephen Samuel (Sam) ***@***.***> wrote: Is there some output that says something like "xxxxx startsWith xxxxxx" On Wed, 13 Feb 2019 at 17:11, bbaldino ***@***.***> wrote: > Just updated the plugin, I still don't see them for my existing tests. > Weirdly, though, if I create a new test file they do show up (I tried > closing and re-opening the pane for my existing test file, but they still > don't show up there, and I had restarted IntelliJ after updating the > plugin). > > But, even with the new test file I notice that when I use the run button > next to one test, all the tests are still run. > > [image: image] > <https://user-images.githubusercontent.com/4016469/52729997-347d1a00-2f6f-11e9-9de5-8d78411ca650.png> > > Clicking the play button on line 32, for example, runs all the tests. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/kotlintest/kotlintest/issues/184#issuecomment-463284036>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAtZGk95r0GjIzmpB7YiHuvoC1jymLO8ks5vNEcjgaJpZM4N24pp> > . >

bbaldino · 2019-02-13T19:01:38Z

Is there some output that says something like "xxxxx startsWith xxxxxx"

I was seeing this when I first tried to run a single test, but I think since I tried running all the tests it no longer shows up (it just runs them all).

As for your existing test - try adding a blank line somewhere to force reparse.

Tried a few different edits but can't get them to show up.

bbaldino · 2019-02-13T19:31:00Z

I also just got a stacktrace from the kotlintest plugin:

java.lang.ArrayIndexOutOfBoundsException: 2
	at io.kotlintest.plugin.intellij.psi.UtilsKt.matchFunction2WithStringAndLambdaArgs(utils.kt:94)
	at io.kotlintest.plugin.intellij.psi.ShouldSpecStyle.tryShould(ShouldSpecStyle.kt:24)
	at io.kotlintest.plugin.intellij.psi.ShouldSpecStyle.testPath(ShouldSpecStyle.kt:35)
	at io.kotlintest.plugin.intellij.psi.ShouldSpecStyle.isTestElement(ShouldSpecStyle.kt:9)
	at io.kotlintest.plugin.intellij.KotlinTestRunLineMarkerContributor.getInfo(KotlinTestRunLineMarkerContributor.kt:22)
	at com.intellij.execution.lineMarker.RunLineMarkerProvider.getLineMarkerInfo(RunLineMarkerProvider.java:46)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.queryProviders(LineMarkersPass.java:171)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.lambda$doCollectInformation$3(LineMarkersPass.java:96)
	at com.intellij.codeInsight.daemon.impl.Divider.divideInsideAndOutsideInOneRoot(Divider.java:79)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.doCollectInformation(LineMarkersPass.java:91)
	at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:69)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$null$1(PassExecutorService.java:423)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1168)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$2(PassExecutorService.java:416)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:582)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:87)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.doRun(PassExecutorService.java:415)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$0(PassExecutorService.java:391)
	at com.intellij.openapi.application.impl.ReadMostlyRWLock.executeByImpatientReader(ReadMostlyRWLock.java:147)
	at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:222)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:389)
	at com.intellij.concurrency.JobLauncherImpl$VoidForkJoinTask$1.exec(JobLauncherImpl.java:161)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

sksamuel · 2019-02-13T21:04:57Z

That error is the same as #652 which I will fix now and release a fix as 3.3.0.5

yuan-raken · 2019-02-23T02:10:54Z

@sksamuel I tested 3.3.0.1901-SNAPSHOT, 3.3.0.1735-SNAPSHOT, 3.3.0.1785-SNAPSHOT, with the latest plugin (3.3.0.7) and still doesn't work for me.

    testCompile ('io.kotlintest:kotlintest-core:3.3.0.1901-SNAPSHOT')
    testCompile ('io.kotlintest:kotlintest-runner-junit5:3.3.0.1901-SNAPSHOT')

I can see the arrows but it doesn't matter which one I click, it runs all the cases in the spec for me

yuan-raken · 2019-02-23T02:19:34Z

Another a problem is once I extend the FeatureSpec to another layer, the arrow won't be shown for me as well.

sksamuel · 2019-02-25T07:51:06Z

This should all be fixed in 3.3.0.8. It is for me anyway :) You'll need to wait for Jetbrains to approve the update - takes 24-48hours then it will appear as an update in Intellij automatically.

…

On Sat, 23 Feb 2019 at 02:19, yuan-raken ***@***.***> wrote: Another a problem is once I extend the FeatureSpec to another layer, the arrow won't be shown for me as well. [image: image] <https://user-images.githubusercontent.com/45988138/53280391-64ea6280-377e-11e9-939f-deef87d27ce7.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtZGlFaTKquqCpUFX8M88WpVG40Lf5Pks5vQKU2gaJpZM4a_Cme> .

atreyas · 2020-02-04T02:08:20Z

I still cannot run single tests with 3.3.0.11. It seems to run all the tests on the IDE, although there are markers for each test.

sksamuel referenced this issue in kotest/kotest Mar 21, 2018

Initial progress on an IntelliJ plug-in #184

8b26734

sksamuel referenced this issue in kotest/kotest Feb 5, 2019

Fixed double console output #184

c5ca736

sksamuel referenced this issue in kotest/kotest Feb 5, 2019

Fixed feature spec style test name order #184

09e3c9b

sksamuel referenced this issue in kotest/kotest Feb 12, 2019

Added support for free, describe and expect specs to plugin #184

43cb758

sksamuel referenced this issue in kotest/kotest Feb 12, 2019

Fixed should spec code highlight when using config methods #184

3c709e0

sksamuel transferred this issue from kotest/kotest Feb 16, 2019

sksamuel closed this as completed Mar 2, 2019

Create an IntelliJ plug-in #2

Create an IntelliJ plug-in #2

Comments

helmbold commented Jun 12, 2017

ghost commented Jul 19, 2017

elect86 commented Jul 19, 2017

ghost commented Jul 19, 2017 • edited by ghost Loading

elect86 commented Jul 19, 2017

sksamuel commented Jul 19, 2017

helmbold commented Jul 20, 2017

elect86 commented Nov 29, 2017

LindsayPeters commented Apr 15, 2018

sksamuel commented Apr 15, 2018

sksamuel commented Apr 15, 2018

LindsayPeters commented Apr 15, 2018

kastork commented Apr 15, 2018

sksamuel commented Apr 15, 2018

bbaldino commented Apr 15, 2018

sksamuel commented Apr 15, 2018

bbaldino commented Apr 15, 2018 • edited Loading

sksamuel commented Apr 15, 2018

LindsayPeters commented Apr 16, 2018

helmbold commented Apr 24, 2018

sksamuel commented Apr 24, 2018 via email

helmbold commented Apr 26, 2018

sksamuel commented Apr 26, 2018

bbaldino commented Apr 26, 2018 • edited Loading

sksamuel commented Apr 26, 2018 via email

kintomiko commented Jul 4, 2018 • edited Loading

IceMimosa commented Jul 28, 2018

sksamuel commented Jul 28, 2018

sksamuel commented Jul 28, 2018

ghostbuster91 commented Aug 11, 2018

sksamuel commented Feb 5, 2019

sksamuel commented Feb 5, 2019

bbaldino commented Feb 6, 2019

sksamuel commented Feb 6, 2019 via email

bbaldino commented Feb 6, 2019

sksamuel commented Feb 6, 2019 via email

bbaldino commented Feb 6, 2019 • edited Loading

sksamuel commented Feb 12, 2019

sksamuel commented Feb 12, 2019

bbaldino commented Feb 12, 2019

sksamuel commented Feb 12, 2019

bbaldino commented Feb 12, 2019

sksamuel commented Feb 12, 2019

sksamuel commented Feb 13, 2019

bbaldino commented Feb 13, 2019

sksamuel commented Feb 13, 2019 via email

sksamuel commented Feb 13, 2019 via email

bbaldino commented Feb 13, 2019

bbaldino commented Feb 13, 2019

sksamuel commented Feb 13, 2019

yuan-raken commented Feb 23, 2019 • edited Loading

yuan-raken commented Feb 23, 2019

sksamuel commented Feb 25, 2019 via email

atreyas commented Feb 4, 2020

ghost commented Jul 19, 2017 •

edited by ghost

Loading

bbaldino commented Apr 15, 2018 •

edited

Loading

bbaldino commented Apr 26, 2018 •

edited

Loading

kintomiko commented Jul 4, 2018 •

edited

Loading

bbaldino commented Feb 6, 2019 •

edited

Loading

yuan-raken commented Feb 23, 2019 •

edited

Loading