Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create an IntelliJ plug-in #2

Closed
helmbold opened this Issue Jun 12, 2017 · 68 comments

Comments

Projects
None yet
@helmbold
Copy link

helmbold commented Jun 12, 2017

I think the single greatest step possible for KotlinTest would be adding a plug-in for IntelliJ. For example it is not possible to run a single test case from the open spec file.

If we decide to develop it, the development should probably happen in another Git repo. This issue is here just as a starting point.

@ghost

This comment has been minimized.

Copy link

ghost commented Jul 19, 2017

What is the status of this issue?
I'd really love to use KotlinTest but as long as I am not able to execute single tests from IntelliJ I have to stick to JUnit.

@elect86

This comment has been minimized.

Copy link

elect86 commented Jul 19, 2017

What do you mean exactly?

Because I am able to run single tests by pressing the circular arrow on the left side of the class test..

@ghost

This comment has been minimized.

Copy link

ghost commented Jul 19, 2017

This runs all tests in the test class. But I cannot run one individual test.

KotlinTest JUnit
kotlintest junit
@elect86

This comment has been minimized.

Copy link

elect86 commented Jul 19, 2017

Ah sorry, I misunderstood single test with single class

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Jul 19, 2017

I think in all honesty this plugin is unlikely to happen as it would be a hard task and I can't see intellij doing it when they have spek.

@helmbold

This comment has been minimized.

Copy link
Author

helmbold commented Jul 20, 2017

@sksamuel Yes, I don't expect contributions from JetBreains, too - at least not before KotlinTests is used by a lot of JetBrains customers ... However maybe we can at least outline what have to be done to write such a plug-in and see then whether it would be possible or not.

@elect86

This comment has been minimized.

Copy link

elect86 commented Nov 29, 2017

The best I came with at the moment is this

class obj : StringSpec() {

    val path = models + "/OBJ/"

    init {
        box(path + "box.obj")
        spider(path + "spider.obj")
        ...
    }
}

comment/uncomment as wished

@LindsayPeters

This comment has been minimized.

Copy link

LindsayPeters commented Apr 15, 2018

I'm reverting back to JUnit. When you already have a lot of tests in the class, it just takes too long to debug the new one you are working on as all the others run too.

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 15, 2018

You can run the tests in a single class by clicking the green play icon by the classname. There's no need to run all the tests.

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 15, 2018

Although that won't help if you have 200 tests in a single file.

@LindsayPeters

This comment has been minimized.

Copy link

LindsayPeters commented Apr 15, 2018

Exactly! Not 200 tests but enough that they may take a minute or more to run. The "workaround" is to put the new test at the top of the test class, then kill the test execution once that new test has run

@kastork

This comment has been minimized.

Copy link

kastork commented Apr 15, 2018

Another workaround is to write all your tests with a config. Tests you're currently developing either don't get the config or get a different config until they graduate to being done. When the whole class is ready, swap the flag.

val INCLUDE_ALL = false  // or true if not working on a single test

init {
  test("developing test") {
    // ...
  }

  test("old test") {
    // ...
  }.config(enabled=INCLUDE_ALL)

  test("another old test") {
    // ...
  }.config(enabled=INCLUDE_ALL)
}
@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 15, 2018

We could also add a feature like spec, whereby if you start the test name with "!" then that test is skipped. So you can just quickly prefix your tests with "!" except the one being worked on.

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Apr 15, 2018

I was thinking something similar, but the opposite. Add something in front of the test name to only run that test.

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 15, 2018

we can do both! So "!" means do not run this test (ie, shorthand for config(enabled=false)) and "somethingelse" means run only this test.

What symbol / short text should be used for this.

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Apr 15, 2018

Karma Jasmine (JavaScript) was what made me think of it...it uses a prefix of "f" (I think for "focus"?) Which can be placed at any context to run only things within it.

EDIT: jasmine actually uses a different function for this (fdescribe vs describe), not 'f' in the description itself...which makes sense since that would probably conflict a lot...so something more along what you described (! for exclude, maybe @ or something for 'only this') would probably be better.

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 15, 2018

I've been thinking and the "test only" might be hard to do in kotlintest, as you have nested scopes, so it would need to execute all scopes to find the names before it knows if one is annotated with the 'test only'

@LindsayPeters

This comment has been minimized.

Copy link

LindsayPeters commented Apr 16, 2018

Good comments and suggestions. However the problem with changing your test code to get a single test to run (like ddescribe or whatever), apart from the inconvenience, is that you run the risk of checking it in like that - then you have to think about commit-hooks and so it goes on...

@helmbold

This comment has been minimized.

Copy link
Author

helmbold commented Apr 24, 2018

What is if the shortcut (!) is not in accordance with the explicit config (config(enabled = ...) )?

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 24, 2018

@helmbold

This comment has been minimized.

Copy link
Author

helmbold commented Apr 26, 2018

I'm not sure if I like two ways to configure the same thing (though I have to admit that the ! is pretty handy). But if both options exist and are used inconsistently together, then, yes, I would throw an exception (UserTooStupidException obviously ;-) ).

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 26, 2018

Two ways is not ideal, but it's a nice timesaver. I suppose it's not as useful as the focus option, although I don't see how we could support the focus option.

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Apr 26, 2018

Two ways is not ideal, but it's a nice timesaver. I suppose it's not as useful as the focus option, although I don't see how we could support the focus option.

Yeah that's a bummer as I agree the focus option seems the more useful of the two. Aren't all tests discovered first in the discovery phase? What if it was applied to the function call itself as opposed to the string description (i.e. fshould("do something...") instead of should("@do something..."). Though then I guess it couldn't be used on the string blocks.

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Apr 26, 2018

@kintomiko

This comment has been minimized.

Copy link

kintomiko commented Jul 4, 2018

our team use meet the same problem and
we worked around it by adding the helper method for the FeatureSpec

    fun qscenario(name: String, test: () -> Unit): TestCase{
        return scenario("[QuickRun] $name", test).tag(QuickRun)
    }

while QuickRun is a tag which is added as includeTags to Intellij Junit default config
The method return a TestCase to support the same config style at the end of the scenario.

All the above is done in 2.x.x version but >3.x.x version, you just can NOT have the extension qscenario return a TestCase and then apply config to it which make our workaround invalid anymore...

@IceMimosa

This comment has been minimized.

Copy link

IceMimosa commented Jul 28, 2018

How is the kotlintest-intellij plugin going? Thanks !!! 😜

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Jul 28, 2018

slowly ;)

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Jul 28, 2018

@kintomiko something like this should work.

    fun qscenario(name: String, test: TestContext.() -> Unit) = {
      context.registerTestCase(name, this@AbstractFeatureSpec, test, TestCaseConfig(tags = setOf(QuickTag)), TestType.Test)
    }

You might need to copy the AbstractFeatureSpec class and extend it to add this function.

@ghostbuster91

This comment has been minimized.

Copy link

ghostbuster91 commented Aug 11, 2018

@sksamuel Hi some time ago I created an intellij idea plugin for other test framework, which does something similar I belive. I don't know if I will have time to develop another one for kotlintest but I think that you might find it helpful.
https://github.com/ghostbuster91/nspek-intellij-plugin
Let me know if you will have any questions

sksamuel referenced this issue in kotlintest/kotlintest Feb 5, 2019

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 5, 2019

@blackmichael did you get chance to have a look ?

sksamuel referenced this issue in kotlintest/kotlintest Feb 5, 2019

sksamuel referenced this issue in kotlintest/kotlintest Feb 5, 2019

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 5, 2019

I've published an update for the plugin. It's pending jetbrains approval.

FeatureSpec support has now been added, and the double test output has been fixed.
@Kerooker

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 6, 2019

@sksamuel is there a list of which specs are supported by the plugin at this point?

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 6, 2019

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 6, 2019

Thanks, I'm trying ShouldSpec but not seeing much difference, but I just noticed in the 3.3 release bug that 3.3 is required for the plugin to work?

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 6, 2019

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 6, 2019

I did manage to get 3.3.0.1785-SNAPSHOT downloaded but still don't notice a difference in intellij. Should I be seeing the 'run' icons next to all the should blocks?

sksamuel referenced this issue in kotlintest/kotlintest Feb 12, 2019

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 12, 2019

Yes, if you're running the 3.3.0.2 plugin, you should be able to see run icons for all tests in should spec, string spec, fun spec, behavior spec and word spec. Are you not ?

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 12, 2019

image

image

image

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 12, 2019

I'm using 3.3.0.2 plugin and kotlintest 3.3.0.1785-SNAPSHOT, but don't see the run icons for individual tests in ShouldSpec:

image

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 12, 2019

There's a bug I see where the outer context's don't work but the inner ones do for me.
image

Just to confirm whether it's a bug in the spec, or the entire plugin, can you create a StringSpec test just to see ?

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 12, 2019

StringSpec looks ok

image

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 12, 2019

Maybe 3.3.0.2 didn't work with should and I'm daydreaming. 3.3.0.3 is pending approval at the moment which is what I just used.

sksamuel referenced this issue in kotlintest/kotlintest Feb 12, 2019

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 13, 2019

@bbaldino 3.3.0.4 has been approved do you want to try that ?

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 13, 2019

Just updated the plugin, I still don't see them for my existing tests. Weirdly, though, if I create a new test file they do show up (I tried closing and re-opening the pane for my existing test file, but they still don't show up there, and I had restarted IntelliJ after updating the plugin).

But, even with the new test file I notice that when I use the run button next to one test, all the tests are still run.

image

Clicking the play button on line 32, for example, runs all the tests.

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 13, 2019

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 13, 2019

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 13, 2019

Is there some output that says something like "xxxxx startsWith xxxxxx"

I was seeing this when I first tried to run a single test, but I think since I tried running all the tests it no longer shows up (it just runs them all).

As for your existing test - try adding a blank line somewhere to force reparse.

Tried a few different edits but can't get them to show up.

@bbaldino

This comment has been minimized.

Copy link

bbaldino commented Feb 13, 2019

I also just got a stacktrace from the kotlintest plugin:

java.lang.ArrayIndexOutOfBoundsException: 2
	at io.kotlintest.plugin.intellij.psi.UtilsKt.matchFunction2WithStringAndLambdaArgs(utils.kt:94)
	at io.kotlintest.plugin.intellij.psi.ShouldSpecStyle.tryShould(ShouldSpecStyle.kt:24)
	at io.kotlintest.plugin.intellij.psi.ShouldSpecStyle.testPath(ShouldSpecStyle.kt:35)
	at io.kotlintest.plugin.intellij.psi.ShouldSpecStyle.isTestElement(ShouldSpecStyle.kt:9)
	at io.kotlintest.plugin.intellij.KotlinTestRunLineMarkerContributor.getInfo(KotlinTestRunLineMarkerContributor.kt:22)
	at com.intellij.execution.lineMarker.RunLineMarkerProvider.getLineMarkerInfo(RunLineMarkerProvider.java:46)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.queryProviders(LineMarkersPass.java:171)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.lambda$doCollectInformation$3(LineMarkersPass.java:96)
	at com.intellij.codeInsight.daemon.impl.Divider.divideInsideAndOutsideInOneRoot(Divider.java:79)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.doCollectInformation(LineMarkersPass.java:91)
	at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:69)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$null$1(PassExecutorService.java:423)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1168)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$2(PassExecutorService.java:416)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:582)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:87)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.doRun(PassExecutorService.java:415)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$0(PassExecutorService.java:391)
	at com.intellij.openapi.application.impl.ReadMostlyRWLock.executeByImpatientReader(ReadMostlyRWLock.java:147)
	at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:222)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:389)
	at com.intellij.concurrency.JobLauncherImpl$VoidForkJoinTask$1.exec(JobLauncherImpl.java:161)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 13, 2019

That error is the same as #652 which I will fix now and release a fix as 3.3.0.5

@sksamuel sksamuel transferred this issue from kotlintest/kotlintest Feb 16, 2019

@yuan-raken

This comment has been minimized.

Copy link

yuan-raken commented Feb 23, 2019

@sksamuel I tested 3.3.0.1901-SNAPSHOT, 3.3.0.1735-SNAPSHOT, 3.3.0.1785-SNAPSHOT, with the latest plugin (3.3.0.7) and still doesn't work for me.

    testCompile ('io.kotlintest:kotlintest-core:3.3.0.1901-SNAPSHOT')
    testCompile ('io.kotlintest:kotlintest-runner-junit5:3.3.0.1901-SNAPSHOT')

image

I can see the arrows but it doesn't matter which one I click, it runs all the cases in the spec for me

@yuan-raken

This comment has been minimized.

Copy link

yuan-raken commented Feb 23, 2019

Another a problem is once I extend the FeatureSpec to another layer, the arrow won't be shown for me as well.

image

@sksamuel

This comment has been minimized.

Copy link
Member

sksamuel commented Feb 25, 2019

@sksamuel sksamuel closed this Mar 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.