New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run tests from other directories than src/main/scala #63

Open
tomqaz opened this Issue Jan 7, 2016 · 6 comments

Comments

Projects
None yet
6 participants
@tomqaz

tomqaz commented Jan 7, 2016

It is possible to keep tests for example in src/test/scala directory?

@ktoso

This comment has been minimized.

Show comment
Hide comment
@ktoso

ktoso Jan 7, 2016

Owner

What tests?
Benchmarks are not tests.

Technically you could, but I'd rather not suggest that (just override the settings)

Owner

ktoso commented Jan 7, 2016

What tests?
Benchmarks are not tests.

Technically you could, but I'd rather not suggest that (just override the settings)

@mheimann

This comment has been minimized.

Show comment
Hide comment
@mheimann

mheimann Jan 27, 2016

Which setting would one need to override for using a custom source directory that contains the benchmarks (such as src/benchmarks)? I've tried sourceDirectory in Jmh := ... and sourceDirectory in (Jmh, compile) := ... but none of that worked.

I guess moving the benchmark code into its own module/project is a decent solution as well, but that would require making it a multi-module sbt project for some people :-).

mheimann commented Jan 27, 2016

Which setting would one need to override for using a custom source directory that contains the benchmarks (such as src/benchmarks)? I've tried sourceDirectory in Jmh := ... and sourceDirectory in (Jmh, compile) := ... but none of that worked.

I guess moving the benchmark code into its own module/project is a decent solution as well, but that would require making it a multi-module sbt project for some people :-).

@jeffrey-aguilera

This comment has been minimized.

Show comment
Hide comment
@jeffrey-aguilera

jeffrey-aguilera May 13, 2016

The gradle plugin for JMH (https://github.com/melix/jmh-gradle-plugin) uses src/jmh in the same module.

jeffrey-aguilera commented May 13, 2016

The gradle plugin for JMH (https://github.com/melix/jmh-gradle-plugin) uses src/jmh in the same module.

@ktoso

This comment has been minimized.

Show comment
Hide comment
@ktoso

ktoso May 20, 2016

Owner

Hey there,
I agree, it's a good idea - would be breaking for many people though hm...
Reason for keeping them in a different project is to avoid compiling them if you don't work on on them.

Would you like to contrib this fix? I'm a bit overloaded right now?

Owner

ktoso commented May 20, 2016

Hey there,
I agree, it's a good idea - would be breaking for many people though hm...
Reason for keeping them in a different project is to avoid compiling them if you don't work on on them.

Would you like to contrib this fix? I'm a bit overloaded right now?

@magro

This comment has been minimized.

Show comment
Hide comment
@magro

magro Jul 5, 2016

Contributor

We're just integrating a previously external jmh project into our multimodule project, so that it's kept in sync with code changes and lower the barrier to run benchmarks after changes. Because our benchmarks depend on the test artifact of another module (which provides e.g. scalacheck generators) and Intellij does not support "compile->test" dependencies (it just ignores the dependency) we're also "forced" to integrate jmh into the test configuration.

Here's a "hack" that seems to do the trick:

In our multimodule build.sbt we have this dependency:

lazy val microbench = project.dependsOn(otherModule % "compile;compile->test").enablePlugins(JmhPlugin)

In microbench/build.sbt:

sourceDirectory in Jmh := (sourceDirectory in Test).value

// JmhPlugin.generateJmhSourcesAndResources uses (classDirectory in Compile).value
classDirectory in Compile := (classDirectory in Test).value

libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.1" // must be in Compile scope, otherwise JmhBytecodeGenerator fails with CNFE org/scalacheck/Gen

// Running a clean 'jmh:run' resulted in 'Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList'
// which could be fixed with a prior 'jmh:compile'. Here we rewire tasks to that a clean 'jmh:run' is possible...
Keys.compile in Jmh <<= (Keys.compile in Jmh) dependsOn (Keys.compile in Test)
run in Jmh <<= (run in Jmh) dependsOn (Keys.compile in Jmh)

Hopefully this helps others until we have a better solution.

Contributor

magro commented Jul 5, 2016

We're just integrating a previously external jmh project into our multimodule project, so that it's kept in sync with code changes and lower the barrier to run benchmarks after changes. Because our benchmarks depend on the test artifact of another module (which provides e.g. scalacheck generators) and Intellij does not support "compile->test" dependencies (it just ignores the dependency) we're also "forced" to integrate jmh into the test configuration.

Here's a "hack" that seems to do the trick:

In our multimodule build.sbt we have this dependency:

lazy val microbench = project.dependsOn(otherModule % "compile;compile->test").enablePlugins(JmhPlugin)

In microbench/build.sbt:

sourceDirectory in Jmh := (sourceDirectory in Test).value

// JmhPlugin.generateJmhSourcesAndResources uses (classDirectory in Compile).value
classDirectory in Compile := (classDirectory in Test).value

libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.1" // must be in Compile scope, otherwise JmhBytecodeGenerator fails with CNFE org/scalacheck/Gen

// Running a clean 'jmh:run' resulted in 'Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList'
// which could be fixed with a prior 'jmh:compile'. Here we rewire tasks to that a clean 'jmh:run' is possible...
Keys.compile in Jmh <<= (Keys.compile in Jmh) dependsOn (Keys.compile in Test)
run in Jmh <<= (run in Jmh) dependsOn (Keys.compile in Jmh)

Hopefully this helps others until we have a better solution.

magro added a commit to magro/sbt-jmh that referenced this issue Jul 5, 2016

Make classDirectory+dependencyClasspath configurable
As mentioned in [ktoso#63|ktoso#63 (comment)]
one can hack around the hardcoded config that is used for `classDirectory`
and `dependencyClasspath` (both are taken from the `Compile` config).

This change allows the user to set

```
classDirectory in Jmh := (classDirectory in Test).value
dependencyClasspath in Jmh := (dependencyClasspath in Test).value
```

so that jmh will use the appropriate values.

magro added a commit to magro/sbt-jmh that referenced this issue Jul 5, 2016

Make classDirectory+dependencyClasspath configurable
As mentioned in [ktoso#63](ktoso#63 (comment))
one can hack around the hardcoded config that is used for `classDirectory`
and `dependencyClasspath` (both are taken from the `Compile` config).

This change allows the user to set

```
classDirectory in Jmh := (classDirectory in Test).value
dependencyClasspath in Jmh := (dependencyClasspath in Test).value
```

so that jmh will use the appropriate values.

ktoso added a commit that referenced this issue Jul 5, 2016

Make classDirectory+dependencyClasspath configurable (#73)
As mentioned in [#63](#63 (comment))
one can hack around the hardcoded config that is used for `classDirectory`
and `dependencyClasspath` (both are taken from the `Compile` config).

This change allows the user to set

```
classDirectory in Jmh := (classDirectory in Test).value
dependencyClasspath in Jmh := (dependencyClasspath in Test).value
```

so that jmh will use the appropriate values.
@alexandrnikitin

This comment has been minimized.

Show comment
Hide comment
@alexandrnikitin

alexandrnikitin Nov 15, 2017

Contributor

It is possible to keep tests for example in src/test/scala directory?

To answer the OP's (@tomqaz) question, the following code in build.sbt worked for me:

sourceDirectory in Jmh := (sourceDirectory in Test).value
classDirectory in Jmh := (classDirectory in Test).value
dependencyClasspath in Jmh := (dependencyClasspath in Test).value
Contributor

alexandrnikitin commented Nov 15, 2017

It is possible to keep tests for example in src/test/scala directory?

To answer the OP's (@tomqaz) question, the following code in build.sbt worked for me:

sourceDirectory in Jmh := (sourceDirectory in Test).value
classDirectory in Jmh := (classDirectory in Test).value
dependencyClasspath in Jmh := (dependencyClasspath in Test).value

Atry added a commit to ThoughtWorksInc/Compute.scala that referenced this issue Nov 16, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment