Fix incremental compilation of runtime/test #8975

Akirathan · 2024-02-05T16:06:19Z

Pull Request Description

Fixes incremental compilation issues of runtime/Test by introducing a separate runtime-integration-tests project. Also move runtime benchmarks to a new runtime-benchmarks project and provide an easier way how to run the benchmarks.

For now, let's move all the tests from runtime/Test to runtime-integration-tests, just to be sure that all the tests run with all the dependencies. In the future, we can add standard unit tests, that do not require the Enso runtime, to runtime/Test.

Important Notes

Introduced runtime-benchmarks project
- Contains all benchmarks previously from runtime/bench.
- Refactored some Scala bits to Java
Introduced runtime-integration-tests project
- Contains all tests previously from runtime/test.
Quality of life improvements to the benchmark running.
- bench and benchOnly commands work as they used to.
- All source code is under src/main/java, no more bench directory.
- There is a main class that delegates to the JMH launcher, you can use run -h to see all the available cmd line options for the launcher.
- No more RegressionTest.scala.
- See the updated benchmarks.md.
Simplification of std-benchmarks project
- Remove the Bench configuration
- Use JPMSPlugin.

How I tested the changes?

Incremental compilation of runtime-tests tested by:

Modification of sources in pkg, runtime, and runtime-instrument-common and then re-running runtime-tests/testOnly.

Tested other commands:

sbt:runtime-integration-tests> withDebug --debugger
sbt:runtime-benchmarks> run - i 1 -wi 1 org.enso.interpreter.bench.benchmarks.semantic.AtomBenchmarks.benchSumList

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

Make sure there are no regressions in the engine benchmarks

Akirathan · 2024-02-05T16:12:30Z

project/JPMSPlugin.scala

@@ -114,8 +114,6 @@ object JPMSPlugin extends AutoPlugin {
        )
    }.toSeq

-    ensureDirectoriesExist(modulePath, log)


These warnings are annoying and they are not useful. Java does not mandate that the directories passed into, e.g., --module-path option exist.

Akirathan · 2024-02-08T13:14:38Z

...untime-integration-tests/src/test/java/org/enso/interpreter/test/asserts/AssertionsTest.java

@@ -49,6 +49,17 @@ public void resetOutput() {
    out.reset();
  }

+  @Test
+  public void jvmAssertionsAreEnabled() {


I have spent ridiculous amount of time debugging RuntimeErrorsTest just to realize that I have not enabled JVM assertions, so the DataFlow errors did not contain stack traces. Let's add this test so that nobody encounters something similar in the future.

mwu-tow

Reviewed Rust parts only. 🟢

JaroslavTulach

Hard to review such a huge shuffling of code, but the intentions I have spotted are good.

JaroslavTulach · 2024-02-08T18:19:57Z

build.sbt

@@ -1650,109 +1617,6 @@ lazy val runtime = (project in file("engine/runtime"))
      necessaryModules ++ langs ++ tools
    }
  )
-  .settings(


I was hoping the setup of runtime project will be simpler. Removal of these lines matches my expectations.

JaroslavTulach · 2024-02-08T18:20:40Z

build.sbt

+  * the `org.enso.runtime` JPMS module, so it is easier to keep them in a separate project.
+  * For standard unit tests, use `runtime/Test`.
+  */
+lazy val `runtime-integration-tests` =


Configuration of this project remains complicated, but at the end, they are integration tests - they can be complicated.

...-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/AtomBenchmarks.java

JaroslavTulach · 2024-02-12T14:19:21Z

build.sbt

@@ -1869,7 +1869,7 @@ lazy val `runtime-benchmarks` =
          (Compile / run).toTask("").tag(Exclusive).value
        }
        .dependsOn(
-          `runtime-fat-jar` / assembly
+          buildEngineDistribution


Akirathan · 2024-02-12T15:25:58Z

Scheduling engine benchmarks at https://github.com/enso-org/enso/actions/runs/7873980506/job/21482523287, and stdlib benchmarks at https://github.com/enso-org/enso/actions/runs/7873983053

GitHub
Benchmark Engine · enso-org/enso@f873840
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.

GitHub
Benchmark Standard Libraries · enso-org/enso@f873840
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.

build.sbt

Akirathan · 2024-02-12T18:27:24Z

Encountered some problems with bench-report.xml uploading in the last bench runs. Starting a new engine bench at https://github.com/enso-org/enso/actions/runs/7875921777/job/21488889969

GitHub
Benchmark Engine · enso-org/enso@00fb45e
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.

hubertp

LGTM (sorry for late review)

Akirathan self-assigned this Feb 5, 2024

Akirathan commented Feb 5, 2024

View reviewed changes

Akirathan force-pushed the wip/akirathan/8968-fix-runtime-test-build branch from c8e91a3 to 7e934d9 Compare February 5, 2024 17:08

enso-bot bot mentioned this pull request Feb 6, 2024

Incremental compilation of runtime/test is broken #8973

Closed

Akirathan force-pushed the wip/akirathan/8968-fix-runtime-test-build branch from c496ae9 to 0997cf2 Compare February 7, 2024 15:32

Akirathan added CI: No changelog needed Do not require a changelog entry for this PR. CI: Clean build required CI runners will be cleaned before and after this PR is built. labels Feb 7, 2024

Akirathan commented Feb 8, 2024

View reviewed changes

Akirathan marked this pull request as ready for review February 8, 2024 13:14

Akirathan requested review from jdunkerley, radeusgd, GregoryTravis, AdRiley, 4e6, JaroslavTulach, hubertp and mwu-tow as code owners February 8, 2024 13:14

mwu-tow approved these changes Feb 8, 2024

View reviewed changes

Akirathan added 12 commits February 8, 2024 16:55

Move all runtime/test to runtime-tests

3fce708

Add a new runtime-tests project

77861af

Add runtime-tests to the enso aggregate

a3a2b2d

Add some GraalVM dependencies to runtime-tests

5c97fcc

Remove warning about non-existing directories for module-path

7673b1f

Move all runtime/bench to runtime-benchmarks

51895ef

Create new runtime-benchmarks project

a94096b

Remove most of Scala code from runtime-benchmarks

69b771e

Add necessary dependencies to runtime-benchmarks project

4a5f3da

Fix bench runtime problems

e27ac91

Add withDebug command to runtime-benchmarks

716fa4c

Make sure Rust build script invokes runtime-benchmarks

664cf7f

Akirathan added 2 commits February 8, 2024 16:55

Rename runtime-tests to runtime-integration-tests

16fba0d

Add JVM assertions are enabled test

5d57cc2

Akirathan force-pushed the wip/akirathan/8968-fix-runtime-test-build branch from ec3f437 to 5d57cc2 Compare February 8, 2024 15:56

JaroslavTulach approved these changes Feb 8, 2024

View reviewed changes

4e6 approved these changes Feb 9, 2024

View reviewed changes

Akirathan added 4 commits February 9, 2024 15:29

Merge branch 'develop' into wip/akirathan/8968-fix-runtime-test-build

63b1c7b

Remove Bench configuration from std-benchmarks

c36b8fa

std-benchmarks depends on the runtime-fat-jar assembly

de7fd6a

fmt

6ed5c85

radeusgd mentioned this pull request Feb 10, 2024

Check type of self in static dispatch #8867

Merged

6 tasks

Akirathan added 4 commits February 12, 2024 11:15

Fix --add-exports for org.slf4j.nop provider

f25de91

runtime-benchmarks/javaOptions does not take value from std-benchmarks.

919606d

fmt

68a6bf7

Merge branch 'develop' into wip/akirathan/8968-fix-runtime-test-build

d1618f6

Akirathan linked an issue Feb 12, 2024 that may be closed by this pull request

Fix "Benchmark Engine" daily job #9026

Closed

runtime-benchmarks/bench depends on buildEngineDistribution

95665cf

JaroslavTulach reviewed Feb 12, 2024

View reviewed changes

Akirathan added 2 commits February 12, 2024 15:58

Update path.yaml in the build script

2377540

Fix bench-report.xml patch in the build script

f873840

Akirathan force-pushed the wip/akirathan/8968-fix-runtime-test-build branch from 596e07e to f873840 Compare February 12, 2024 15:23

Akirathan added the p-high Should be completed in the next sprint label Feb 12, 2024

radeusgd reviewed Feb 12, 2024

View reviewed changes

build.sbt Show resolved Hide resolved

radeusgd approved these changes Feb 12, 2024

View reviewed changes

One more fix bench-report.xml patch in the build script

00fb45e

Akirathan merged commit 5919eda into develop Feb 13, 2024
27 of 29 checks passed

Akirathan deleted the wip/akirathan/8968-fix-runtime-test-build branch February 13, 2024 09:05

hubertp reviewed Feb 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix incremental compilation of runtime/test #8975

Fix incremental compilation of runtime/test #8975

Akirathan commented Feb 5, 2024 •

edited

Akirathan Feb 5, 2024

Akirathan Feb 8, 2024

mwu-tow left a comment

JaroslavTulach left a comment

JaroslavTulach Feb 8, 2024

JaroslavTulach Feb 8, 2024

JaroslavTulach Feb 12, 2024

Akirathan commented Feb 12, 2024 •

edited by unfurl-links bot

Akirathan commented Feb 12, 2024 •

edited by unfurl-links bot

hubertp left a comment

Fix incremental compilation of runtime/test #8975

Fix incremental compilation of runtime/test #8975

Conversation

Akirathan commented Feb 5, 2024 • edited

Pull Request Description

Important Notes

How I tested the changes?

Checklist

Akirathan Feb 5, 2024

Choose a reason for hiding this comment

Akirathan Feb 8, 2024

Choose a reason for hiding this comment

mwu-tow left a comment

Choose a reason for hiding this comment

JaroslavTulach left a comment

Choose a reason for hiding this comment

JaroslavTulach Feb 8, 2024

Choose a reason for hiding this comment

JaroslavTulach Feb 8, 2024

Choose a reason for hiding this comment

JaroslavTulach Feb 12, 2024

Choose a reason for hiding this comment

Akirathan commented Feb 12, 2024 • edited by unfurl-links bot

Akirathan commented Feb 12, 2024 • edited by unfurl-links bot

hubertp left a comment

Choose a reason for hiding this comment

Akirathan commented Feb 5, 2024 •

edited

Akirathan commented Feb 12, 2024 •

edited by unfurl-links bot

Akirathan commented Feb 12, 2024 •

edited by unfurl-links bot