Skip to content

2.31.0.0-b196

2.31.0.0-b196: [#32290] docdb: Speed up running java test
@ttyusupov ttyusupov tagged this 24 Jun 19:14 
Summary:
Re-running even simple java test without changes to both C++ and Java code takes ~37 seconds on GCP `n2-highmem-4` dev-server.

Implemented the following changes:
1) Updated maven-jar-plugin dependency from 3.0.2 to 3.5.0 that uses plexus-archiver with a much faster staleness scan
2) When single `--java-test` is specified:
  - only build required java modules instead of all 14.
  - skip building -sources.jar and -test-sources.jar (via -Dmaven.source.skip=true) for every module built for the test, not just the test's own module, since source jars are never needed to run a test.

This saved ~35 seconds for the first time running org.yb.client.TestBytes#test and ~23-24 seconds for re-running after production or test code changes.

Test Plan:
**Perf:**
```
BUILD_OPTS="release --no-ybc --skip-pg-parquet --no-odyssey"
TEST_OPTS="--java-test org.yb.client.TestBytes#test"
echo ">> Build yugabyte-db"
git ls-files -o java | xargs -r rm
ybd $BUILD_OPTS --remote --sj packaged
echo ">> Run test first time"
(set -x; time ybd $BUILD_OPTS --jo $TEST_OPTS)
echo ">> Change test code and re-run test"
echo "---"
(set -x; touch java/yb-client/src/test/java/org/yb/client/TestBytes.java && time ybd $BUILD_OPTS --jo $TEST_OPTS)
echo ">> Change C++ code and re-run test"
(set -x; touch src/yb/tserver/tserver_flags.cc && time ybd $BUILD_OPTS $TEST_OPTS)
echo ">>"
```

Before the change:
```
>> Run test first time
real	94-97s
>> Change test code and re-run test
real	39-40s
>> Change C++ code and re-run test
real	39-40s
```

After the change:
```
>> Run test first time
real ~59-61s
>> Change test code and re-run test
real	~16s
>> Change C++ code and re-run test
real ~15s
```

**Correctness:**
```
# 1) full test name - builds only yb-client + deps (3 modules)
ybd release --jo --java-test org.yb.client.TestBytes#test

# 2) Bare class name - builds only yb-client (3 modules)
ybd release --jo --java-test TestBytes#test

# 3) Unresolvable test - fail-fast, errors before any build, 0 modules
ybd release --jo --java-test org.yb.client.NoSuchTestXyz#test

# 4) --no-scoped-java-build - builds all 14 modules
ybd release --jo --java-test org.yb.client.TestBytes#test --no-scoped-java-build
```

Reviewers: jason

Reviewed By: jason

Subscribers: jason, yql, ybase

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D54806
Assets 2
Loading