1) Optimize 50k & 100k splitter regular expressions - 10.5s to 8.9s #75

l0rinc · 2023-12-22T11:21:48Z

As the first step in optimizing the c100k parser mostly (used for GPT 3.5 & 4), here's the regex optimization applying to all 50k and the 100k parsers.

The difference is not huge, but measurable:

Before:

Benchmark                                    (dataFolderPath)  Mode  Cnt   Score   Error  Units
SingleThreadedBenchmark.benchmarkCl100kBase              data    ss   10  10.548 ± 0.885   s/op
SingleThreadedBenchmark.benchmarkP50kBase                data    ss   10   9.999 ± 0.097   s/op
SingleThreadedBenchmark.benchmarkP50kEdit                data    ss   10  10.184 ± 0.131   s/op
SingleThreadedBenchmark.benchmarkR50kBase                data    ss   10   9.938 ± 0.076   s/op

After:

Benchmark                                    (dataFolderPath)  Mode  Cnt  Score   Error  Units
SingleThreadedBenchmark.benchmarkCl100kBase              data    ss   10  8.947 ± 0.109   s/op
SingleThreadedBenchmark.benchmarkP50kBase                data    ss   10  9.419 ± 0.082   s/op
SingleThreadedBenchmark.benchmarkP50kEdit                data    ss   10  9.365 ± 0.073   s/op
SingleThreadedBenchmark.benchmarkR50kBase                data    ss   10  8.403 ± 0.080   s/op

Please review commit-by-commit for the changes to make sense:

Feel free to either comment - or if it's simpler -, add commits on top of these.

benchmark/build.gradle.kts

benchmark/src/jmh/java/com/knuddels/jtokkit/AbstractBenchmark.java

benchmark/src/jmh/java/com/knuddels/jtokkit/BenchmarkUtils.java

benchmark/src/jmh/java/com/knuddels/jtokkit/DataDownloader.java

lib/build.gradle.kts

lib/src/main/java/com/knuddels/jtokkit/EncodingFactory.java

lib/src/test/java/com/knuddels/jtokkit/reference/Cl100kBaseTest.java

lib/src/main/java/com/knuddels/jtokkit/EncodingFactory.java

… source codes and extreme DOS content

Benchmark (dataFolderPath) Mode Cnt Score Error Units SingleThreadedBenchmark.benchmarkCl100kBase data ss 10 10.548 ± 0.885 s/op SingleThreadedBenchmark.benchmarkP50kBase data ss 10 9.999 ± 0.097 s/op SingleThreadedBenchmark.benchmarkP50kEdit data ss 10 10.184 ± 0.131 s/op SingleThreadedBenchmark.benchmarkR50kBase data ss 10 9.938 ± 0.076 s/op

Before: Benchmark (dataFolderPath) Mode Cnt Score Error Units SingleThreadedBenchmark.benchmarkCl100kBase data ss 10 10.548 ± 0.885 s/op SingleThreadedBenchmark.benchmarkP50kBase data ss 10 9.999 ± 0.097 s/op SingleThreadedBenchmark.benchmarkP50kEdit data ss 10 10.184 ± 0.131 s/op SingleThreadedBenchmark.benchmarkR50kBase data ss 10 9.938 ± 0.076 s/op After: Benchmark (dataFolderPath) Mode Cnt Score Error Units SingleThreadedBenchmark.benchmarkCl100kBase data ss 10 8.947 ± 0.109 s/op SingleThreadedBenchmark.benchmarkP50kBase data ss 10 9.419 ± 0.082 s/op SingleThreadedBenchmark.benchmarkP50kEdit data ss 10 9.365 ± 0.073 s/op SingleThreadedBenchmark.benchmarkR50kBase data ss 10 8.403 ± 0.080 s/op

.github/workflows/build-pull-request.yml

tox-p

Thanks for the great PR, the structure and the helpful comments!

I have commented some questions that I would love to get resolved before merging, but overall they are quite minor 🙂

benchmark/src/jmh/java/com/knuddels/jtokkit/BenchmarkUtils.java

lib/src/main/java/com/knuddels/jtokkit/EncodingFactory.java

lib/src/test/java/com/knuddels/jtokkit/reference/Cl100kBaseTest.java

benchmark/src/jmh/java/com/knuddels/jtokkit/AbstractBenchmark.java

lib/src/main/java/com/knuddels/jtokkit/EncodingFactory.java

lib/src/test/java/com/knuddels/jtokkit/Cl100kBaseTest.java

JUnit 5 doesn't require public modifiers anymore. Private modifiers also don't make sense for non-helper tests since they aren't inherited. Renamed Cl100kBaseTestTest to Cl100kBaseTest. Removed dangling final modifiers. Used `var` in all local declarations.

lib/src/test/java/com/knuddels/jtokkit/BaseEncodingRegistryTest.java

lib/src/test/java/com/knuddels/jtokkit/reference/Cl100kBaseTest.java

tox-p

As stated in the in-line comments: I can easily add testing with all LTS JDKs after your changes are merged. Let's not let that delay us moving forward with the other PRs 🙂

l0rinc requested a review from tox-p as a code owner December 22, 2023 11:21