Skip to content

Commit

Permalink
bench: using benchmark storage, extracted project (#422)
Browse files Browse the repository at this point in the history
Using special benchmark storage from artipie/asto
for benchmarks. Moved benchmarks project to separate
Maven project.

Signed-off-by: Kirill Ch. <g4s8.public@gmail.com>
Close: #421
Ref: artipie/asto#314
  • Loading branch information
g4s8 committed Jun 24, 2021
1 parent f94c745 commit cd05bac
Show file tree
Hide file tree
Showing 8 changed files with 141 additions and 117 deletions.
32 changes: 0 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,35 +122,3 @@ $ mvn clean install -Pqulice
```

To avoid build errors use Maven 3.2+.

## Benchmarking

To run benchmarks:
1. Build `rpm-adapter` with `bench` Maven profile enabled: `mvn package -Pbench`
2. Copy dependencies to `target/` dir: `mvn dependency:copy-dependencies`
3. Create directory for tests and copy test resources to this directory
4. Run benchmarks with `env BENCH_DIR=/tmp/rpm-test java -cp "target/benchmarks.jar:target/classes/*:target/dependency/*" org.openjdk.jmh.Main BenchToRun`, where `/tmp/rpm-test` is a directory with test data.

Available benchmark classes:

### RpmBench

This benchmark class creates/updates repository indexes over provided RPM packages, it calls
`com.artipie.rpm.Rpm.batchUpdateIncrementally` and requires the set of the RPMs in the test directory.
There are available bundles:
- https://artipie.s3.amazonaws.com/rpm-test/bundle100.tar.gz
- https://artipie.s3.amazonaws.com/rpm-test/bundle1000.tar.gz

### RpmMetadataRemoveBench

This benchmark class removes RPM packages records from the repository index files, it works with
`com.artipie.rpm.RpmMetadata.Remove` class and requires xml (unpacked) indexes in the test directory.
Example repository index xmls can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/centos-7-os-x86_64-repodata.tar.gz).

### RpmMetadataAppendBench

`RpmMetadataAppendBench` updates repository metadata with the list of the provided `.rpm` packages,
it works with `com.artipie.rpm.RpmMetadata.Append` class and requires xml (unpacked) indexes and
`.rpm` packages to add in the test directory. Example data set for the benchmark can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/rpm-metadata-append-bench.tar.gz).
31 changes: 31 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# RPM adapter benchmarks

To run benchmarks:
1. Install snapshot locally of `rpm-adapter`: `mvn install`
2. Build `rpm-bench` project: `mvn package -f ./benchmarks`
4. Create directory for tests and copy test resources to this directory
5. Run benchmarks with `env BENCH_DIR=${test-dir} java -cp "benchmarks/target/benchmarks.jar" org.openjdk.jmh.Main ${bench-name}`, where `${test-dir}` is a directory with test data, and `${bench-name}` is a benchbmark name.

## Benchmarks

### RpmBench

This benchmark class creates/updates repository indexes over provided RPM packages, it calls
`com.artipie.rpm.Rpm.batchUpdateIncrementally` and requires the set of the RPMs in the test directory.
There are available bundles:
- https://artipie.s3.amazonaws.com/rpm-test/bundle100.tar.gz
- https://artipie.s3.amazonaws.com/rpm-test/bundle1000.tar.gz

### RpmMetadataRemoveBench

This benchmark class removes RPM packages records from the repository index files, it works with
`com.artipie.rpm.RpmMetadata.Remove` class and requires xml (unpacked) indexes in the test directory.
Example repository index xmls can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/centos-7-os-x86_64-repodata.tar.gz).

### RpmMetadataAppendBench

`RpmMetadataAppendBench` updates repository metadata with the list of the provided `.rpm` packages,
it works with `com.artipie.rpm.RpmMetadata.Append` class and requires xml (unpacked) indexes and
`.rpm` packages to add in the test directory. Example data set for the benchmark can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/rpm-metadata-append-bench.tar.gz).
96 changes: 96 additions & 0 deletions benchmarks/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>com.artipie</groupId>
<artifactId>ppom</artifactId>
<version>0.5.1</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>com.artipie</groupId>
<artifactId>rpm-bench</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<jmh.version>1.29</jmh.version>
<asto.version>1.2.0</asto.version>
</properties>
<dependencies>
<dependency>
<groupId>com.artipie</groupId>
<artifactId>rpm-adapter</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>com.artipie</groupId>
<artifactId>asto</artifactId>
<version>${asto.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
<annotationProcessorPaths>
<path>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>benchmarks</finalName>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.openjdk.jmh.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
<filters>
<filter>
<!--
Shading signed JARs will fail without this.
http://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,21 @@

import com.artipie.asto.Key;
import com.artipie.asto.Storage;
import com.artipie.asto.blocking.BlockingStorage;
import com.artipie.asto.fs.FileStorage;
import com.artipie.asto.memory.InMemoryStorage;
import com.artipie.asto.memory.BenchmarkStorage;
import com.artipie.asto.rx.RxStorageWrapper;
import com.artipie.rpm.Rpm;
import hu.akarnokd.rxjava2.interop.CompletableInterop;
import hu.akarnokd.rxjava2.interop.SingleInterop;
import io.reactivex.Observable;
import io.reactivex.Single;
import java.nio.file.Paths;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Level;
Expand Down Expand Up @@ -52,32 +57,26 @@ public class RpmBench {
private static final String BENCH_DIR = System.getenv("BENCH_DIR");

/**
* Repository storage.
* Repository source storage.
*/
private Storage storage;
private InMemoryStorage readonly;

@Setup
public void setup() {
if (RpmBench.BENCH_DIR == null) {
throw new IllegalStateException("BENCH_DIR environment variable must be set");
}
this.storage = new InMemoryStorage();
this.readonly = new InMemoryStorage();
final Storage src = new FileStorage(Paths.get(RpmBench.BENCH_DIR));
RpmBench.sync(src, this.storage);
}

@Setup(Level.Iteration)
public void setupIter() {
final RxStorageWrapper rxst = new RxStorageWrapper(this.storage);
rxst.list(new Key.From("repodata"))
.flatMapObservable(Observable::fromIterable)
.flatMapCompletable(key -> rxst.delete(key))
.to(CompletableInterop.await()).toCompletableFuture().join();
final BlockingStorage bsto = new BlockingStorage(src);
bsto.list(new Key.From("repodata")).forEach(key -> bsto.delete(key));
RpmBench.sync(src, this.readonly);
}

@Benchmark
public void run(final Blackhole bhl) {
new Rpm(this.storage).batchUpdateIncrementally(Key.ROOT)
new Rpm(new BenchmarkStorage(this.readonly))
.batchUpdateIncrementally(Key.ROOT)
.to(CompletableInterop.await())
.toCompletableFuture().join();
}
Expand Down
72 changes: 1 addition & 71 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,11 @@ SOFTWARE.
</parent>
<artifactId>rpm-adapter</artifactId>
<version>1.0-SNAPSHOT</version>
<name>rpm-files</name>
<name>rpm-adapter</name>
<description>Turns your files/objects into RPM artifacts</description>
<url>https://github.com/artipie/rpm-adapter</url>
<inceptionYear>2019</inceptionYear>
<properties>
<jmh.version>1.29</jmh.version>
<qulice.license>${project.basedir}/LICENSE.header</qulice.license>
</properties>
<licenses>
Expand Down Expand Up @@ -221,16 +220,6 @@ SOFTWARE.
<version>1.7.30</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>com.jcabi</groupId>
<artifactId>jcabi-aspects</artifactId>
Expand Down Expand Up @@ -424,64 +413,5 @@ SOFTWARE.
</plugins>
</build>
</profile>
<profile>
<id>bench</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>benchmarks</finalName>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.openjdk.jmh.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
<filters>
<filter>
<!--
Shading signed JARs will fail without this.
http://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
</execution>
</executions>
<configuration>
<includeScope>runtime</includeScope>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

0 comments on commit cd05bac

Please sign in to comment.