Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bench: using benchmark storage, extracted project #422

Merged
merged 1 commit into from
Jun 24, 2021
Merged

bench: using benchmark storage, extracted project #422

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,35 +122,3 @@ $ mvn clean install -Pqulice
```

To avoid build errors use Maven 3.2+.

## Benchmarking

To run benchmarks:
1. Build `rpm-adapter` with `bench` Maven profile enabled: `mvn package -Pbench`
2. Copy dependencies to `target/` dir: `mvn dependency:copy-dependencies`
3. Create directory for tests and copy test resources to this directory
4. Run benchmarks with `env BENCH_DIR=/tmp/rpm-test java -cp "target/benchmarks.jar:target/classes/*:target/dependency/*" org.openjdk.jmh.Main BenchToRun`, where `/tmp/rpm-test` is a directory with test data.

Available benchmark classes:

### RpmBench

This benchmark class creates/updates repository indexes over provided RPM packages, it calls
`com.artipie.rpm.Rpm.batchUpdateIncrementally` and requires the set of the RPMs in the test directory.
There are available bundles:
- https://artipie.s3.amazonaws.com/rpm-test/bundle100.tar.gz
- https://artipie.s3.amazonaws.com/rpm-test/bundle1000.tar.gz

### RpmMetadataRemoveBench

This benchmark class removes RPM packages records from the repository index files, it works with
`com.artipie.rpm.RpmMetadata.Remove` class and requires xml (unpacked) indexes in the test directory.
Example repository index xmls can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/centos-7-os-x86_64-repodata.tar.gz).

### RpmMetadataAppendBench

`RpmMetadataAppendBench` updates repository metadata with the list of the provided `.rpm` packages,
it works with `com.artipie.rpm.RpmMetadata.Append` class and requires xml (unpacked) indexes and
`.rpm` packages to add in the test directory. Example data set for the benchmark can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/rpm-metadata-append-bench.tar.gz).
31 changes: 31 additions & 0 deletions benchmarks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# RPM adapter benchmarks

To run benchmarks:
1. Install snapshot locally of `rpm-adapter`: `mvn install`
2. Build `rpm-bench` project: `mvn package -f ./benchmarks`
4. Create directory for tests and copy test resources to this directory
5. Run benchmarks with `env BENCH_DIR=${test-dir} java -cp "benchmarks/target/benchmarks.jar" org.openjdk.jmh.Main ${bench-name}`, where `${test-dir}` is a directory with test data, and `${bench-name}` is a benchbmark name.

## Benchmarks

### RpmBench

This benchmark class creates/updates repository indexes over provided RPM packages, it calls
`com.artipie.rpm.Rpm.batchUpdateIncrementally` and requires the set of the RPMs in the test directory.
There are available bundles:
- https://artipie.s3.amazonaws.com/rpm-test/bundle100.tar.gz
- https://artipie.s3.amazonaws.com/rpm-test/bundle1000.tar.gz

### RpmMetadataRemoveBench

This benchmark class removes RPM packages records from the repository index files, it works with
`com.artipie.rpm.RpmMetadata.Remove` class and requires xml (unpacked) indexes in the test directory.
Example repository index xmls can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/centos-7-os-x86_64-repodata.tar.gz).

### RpmMetadataAppendBench

`RpmMetadataAppendBench` updates repository metadata with the list of the provided `.rpm` packages,
it works with `com.artipie.rpm.RpmMetadata.Append` class and requires xml (unpacked) indexes and
`.rpm` packages to add in the test directory. Example data set for the benchmark can be found
[here](https://artipie.s3.amazonaws.com/rpm-test/rpm-metadata-append-bench.tar.gz).
96 changes: 96 additions & 0 deletions benchmarks/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>com.artipie</groupId>
<artifactId>ppom</artifactId>
<version>0.5.1</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>com.artipie</groupId>
<artifactId>rpm-bench</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<jmh.version>1.29</jmh.version>
<asto.version>1.2.0</asto.version>
</properties>
<dependencies>
<dependency>
<groupId>com.artipie</groupId>
<artifactId>rpm-adapter</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>com.artipie</groupId>
<artifactId>asto</artifactId>
<version>${asto.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
<annotationProcessorPaths>
<path>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>benchmarks</finalName>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.openjdk.jmh.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
<filters>
<filter>
<!--
Shading signed JARs will fail without this.
http://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
27 changes: 13 additions & 14 deletions .../com/artipie/rpm/benchmarks/RpmBench.java → .../com/artipie/rpm/benchmarks/RpmBench.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,21 @@

import com.artipie.asto.Key;
import com.artipie.asto.Storage;
import com.artipie.asto.blocking.BlockingStorage;
import com.artipie.asto.fs.FileStorage;
import com.artipie.asto.memory.InMemoryStorage;
import com.artipie.asto.memory.BenchmarkStorage;
import com.artipie.asto.rx.RxStorageWrapper;
import com.artipie.rpm.Rpm;
import hu.akarnokd.rxjava2.interop.CompletableInterop;
import hu.akarnokd.rxjava2.interop.SingleInterop;
import io.reactivex.Observable;
import io.reactivex.Single;
import java.nio.file.Paths;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Level;
Expand Down Expand Up @@ -52,32 +57,26 @@ public class RpmBench {
private static final String BENCH_DIR = System.getenv("BENCH_DIR");

/**
* Repository storage.
* Repository source storage.
*/
private Storage storage;
private InMemoryStorage readonly;

@Setup
public void setup() {
if (RpmBench.BENCH_DIR == null) {
throw new IllegalStateException("BENCH_DIR environment variable must be set");
}
this.storage = new InMemoryStorage();
this.readonly = new InMemoryStorage();
final Storage src = new FileStorage(Paths.get(RpmBench.BENCH_DIR));
RpmBench.sync(src, this.storage);
}

@Setup(Level.Iteration)
public void setupIter() {
final RxStorageWrapper rxst = new RxStorageWrapper(this.storage);
rxst.list(new Key.From("repodata"))
.flatMapObservable(Observable::fromIterable)
.flatMapCompletable(key -> rxst.delete(key))
.to(CompletableInterop.await()).toCompletableFuture().join();
final BlockingStorage bsto = new BlockingStorage(src);
bsto.list(new Key.From("repodata")).forEach(key -> bsto.delete(key));
RpmBench.sync(src, this.readonly);
}

@Benchmark
public void run(final Blackhole bhl) {
new Rpm(this.storage).batchUpdateIncrementally(Key.ROOT)
new Rpm(new BenchmarkStorage(this.readonly))
.batchUpdateIncrementally(Key.ROOT)
.to(CompletableInterop.await())
.toCompletableFuture().join();
}
Expand Down
0 ...pm/benchmarks/RpmMetadataAppendBench.java → ...pm/benchmarks/RpmMetadataAppendBench.java
View file
Open in desktop
File renamed without changes.
0 ...pm/benchmarks/RpmMetadataRemoveBench.java → ...pm/benchmarks/RpmMetadataRemoveBench.java
View file
Open in desktop
File renamed without changes.
0 .../artipie/rpm/benchmarks/package-info.java → .../artipie/rpm/benchmarks/package-info.java
View file
Open in desktop
File renamed without changes.
72 changes: 1 addition & 71 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,11 @@ SOFTWARE.
</parent>
<artifactId>rpm-adapter</artifactId>
<version>1.0-SNAPSHOT</version>
<name>rpm-files</name>
<name>rpm-adapter</name>
<description>Turns your files/objects into RPM artifacts</description>
<url>https://github.com/artipie/rpm-adapter</url>
<inceptionYear>2019</inceptionYear>
<properties>
<jmh.version>1.29</jmh.version>
<qulice.license>${project.basedir}/LICENSE.header</qulice.license>
</properties>
<licenses>
Expand Down Expand Up @@ -221,16 +220,6 @@ SOFTWARE.
<version>1.7.30</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>com.jcabi</groupId>
<artifactId>jcabi-aspects</artifactId>
Expand Down Expand Up @@ -424,64 +413,5 @@ SOFTWARE.
</plugins>
</build>
</profile>
<profile>
<id>bench</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>benchmarks</finalName>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.openjdk.jmh.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
<filters>
<filter>
<!--
Shading signed JARs will fail without this.
http://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
</execution>
</executions>
<configuration>
<includeScope>runtime</includeScope>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>