generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 16
feat(PerfTest): Adds Kms and Aes PerfTests #166
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
1a3ac3f
feat(PerfTest): Adds Kms and Aes PerfTests
ShubhamChaturvedi7 f4c8362
feat(PerfTest): Adds v2 Tests
ShubhamChaturvedi7 8a0a4c1
feat(PerfTest): Adds size metrics
ShubhamChaturvedi7 5b262b3
feat(PerfTest): Adds nested and flat payloads
ShubhamChaturvedi7 6308837
feat(PerfTest): Fix Heirarchy Test Sort Key
ShubhamChaturvedi7 b426022
Merge branch 'main' into scchatur/PerfTest
ShubhamChaturvedi7 e4d7b7d
Merge branch 'main' into scchatur/PerfTest
ShubhamChaturvedi7 e42c28f
fix(PerfTest): Rebase to the latest dbEsdk changes
ShubhamChaturvedi7 08f4ceb
fix(PerfTest): fix Hierarchy Test
ShubhamChaturvedi7 5d70d74
Merge branch 'main' into scchatur/PerfTest
ShubhamChaturvedi7 b10839b
Merge branch 'main' into scchatur/PerfTest
ShubhamChaturvedi7 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# | ||
# https://help.github.com/articles/dealing-with-line-endings/ | ||
# | ||
# Linux start script should use lf | ||
/gradlew text eol=lf | ||
|
||
# These are Windows script files and should use crlf | ||
*.bat text eol=crlf | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# Ignore Gradle project-specific cache directory | ||
.gradle | ||
|
||
# Ignore Gradle build output directory | ||
build |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
# AWS Database Encryption SDK Performance Test | ||
|
||
This project contains tests to benchmark the old v2 and the new v3 AWS Database Encryption SDK for DynamoDb in Java. | ||
|
||
## Benchmarking | ||
### Prerequisites | ||
You'd need the appropriate security credentials to run the Kms Key based tests. Also, build all the Gazelle dependecies and deploy to maven local. | ||
The idea of perf test is to run everything from local, before the changes make it to maven central. | ||
|
||
1. `ada credentials update --account=370957321024 --role=Admin --once` | ||
2. `cd ../../../../DynamoDbEncryption && make build_java mvn_local_deploy_dependencies mvn_local_deploy` | ||
|
||
In case of any failures, look into the `README` for `DynamoDbEncryption`. | ||
|
||
### Gradle Jmh | ||
|
||
`` ./gradlew jmh`` | ||
|
||
This will generate a report on console as well as in `build/results/jmh/results.txt` with the following format: | ||
``` | ||
Benchmark (plainTextFile) Mode Cnt Score Error Units | ||
s.a.c.p.i.v2.AesKeyProviderTest.decrypt single_attribute.json avgt 3 2.428 ± 1.484 ms/op | ||
s.a.c.p.i.v2.AesKeyProviderTest.encrypt single_attribute.json avgt 3 2.691 ± 2.510 ms/op | ||
s.a.c.p.i.v3.RawAesKeyringTest.decrypt single_attribute.json avgt 3 56.868 ± 14.556 ms/op | ||
s.a.c.p.i.v3.RawAesKeyringTest.encrypt single_attribute.json avgt 3 42.814 ± 1.302 ms/op | ||
... | ||
``` | ||
|
||
Size report will also be generated in the same directory, `build/results/size.txt` with the following format: | ||
|
||
``` | ||
AesKeyProviderTest_decrypt_jmhTest 205152 274001 68849 | ||
AesKeyProviderTest_encrypt_jmhTest 205152 274001 68849 | ||
AwsKmsKeyProviderTest_decrypt_jmhTest 205152 274349 69197 | ||
AwsKmsKeyProviderTest_encrypt_jmhTest 205152 274349 69197 | ||
... | ||
``` | ||
|
||
where the columns are `OperationName OriginalSize EncryptedSize Diff` | ||
|
||
### IDE (or Main Class) | ||
1. Run the `main()` method in the TestBase class, or you can also run individual tests by executing the `main()` method in the respective classes. | ||
2. This serves the purpose of quickly debugging benchmarking/jmh issues. | ||
|
||
## Profiling (Flamegraph) in IntelliJ | ||
1. Use the main method in each Test class and `Run with Profiler`. | ||
2. The graph would show the stack and the relative time taken by each of the method call |
132 changes: 132 additions & 0 deletions
132
PerfTest/runtimes/java/DynamoDbEncryption/build.gradle.kts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
import java.net.URI | ||
import javax.annotation.Nullable | ||
import org.gradle.api.tasks.testing.logging.TestExceptionFormat | ||
import org.gradle.api.tasks.testing.logging.TestLogEvent | ||
import java.awt.SystemColor.info | ||
import java.nio.file.Files.delete | ||
|
||
plugins { | ||
`java` | ||
`java-library` | ||
id("me.champeau.jmh") version "0.7.0" | ||
} | ||
|
||
group = "software.amazon.cryptography" | ||
version = "1.0-SNAPSHOT" | ||
description = "DynamoDbEncryptionPerformanceTest" | ||
|
||
java { | ||
toolchain.languageVersion.set(JavaLanguageVersion.of(17)) | ||
sourceSets["main"].java { | ||
srcDir("src/main/java") | ||
} | ||
sourceSets["test"].java { | ||
srcDir("src/test/java") | ||
} | ||
} | ||
|
||
sourceSets { | ||
this.jmh { | ||
java.setSrcDirs(mutableListOf("src/main/java")) | ||
} | ||
} | ||
|
||
var caUrl: URI? = null | ||
@Nullable | ||
val caUrlStr: String? = System.getenv("CODEARTIFACT_URL_JAVA_CONVERSION") | ||
if (!caUrlStr.isNullOrBlank()) { | ||
caUrl = URI.create(caUrlStr) | ||
} | ||
|
||
var caPassword: String? = null | ||
@Nullable | ||
val caPasswordString: String? = System.getenv("CODEARTIFACT_AUTH_TOKEN") | ||
if (!caPasswordString.isNullOrBlank()) { | ||
caPassword = caPasswordString | ||
} | ||
|
||
repositories { | ||
mavenCentral() | ||
mavenLocal() | ||
|
||
// While commented out, this project will not pull from CodeArtifact, you must build and deploy dependencies locally | ||
// | ||
// maven { | ||
// name = "CodeArtifact" | ||
// url = URI.create("https://avp10745-648638458147.d.codeartifact.us-west-2.amazonaws.com/maven/AVP-10745/") | ||
// credentials { | ||
// username = "aws" | ||
// password = System.getenv("CODEARTIFACT_AUTH_TOKEN") | ||
// } | ||
//} | ||
} | ||
|
||
dependencies { | ||
implementation("software.amazon.cryptography:aws-database-encryption-sdk-dynamodb:1.0-SNAPSHOT") | ||
implementation("software.amazon.cryptography:AwsCryptographicMaterialProviders:1.0-SNAPSHOT") | ||
|
||
implementation(platform("software.amazon.awssdk:bom:2.19.1")) | ||
implementation("software.amazon.awssdk:dynamodb:2.20.64") | ||
implementation("software.amazon.awssdk:dynamodb-enhanced") | ||
implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.409") | ||
implementation("software.amazon.awssdk:kms") | ||
|
||
implementation("org.openjdk.jmh:jmh-core:1.36") | ||
implementation("org.openjdk.jmh:jmh-generator-annprocess:1.36") | ||
annotationProcessor ("org.openjdk.jmh:jmh-generator-annprocess:1.36") | ||
implementation("com.fasterxml.jackson.core:jackson-databind:2.15.0") | ||
implementation("com.univocity:univocity-parsers:2.9.1") | ||
implementation("javax.xml.bind:jaxb-api:2.3.1") | ||
implementation("org.apache.commons:commons-lang3:3.8.1") | ||
|
||
// https://mvnrepository.com/artifact/org.testng/testng | ||
testImplementation("org.testng:testng:7.5") | ||
} | ||
|
||
tasks.withType<JavaCompile>() { | ||
options.encoding = "UTF-8" | ||
} | ||
|
||
gradle.taskGraph.whenReady { | ||
delete { | ||
file("build/results/").deleteRecursively() | ||
} | ||
} | ||
|
||
tasks.test { | ||
useTestNG() | ||
|
||
// This will show System.out.println statements | ||
testLogging.showStandardStreams = true | ||
|
||
testLogging { | ||
lifecycle { | ||
events = mutableSetOf(TestLogEvent.FAILED, TestLogEvent.PASSED, TestLogEvent.SKIPPED) | ||
exceptionFormat = TestExceptionFormat.FULL | ||
showExceptions = true | ||
showCauses = true | ||
showStackTraces = true | ||
showStandardStreams = true | ||
} | ||
info.events = lifecycle.events | ||
info.exceptionFormat = lifecycle.exceptionFormat | ||
} | ||
|
||
// See https://github.com/gradle/kotlin-dsl/issues/836 | ||
addTestListener(object : TestListener { | ||
override fun beforeSuite(suite: TestDescriptor) {} | ||
override fun beforeTest(testDescriptor: TestDescriptor) {} | ||
override fun afterTest(testDescriptor: TestDescriptor, result: TestResult) {} | ||
|
||
override fun afterSuite(suite: TestDescriptor, result: TestResult) { | ||
if (suite.parent == null) { // root suite | ||
logger.lifecycle("----") | ||
logger.lifecycle("Test result: ${result.resultType}") | ||
logger.lifecycle("Test summary: ${result.testCount} tests, " + | ||
"${result.successfulTestCount} succeeded, " + | ||
"${result.failedTestCount} failed, " + | ||
"${result.skippedTestCount} skipped") | ||
} | ||
} | ||
}) | ||
} |
Binary file added
BIN
+60.1 KB
PerfTest/runtimes/java/DynamoDbEncryption/gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
6 changes: 6 additions & 0 deletions
6
PerfTest/runtimes/java/DynamoDbEncryption/gradle/wrapper/gradle-wrapper.properties
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
distributionBase=GRADLE_USER_HOME | ||
distributionPath=wrapper/dists | ||
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6-bin.zip | ||
networkTimeout=10000 | ||
zipStoreBase=GRADLE_USER_HOME | ||
zipStorePath=wrapper/dists |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.