feat: perf-test for python #813

ShubhamChaturvedi7 · 2025-09-03T00:08:18Z

Issue #, if available:

Description of changes:
This PR adds a performance tests suite for python. The code was generated by agentic AI and was refactored by author.
Squash/merge commit message, if applicable:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

esdk-performance-testing/benchmarks/python/requirements.txt

lucasmcdonald3 · 2025-09-03T15:52:54Z

esdk-performance-testing/benchmarks/python/requirements.txt

@@ -0,0 +1,31 @@
+# ESDK Performance Testing - Python Dependencies


Do we need all these dependencies? A fair amount seem unused

I was relying on linter and formatter to fix this, but apparently it didn't. Fixed it now.

lucasmcdonald3 · 2025-09-03T15:57:21Z

esdk-performance-testing/benchmarks/python/benchmark.py

+
+    def _create_keyring(self):
+        """Create raw AES keyring"""
+        import secrets


These imports should go up top -- I think the MPL is intended to be required for this project so shouldn't need any special consideration

lucasmcdonald3 · 2025-09-03T15:58:22Z

esdk-performance-testing/benchmarks/python/benchmark.py

+try:
+    from aws_encryption_sdk import EncryptionSDKClient, CommitmentPolicy
+except ImportError as e:
+    print(f"Warning: Could not import ESDK modules: {e}")
+    print("Please install the AWS Encryption SDK: pip install aws-encryption-sdk")
+    sys.exit(1)


I think you can just import this?

Suggested change

try:

from aws_encryption_sdk import EncryptionSDKClient, CommitmentPolicy

except ImportError as e:

print(f"Warning: Could not import ESDK modules: {e}")

print("Please install the AWS Encryption SDK: pip install aws-encryption-sdk")

sys.exit(1)

from aws_encryption_sdk import EncryptionSDKClient, CommitmentPolicy

lucasmcdonald3 · 2025-09-03T16:09:17Z

esdk-performance-testing/results/raw-data/python_results.json

+      "peak_memory_mb": 0.0,
+      "avg_memory_mb": 0.0,
+      "cumulative_allocations_mb": 0.0,
+      "memory_efficiency_ratio": 0.0,


Is there a README or similar that explains the results JSON? Some fields are obvious but things like this could use an explanation

There's a README but added a bit more info.

Co-authored-by: Lucas McDonald <lucasmcdonald3@gmail.com>

## [4.2.0](v4.1.0...v4.2.0) (2025-10-03) ### Features * adds polymorped go files for esdk ([7ac872c](7ac872c)) * adds temp patch file for Go ([d3cd446](d3cd446)) * All shapes complete ([0067712](0067712)) * Go ESDK v0.0.1 release ([#741](#741)) ([714e177](714e177)) * **Go:** esdk-go release v0.2.0 ([#769](#769)) ([61475fe](61475fe)) * Golang ([2db84c0](2db84c0)) * **go:** mpl bump ([#737](#737)) ([babfd01](babfd01)) * **Go:** v0.1.1 release ([#757](#757)) ([fc17427](fc17427)) * perf-test for go ([#812](#812)) ([a825857](a825857)) * perf-test for java ([#815](#815)) ([51f031b](51f031b)) * perf-test for net ([#814](#814)) ([472a65d](472a65d)) * perf-test for python ([#813](#813)) ([6e5f59e](6e5f59e)) * perf-test for rust ([#810](#810)) ([751f286](751f286)) * **Rust:** Add support for Rust Encryption SDK ([#689](#689)) ([1628e66](1628e66)) ### Fixes * **.NET:** Improve Collection of Errors string ([#728](#728)) ([d66248b](d66248b)) * Bump more CI to Dafny 4.8 ([#683](#683)) ([170b398](170b398)) * Bump more CI to Dafny 4.8 ([#683](#683)) ([56fcd13](56fcd13)) * CI ([0ba9ada](0ba9ada)) * CI ([70b3891](70b3891)) * CI ([9650048](9650048)) * CI ([69725ef](69725ef)) * CI ([3533565](3533565)) * CI ([1862a4c](1862a4c)) * CI ([c56dd69](c56dd69)) * **Example:** correct doc link in limit EDK Rust example ([#729](#729)) ([fa1ca01](fa1ca01)) * **Go:** adds high unicode char Test Vectors ([#762](#762)) ([5604983](5604983)) * **go:** fix and refactor go mod ([#793](#793)) ([9381006](9381006)) * Remove --recursive from action to update mpl submodule ([#672](#672)) ([8150ee0](8150ee0)) * repo rename ([#739](#739)) ([d580c4a](d580c4a)) * repo rename ([#740](#740)) ([ae8b4b7](ae8b4b7)) * Use relative path for DafnyRuntimeGo ([f33187c](f33187c)) ### Maintenance * add C to list of languages checked for interoperability ([#773](#773)) ([3b9297a](3b9297a)) * add check only keyword action ([#679](#679)) ([1839c53](1839c53)) * Add go to smithy dependencies ([#818](#818)) ([ac3448a](ac3448a)) * add KMS Keyring example in an multithreaded environment for Go ([#735](#735)) ([246aea1](246aea1)) * bump dafny verification to 4.7.0 ([#666](#666)) ([a6ccaf7](a6ccaf7)) * bump macos from 12 to 13 in CI ([#691](#691)) ([756888e](756888e)) * bump mpl ([#787](#787)) ([1be4d62](1be4d62)) * Bump MPL dependency to 1.9.0, consolidate `project.properties` files ([#753](#753)) ([366e6e7](366e6e7)) * bump mpl to allow nightly build ([#816](#816)) ([95a5cd3](95a5cd3)) * bump mpl to latest ([#809](#809)) ([3c01934](3c01934)) * bump MPL to latest version ([#775](#775)) ([5ea20b7](5ea20b7)) * bump MPL version ([#778](#778)) ([dcaa266](dcaa266)) * bump smithy-dafny to latest ([#774](#774)) ([9fd4b48](9fd4b48)) * bump to mpl latest ([#820](#820)) ([0a2013b](0a2013b)) * **cfn/CI.yaml:** setup region based roles for Client Supplier Example ([#684](#684)) ([ffe0064](ffe0064)) * **cfn/CI.yaml:** setup region based roles for Client Supplier Example ([#684](#684)) ([74b0107](74b0107)) * change the dependency from the ESDK to the MPL to a relative path ProjectReference ([#656](#656)) ([9c73845](9c73845)) * **CI.yaml:** update cfn template ([#687](#687)) ([568abc2](568abc2)) * **CI.yaml:** update cfn template ([#687](#687)) ([864c2e2](864c2e2)) * **CI:** Add a diff check for polymorph in CI ([#738](#738)) ([af1e677](af1e677)) * **CI:** add backwards compatability tests for nightly ([#790](#790)) ([d350e84](d350e84)) * **CI:** add slack notifications ([#797](#797)) ([902bab0](902bab0)) * **CI:** Add smithy diff checker gha ([#670](#670)) ([99c489d](99c489d)) * **CI:** Clean up unused attributes ([#772](#772)) ([ba242a3](ba242a3)) * **CI:** Fix CI ([#803](#803)) ([bc63241](bc63241)) * **CI:** Fix daily CI ([#771](#771)) ([36079de](36079de)) * **CI:** Partially fix nightly build ([#785](#785)) ([ed63f70](ed63f70)) * **CI:** use require workflow ci check instead ([#719](#719)) ([1c4d1c4](1c4d1c4)) * configure with project.properties ([#750](#750)) ([3ce52fa](3ce52fa)) * **dafny-verification:** fix timeout for Dafny 4.8.1 ([#690](#690)) ([bc46d0f](bc46d0f)) * **dafny:** allow FileIO to deal in uint8, rather than bv8 ([#770](#770)) ([90f55db](90f55db)) * **dafny:** remove more usage of BigInteger ([#781](#781)) ([dbe55d7](dbe55d7)) * **dafny:** update makefile to only use prettier 3.5.3 ([#786](#786)) ([d2f9d82](d2f9d82)) * **dependabot:** Add dependabot for github actions and rust ([#712](#712)) ([97dd65f](97dd65f)) * ensure format checking is with dotnet 6 ([#746](#746)) ([b3cb599](b3cb599)) * **examples; MPL bump:** Shared cache across Hierarchical Keyrings; Bump MPL to 1.7.0 ([#680](#680)) ([f41a689](f41a689)) * **examples; MPL bump:** Shared cache across Hierarchical Keyrings; Bump MPL to 1.7.0 ([#680](#680)) ([d8ca214](d8ca214)) * **examples:** Add examples for Go with CI ([#733](#733)) ([c3ba5e3](c3ba5e3)) * **examples:** add version branch key id example ([#710](#710)) ([d03a340](d03a340)) * **examples:** fix readme of examples in all languages ([#734](#734)) ([f4683de](f4683de)) * Fail verification on warnings ([#766](#766)) ([7236084](7236084)) * fix permissions and add env vars ([#676](#676)) ([4f03595](4f03595)) * **GHA:** add dafny interoperability action ([#674](#674)) ([64067c7](64067c7)) * **gitignore:** Ignore *-cs.dtr and node_modules ([#686](#686)) ([2bfe9d4](2bfe9d4)) * **gitignore:** Ignore *-cs.dtr and node_modules ([#686](#686)) ([83744db](83744db)) * **go:** add go 1.25 to CI ([#800](#800)) ([fb62f77](fb62f77)) * **go:** add Go release workflow ([#792](#792)) ([fe49c73](fe49c73)) * **Go:** Add Go to supported languages in the readme ([#780](#780)) ([3d920b9](3d920b9)) * **go:** cleanup files generated by examples ([#806](#806)) ([9f51159](9f51159)) * **go:** update go test matrix and clean up setup ([#788](#788)) ([f39702b](f39702b)) * improve performance ([#764](#764)) ([79e1306](79e1306)) * install polymorph deps in github workflows ([#783](#783)) ([e284b64](e284b64)) * let interop testing launch N tasks instead of N^2 ([#789](#789)) ([f5f8718](f5f8718)) * merge Golang/dev branch with mainline ([#704](#704)) ([6db9ace](6db9ace)) * **mpl;Rust:** bump mpl to include rust input validation, aes-192 tests ([#722](#722)) ([bea3f29](bea3f29)) * **MPL:** bump MPL to 1.5.1 ([#667](#667)) ([7958020](7958020)) * **README:** update README; add copyright string ([#708](#708)) ([c133d77](c133d77)) * Reduce resource usage of assertion in ReadFramedMessageBody ([#665](#665)) ([c49af85](c49af85)) * regen code for 4.8.0 and bump mpl ([#681](#681)) ([68e75fb](68e75fb)) * Release of 1.0.0 for Rust ([#751](#751)) ([7e492d7](7e492d7)) * remove aws sdk core dep ([#822](#822)) ([54d3685](54d3685)) * Rename AtomicPrimitives Dafny module name ([#673](#673)) ([8eeb14d](8eeb14d)) * Rename AtomicPrimitives Dafny module name ([#673](#673)) ([e042e92](e042e92)) * rename EncryptionSdk Dafny module name ([#707](#707)) ([689624d](689624d)) * **rust-release:** add Rust release directory ([#709](#709)) ([7acf387](7acf387)) * **rust:** add fips feature ([#805](#805)) ([d7d8bf1](d7d8bf1)) * **rust:** add more memory usage statistics to benchmarks ([#819](#819)) ([c0f9dc9](c0f9dc9)) * **rust:** bump mpl to fix warnings ([#823](#823)) ([e971002](e971002)) * **rust:** clean-up lib.rs ([#725](#725)) ([d03ab27](d03ab27)) * **rust:** prepare 1.1.1 release ([#784](#784)) ([3aeddc9](3aeddc9)) * **rust:** prepare release of 1.2.0 ([#807](#807)) ([678ca04](678ca04)) * **rust:** prepare rust 1.1.0 release ([#782](#782)) ([b321a46](b321a46)) * **rust:** release 0.2.0 ([#726](#726)) ([a5282af](a5282af)) * **rust:** update smithy-dafny and use small-int feature ([#777](#777)) ([2ea5e47](2ea5e47)) * **rust:** use Dafny 4.10.0 for Rust codegen ([#824](#824)) ([d70d88c](d70d88c)) * **test vectors:** Add interop test in Go with CI ([#731](#731)) ([d04ddc0](d04ddc0)) * **test-vectors:** Decrypt .NET Retry flag manifests using Rust ([#721](#721)) ([3117b0c](3117b0c)) * **TestVectors:** add DefaultCmm Tests ([#698](#698)) ([a287d45](a287d45)) * Try to update existing issues ([#661](#661)) ([70c0d14](70c0d14)), closes [#660](#660) * update mpl ([95fe4e9](95fe4e9)) * update mpl ([#821](#821)) ([1e4d0d0](1e4d0d0)) * update mpl submodule name gha ([#675](#675)) ([959d22e](959d22e)) * Update SUPPORT_POLICY.rst ([#802](#802)) ([bcf4220](bcf4220)) * update to async-friendly Rust ([#749](#749)) ([fad048c](fad048c)) * update to MPL 1.4 ([8725990](8725990)) * use correct dirs in gha ([#677](#677)) ([25c3625](25c3625))

PerfTests: python

54b18b2

ShubhamChaturvedi7 requested a review from a team as a code owner September 3, 2025 00:08

ShubhamChaturvedi7 temporarily deployed to Duvet CI September 3, 2025 00:08 — with GitHub Actions Inactive

ShubhamChaturvedi7 changed the title ~~feat: perf-test for go~~ feat: perf-test for python Sep 3, 2025

lucasmcdonald3 reviewed Sep 3, 2025

View reviewed changes

fix: update esdk dependency

f736ae6

Co-authored-by: Lucas McDonald <lucasmcdonald3@gmail.com>

ShubhamChaturvedi7 temporarily deployed to Duvet CI September 3, 2025 16:29 — with GitHub Actions Inactive

Merge branch 'mainline' into PerfTests/python

ca28a62

ShubhamChaturvedi7 temporarily deployed to Duvet CI September 3, 2025 20:33 — with GitHub Actions Inactive

fix: PR comments

9dee361

ShubhamChaturvedi7 temporarily deployed to Duvet CI September 3, 2025 20:42 — with GitHub Actions Inactive

fix: linting

a8936da

ShubhamChaturvedi7 temporarily deployed to Duvet CI September 3, 2025 20:46 — with GitHub Actions Inactive

lucasmcdonald3 approved these changes Sep 3, 2025

View reviewed changes

ShubhamChaturvedi7 merged commit 6e5f59e into mainline Sep 3, 2025
85 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: perf-test for python #813

feat: perf-test for python #813

Uh oh!

ShubhamChaturvedi7 commented Sep 3, 2025

Uh oh!

Uh oh!

lucasmcdonald3 Sep 3, 2025

Uh oh!

ShubhamChaturvedi7 Sep 3, 2025

Uh oh!

lucasmcdonald3 Sep 3, 2025

Uh oh!

lucasmcdonald3 Sep 3, 2025

Uh oh!

lucasmcdonald3 Sep 3, 2025

Uh oh!

ShubhamChaturvedi7 Sep 3, 2025

Uh oh!

Uh oh!

Uh oh!

		@@ -0,0 +1,31 @@
		# ESDK Performance Testing - Python Dependencies

feat: perf-test for python #813

feat: perf-test for python #813

Uh oh!

Conversation

ShubhamChaturvedi7 commented Sep 3, 2025

Uh oh!

Uh oh!

lucasmcdonald3 Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

ShubhamChaturvedi7 Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

lucasmcdonald3 Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

lucasmcdonald3 Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

lucasmcdonald3 Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

ShubhamChaturvedi7 Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!