Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Echo and KVcounter integration benchmark tests #552

Merged
merged 9 commits into from
Feb 28, 2023

Conversation

brooksmtownsend
Copy link
Member

@brooksmtownsend brooksmtownsend commented Feb 23, 2023

Feature or Problem

This PR aims to add benchmark integration tests for the echo and kvcounter sample actors so that we can compare performance across new features and bug fixes in the OTP host.

Release Information

next release

Consumer Impact

The only impact this PR has is that it might slow down some of our integration tests by about a minute since they will be benchmarking.

Testing

Tested on platform(s)

  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
  • x86_64-windows

Unit Test(s)

Modified echo, added echo_wasi benchmark

Acceptance or Integration

Ideally, I'd like to include a way to upload benchmark results so we can compare over time.

Manual Verification

I'm just running this locally:

MIX_ENV=test mix test test/host_core/benchmark/echo_test.exs

@brooksmtownsend brooksmtownsend changed the title Moved echo, added echo wasi, benchmark tests Echo and KVcounter integration benchmark tests Feb 23, 2023
@connorsmith256
Copy link
Contributor

Ideally, I'd like to include a way to upload benchmark results so we can compare over time.

Would this be helpful? I.e. writing out benchmark results to a file and attaching it to a run as an artifact? https://github.com/actions/upload-artifact

Copy link
Contributor

@connorsmith256 connorsmith256 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only noticed this was still in draft when I went to submit the review, so ignore me if these comments aren't relevant :)

host_core/test/host_core/benchmark/kvcounter_test.exs Outdated Show resolved Hide resolved
host_core/test/host_core/benchmark/kvcounter_test.exs Outdated Show resolved Hide resolved
host_core/test/host_core/benchmark/echo_test.exs Outdated Show resolved Hide resolved
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
connorsmith256
connorsmith256 previously approved these changes Feb 27, 2023
Copy link
Contributor

@connorsmith256 connorsmith256 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just nits. LGTM!

host_core/test/support/common.exs Outdated Show resolved Hide resolved
host_core/test/host_core/benchmark/echo_test.exs Outdated Show resolved Hide resolved
host_core/test/support/common.exs Outdated Show resolved Hide resolved
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
Signed-off-by: Brooks Townsend <brooks@cosmonic.com>
@brooksmtownsend brooksmtownsend merged commit 0a4e705 into main Feb 28, 2023
@brooksmtownsend brooksmtownsend deleted the ci/moar-benchmark-tests branch February 28, 2023 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants