Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provides common performance tests scenarios #24

Merged
merged 3 commits into from
Mar 17, 2021
Merged

provides common performance tests scenarios #24

merged 3 commits into from
Mar 17, 2021

Conversation

dnadoba
Copy link
Member

@dnadoba dnadoba commented Mar 10, 2021

Add initial performance tests and make CI aware of performance tests

Motivation

We want to optimise the library and improve performance.
If want to improve performance based on numbers, we need performance tests.

Modifications

  • Add a new test target for core.
  • Move TestRSocket and TestUnidirectionalStream into new RSocketTestUtilities target
  • Split CI testing into two jobs
    • one for normal tests which are build in debug mode
    • one for performance test which are build in release mode
  • add end to end performance tests which use TCP as transport and Berkeley sockets as the underlying API

Result

We can now start looking into performance related topics and also add more performance tests

Discussion

Performance Test do not fail if they take too long. I could not find a way to generate and define a baseline.
Relevant discussion I could find from the swift mailing list (2016): https://forums.swift.org/t/performance-testing-via-swiftpm-and-xctest/3529/4
I think it is still useful to run the performance test in CI:

  • Other CI jobs only build the package in debug mode and not release mode.
    • This verifies that the code compiles in release mode too i.e. with optimisations
    • Should not make a difference but actually can if there is a bug in the compiler (which happens more frequently when it should)
  • Performance Tests send messages in parallel and could highlight some thread safety issues

Signed-off-by: David Nadoba <dnadoba@gmail.com>
Signed-off-by: David Nadoba <dnadoba@gmail.com>
Sources/RSocketTestUtilities/Payload.swift Outdated Show resolved Hide resolved
Signed-off-by: David Nadoba <dnadoba@gmail.com>
@OlegDokuka OlegDokuka changed the title Performance tests provides common performance tests scenarios Mar 17, 2021
@OlegDokuka OlegDokuka merged commit 33b3718 into rsocket:main Mar 17, 2021
@dnadoba dnadoba deleted the performance-tests branch March 18, 2021 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants