Fixed order dependent flakiness in SensitiveValueTest.java in cloudslang-entities module #1427
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixed the order dependent flaky test in
SensitiveValueTest.java
incloudslang-entities
module.Root Cause
The order dependent flakiness was detected in the cloudslang-entities module. If the
EncryptorConfigTest.java
was run before SensitiveValueTest.java, there will be an error.The error message was below:
The flakiness is due to the shared state of the
EncryptionProvider
anEncryption
beans between the two test classes. BothEncryptorConfigTest
andSensitiveValueTest
test define beans for theEncryption
interface (getEncryption()
andgetTestEncryption()
). Depending on the order of execution, it might retain one definition over the other. InEncryptorConfigTest
, we manipulating theEncryptionProvider
in multiple tests, and those can affect the state of theEncryptionProvider
andEncryption
. However, they were not reseted properly after this test class completes. And then theSensitiveValueTest
depends on those shared state, so it cause failures when run afterEncryptorConfigTest
.Fix log
To fix this issue, I add the helper function
afterTest()
that callscleanup()
to be run after each test method, to clean up the state ofEncryptionProvider
andEncryption
.After the fix, if we run
EncryptorConfigTest.java
beforeSensitiveValueTest.java
, the order dependent flakiness no longer exists.[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
How this has been tested
Java: openjdk version "11.0.20.1"
Maven: Apache Maven 3.9.5
How to reproduce the failure
mvn install -pl cloudslang-entities -am -DskipTests
Signed-off-by: Yi-Ju Tsai [yijujt2@illinois.edu]