Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed Flaky Test caused by the Map.keySet function #1382

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bmk15897
Copy link

@bmk15897 bmk15897 commented Nov 4, 2022

Description

Flaky Test found using NonDex by running the command -
mvn -pl cloudslang-runtime edu.illinois:nondex-maven-plugin:1.1.2:nondex -Dtest=io.cloudslang.lang.runtime.bindings.scripts.ScriptExecutorTest#testExternalPythonValid

The logged failure for the io.cloudslang.lang.runtime.bindings.scripts.ScriptExecutorTest.testExternalPythonValid -

[ERROR] Failures: 
[ERROR]   ScriptExecutorTest.testExternalPythonValid:128 arrays first differed at element [0]; expected:<input[1]> but was:<input[2]>
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Investigation

The test fails at ScriptExecutorTest.java:128 with a comparison error while comparing an expected keyset array and the result from the captured keyset array. The keySet function of the Map class makes no guarantees as to the iteration order of the keys in the Map object. This makes the test outcome non-deterministic, and the test fails whenever keySet returns a mismatch in order of the keys both the Maps. To fix this, the expected and actual maps' keys should be checked in a more deterministic way so that the assertions do not fail.

Fix

Expected and actual arrays can be compared without making assumptions about the order of the keys in these Maps. As the Matchers.containsInAnyOrder function compares the values inside the set of keys without needing order, the test becomes deterministic and ensures that the flakiness from the test is removed.

The PR does not introduce a breaking change.

Bharati Kulkarni bmk15897@gmail.com

Bharati Kulkarni <bmk15897@gmail.com>
@bmk15897
Copy link
Author

Can you please take a look at this pull request and approve it?

This was referenced Nov 25, 2022
@QueueUpX
Copy link
Collaborator

@bmk15897 This will be merged when a new release is planned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants