Fix flaky tungsten test using comparator #8232
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes the non deterministic behaviour of 2 tests. This happens as the tests assume the return value of an API to be ordered when it is actually not guaranteed. This has been detected using NonDex tool.
Reproduction of issue
mvn install -pl plugins/network-elements/tungsten
mvn -pl plugins/network-elements/tungsten edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.cloudstack.network.tungsten.service.TungstenApiTest#listTungstenNetworkTest -DnondexMode=ONE
Failure log:
Root cause
The test class
org.apache.cloudstack.network.tungsten.service.TungstenApiTest
uses ApiConnectorMock for all tests. The testslistTungstenNetworkTest
,listTungstenPolicyTest
use thelist
method of this connector Tungsten API connector call in their respective tests.The 3rd party library juniper-tungsten-api's
net.juniper.tungsten.api.ApiConnectorMock
'slist
method does guarantee the ordering of results. Unfortunately, the source code is not on github for direct inspection. Upon downloading and decompiling thejuniper-tungsten-api-2.0.jar
, it can be seen that this method usesHashMap.values()
which is unordered, for the population of the returned list. A screenshot has been added below. Decompilation was done using Java-decompiler.Fix
The test class already defines a comparator to sort objects of
ApiObjectBase
here. This has been used to ensure consistent test behaviour.Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
mvn -pl plugins/network-elements/tungsten test -Dtest=org.apache.cloudstack.network.tungsten.service.TungstenApiTest
)mvn -pl plugins/network-elements/tungsten edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.cloudstack.network.tungsten.service.TungstenApiTest
)How did you try to break this feature and the system with this change?
The change only touches test files and not source code. Hence it will not affect production. Test behavior consistency has also been verified