-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flaky test fails intermittently in GitHub Action #821
Comments
I think this test case should be removed, as it never guarantees the behavior. To identify the max depth supported, we can test through configuration. And environment resource related runtime error expectation need not be tested. |
Might be better to see if the test can be fixed, although if you see a chance to add a unit test, that is fine too. |
@stleary , I'm leaning towards just marking the test as a "skip/ignore" or removing it altogether. It seems like this test would need regular maintenance to ensure that it was consistently hitting stack overflows. @Ignore("This test relies on system constraints and may not always pass.")
@Test(expected = JSONException.class)
public void issue654StackOverflowInputWellFormed() {
//String input = new String(java.util.Base64.getDecoder().decode(base64Bytes));
final InputStream resourceAsStream = JSONObjectTest.class.getClassLoader().getResourceAsStream("Issue654WellFormedObject.json");
JSONTokener tokener = new JSONTokener(resourceAsStream);
JSONObject json_input = new JSONObject(tokener);
assertNotNull(json_input);
fail("Excepected Exception.");
} |
@johnjaylward Fair enough, better to silence the test, than have it fail intermittently. |
@johnjaylward - are you annotating the test with Ignore? |
add ignore annotation to tests that may fail due to differences in machine resources and can't be controlled via the tests
add ignore annotation to tests that may fail due to differences in machine resources and can't be controlled via the tests
Test case passes on some runs and fails on others:
org.json.junit.JSONObjectTest.issue654StackOverflowInputWellFormed
JSON-java/src/test/java/org/json/junit/JSONObjectTest.java
Lines 3668 to 3676 in b5b9f63
Appears to be due to differences in resources available on each run. The exception expected is a "JSONException", but that's because it is wrapping a "StackOverflowError". StackOverflowError is going to be dependent on the environment and may not hit consistently.
The text was updated successfully, but these errors were encountered: