[CALCITE-3140][CALCITE-3141] Failures in slow tests & CI#1375
Merged
zabetak merged 2 commits intoapache:masterfrom Nov 27, 2019
Merged
[CALCITE-3140][CALCITE-3141] Failures in slow tests & CI#1375zabetak merged 2 commits intoapache:masterfrom
zabetak merged 2 commits intoapache:masterfrom
Conversation
danny0405
reviewed
Aug 14, 2019
core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
Outdated
Show resolved
Hide resolved
zabetak
changed the title
vlsi
reviewed
Nov 21, 2019
vlsi
reviewed
Nov 21, 2019
vlsi
reviewed
Nov 21, 2019
vlsi
reviewed
Nov 21, 2019
Member
Author
|
@vlsi I am planning to rebase soon in order to remove all those messy commits and take care of your comments. Let me know if you want me to wait a bit longer. |
rubenada
reviewed
Nov 22, 2019
Contributor
Please feel free to rebase whenever you think it is right. |
zabetak
force-pushed
the
calcite-3140-3141
branch
2 times, most recently
from
ef44935 to
63f89c3
Compare
zabetak
changed the title
vlsi
reviewed
Nov 22, 2019
Contributor
|
There's https://stackoverflow.com/a/58618280/1261287 , so we could add the following (to the regular CI workflow) to enable running slow tests in case it is needed (e.g. in your case when you are altering the job config, and nobody is sure it would work without trying) WDYT? |
Member
Author
|
Adding a conditional job seems like a good idea! I will give it a try. Other than that there are ways to test that the new config works (e.g., https://github.com/zabetak/calcite/runs/315831584). |
zabetak
commented
Nov 24, 2019
1. Handle errors in SqlOperatorBaseTest#testArgumentBounds skipping problematic calls with appropriate explanation. 2. Introduce Bug#CALCITE_3243 related with failures in SqlOperatorBaseTest#testArgumentBounds. 3. Errors and exceptions in SqlOperatorBaseTest#testArgumentBounds are not that different so log everything instead of rethrowing errors. 4. Improve the error messages in SqlOperatorBaseTest#testArgumentBounds by trying to find the most descriptive cause for the failure. 5. Use logger instead of System.out in SqlOperatorBaseTest#testArgumentBounds for printing messages. 6. Skip queries 2 and 7 of TpchTest due to CALCITE-2223 and infinite planning bug. 7. Increase the timeout by 2 in FoodmartTest to avoid (marginal) failures (i.e., queries 2420, 5894) in slower machines. 8. Update stale plan in TpcdsTest#testQuery17Plan.
1. Replace @category(SlowTests.class) with @tag("slow"). 2. Replace junit4 APIs with junit5 APIs when @tag annotation is used since mixed usages do not work well together. 3. Use exclusively the @tag("slow") annotation for tagging slow tests. 4. Remove usages of calcite.test.slow system property for annotating slow tests. 5. Replace @ignore("slow") with @tag("slow") enabling a few more slow tests. 6. Add testSlow task in Gradle for running slow tests. 7. Exclude slow test execution from normal Gradle build and test. 8. Add GitHub Action for running slow tests on demand (by adding 'slow-tests-needed' label) in a PR and on every push to the master. 9. Update site instructions for running slow tests. 10. Silence logger in SqlOperatorBaseTest since it generates huge logs. 11. Remove execution of slow test from Travis. 12. Refactor FoodmartTest based on the needs of the new @ParameterizedTest API. 13. Add header and description in main.yml file.
zabetak
force-pushed
the
calcite-3140-3141
branch
from
fc27344 to
f0ad6c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.