-
Notifications
You must be signed in to change notification settings - Fork 13.8k
[FLINK-14694] [table-planner-blink] Most tests from package o.a.f.table.planner.functions.aggfunctions are not executed during mvn test and fix BinaryGeneric comparison failure #10158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…le.planner.functions.aggfunctions are not executed during mvn test and fix BinaryGeneric comparison failure
|
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community Automated ChecksLast check on commit 6905159 (Wed Dec 04 15:23:50 UTC 2019) Warnings:
Mention the bot in a comment to re-run the automated checks. Review Progress
Please see the Pull Request Review Guide for a full explanation of the review process. DetailsThe Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commandsThe @flinkbot bot supports the following commands:
|
|
cc @dawidwys |
dawidwys
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @godfreyhe thank you for the PR. I would suggest improving the code quality of the AggFunctionTestBase#validateResult method.
As for the general problem with the tests I think those tests are much better suited for parameterized tests. You can see some examples of such tests e.g. here: org.apache.flink.table.types.LogicalTypeParserTest. They change just the type of the accumulator, if I understood the tests correctly.
I am fine though with the changes in this PR for now, but would strongly encourage to not introduce any more of such complex tests hierarchy (at least 3 levels of inheritance)
| assertEquals(e.mapSize, r.mapSize); | ||
| } else { | ||
| // verify null | ||
| if (expected == null || result == null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code block can be significantly improved:
protected <E> void validateResult(E expected, E result, TypeInformation<?> typeInfo) {
if (expected instanceof BigDecimal && result instanceof BigDecimal) {
// BigDecimal.equals() value and scale but we are only interested in value.
assertEquals(0, ((BigDecimal) expected).compareTo((BigDecimal) result));
} else if (expected instanceof MinWithRetractAccumulator &&
result instanceof MinWithRetractAccumulator) {
MinWithRetractAccumulator e = (MinWithRetractAccumulator) expected;
MinWithRetractAccumulator r = (MinWithRetractAccumulator) result;
assertEquals(e.min, r.min);
assertEquals(e.mapSize, r.mapSize);
} else if (expected instanceof MaxWithRetractAccumulator &&
result instanceof MaxWithRetractAccumulator) {
MaxWithRetractAccumulator e = (MaxWithRetractAccumulator) expected;
MaxWithRetractAccumulator r = (MaxWithRetractAccumulator) result;
assertEquals(e.max, r.max);
assertEquals(e.mapSize, r.mapSize);
} else if (expected instanceof BinaryGeneric && result instanceof BinaryGeneric) {
TypeSerializer<?> serializer = typeInfo.createSerializer(new ExecutionConfig());
assertThat(
(BinaryGeneric) result,
equivalent((BinaryGeneric) expected, new BinaryGenericSerializer<>(serializer)));
} else if (expected instanceof GenericRow && result instanceof GenericRow) {
validateGenericRow(
(GenericRow) expected,
(GenericRow) result,
(BaseRowTypeInfo) typeInfo);
} else {
assertEquals(expected, result);
}
}
private void validateGenericRow(GenericRow expected, GenericRow result, BaseRowTypeInfo typeInfo) {
assertEquals(expected.getArity(), result.getArity());
for (int i = 0; i < expected.getArity(); ++i) {
Object expectedObj = expected.getField(i);
Object resultObj = result.getField(i);
validateResult(expectedObj, resultObj, typeInfo.getTypeAt(i));
}
}
use Parameterized instead of Enclosed to simplify test cases
|
@dawidwys thanks so much for the suggestion, and I have updated the PR |
…est compilation error due to incompatible types This partially reverts the commit dcc1330 and uses the initial approach from apache#10158 of using Enclosed annotation. The problem with the TestSpec approach was that the top level classes shouldn't use generics as they are stripped when instantiated by JUnit runner which may lead to generic signature mismatch.
…est compilation error due to incompatible types This partially reverts the commit dcc1330 and uses the initial approach from #10158 of using Enclosed annotation. The problem with the TestSpec approach was that the top level classes shouldn't use generics as they are stripped when instantiated by JUnit runner which may lead to generic signature mismatch. This closes #10915
…est compilation error due to incompatible types This partially reverts the commit dcc1330 and uses the initial approach from #10158 of using Enclosed annotation. The problem with the TestSpec approach was that the top level classes shouldn't use generics as they are stripped when instantiated by JUnit runner which may lead to generic signature mismatch. This closes #10915
…est compilation error due to incompatible types This partially reverts the commit dcc1330 and uses the initial approach from apache#10158 of using Enclosed annotation. The problem with the TestSpec approach was that the top level classes shouldn't use generics as they are stripped when instantiated by JUnit runner which may lead to generic signature mismatch. This closes apache#10915
What is the purpose of the change
Most tests from package o.a.f.table.planner.functions.aggfunctions are not executed during mvn test because those test case classed are abstract class and will be ignored when running
mvn test. In FLINK-13702,BinaryGenericcan't be compared directly, so those failures are also ignored. This pr aims to fix both. The first one's solution is introducing Enclosed runner, and moving the abstract class as an inner class, and making sure the outer class is a non-abstract class. The second one's solution is verifyingBinaryGenericusingBinaryGenericAsserter.equivalent(should also consider thatGenericRowhasBinaryGenericsub-type)Brief change log
Enclosedrunner and make sure the outer class is a non-abstract classBinaryGenericusingBinaryGenericAsserter.equivalentVerifying this change
This change is already covered by existing tests.
Does this pull request potentially affect one of the following parts:
@Public(Evolving): (yes / no)Documentation