[SPARK-22726] [TEST] Basic tests for Binary Comparison and ImplicitTypeCasts #19918
Conversation
| -- distributed under the License is distributed on an "AS IS" BASIS, | ||
| -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| -- See the License for the specific language governing permissions and | ||
| -- limitations under the License. |
There was a problem hiding this comment.
Since the test cases are based on Apache Derby, I keep Apache license here.
| @@ -1,5 +1,5 @@ | |||
| -- Automatically generated by SQLQueryTestSuite | |||
| -- Number of queries: 31 | |||
| -- Number of queries: 32 | |||
There was a problem hiding this comment.
It sounds like the previous PR did not use our SPARK_GENERATE_GOLDEN_FILES to generate the result file.
| @@ -300,7 +300,7 @@ class SQLQueryTestSuite extends QueryTest with SharedSQLContext { | |||
| Locale.setDefault(originalLocale) | |||
|
|
|||
| // For debugging dump some statistics about how much time was spent in various optimizer rules | |||
| logInfo(RuleExecutor.dumpTimeSpent()) | |||
| logWarning(RuleExecutor.dumpTimeSpent()) | |||
There was a problem hiding this comment.
I have to change it to logWarning; otherwise, the time for each rule is not shown in the test result log. cc @cloud-fan
|
cc @cloud-fan @wangyum This is just an example. You can follow it to add more test cases for each type coercion rule. Later, we need to run the same SQLs in the Hive side too. |
|
Test build #84597 has finished for PR 19918 at commit
|
|
|
||
| CREATE TEMPORARY VIEW t AS SELECT 1; | ||
|
|
||
| SELECT cast(1 as binary) = '1' FROM t; |
There was a problem hiding this comment.
Seems binary comparison without <=>.
There was a problem hiding this comment.
yea <=> is missed. Do Derby tests also miss it?
There was a problem hiding this comment.
might be OK as <=> should have same type coercion rule as =
There was a problem hiding this comment.
=== is consistent with <=>
I think we can skip it. Then, we can run the whole suite to Hive with minor changes.
|
LGTM |
|
retest this please |
|
Test build #84719 has finished for PR 19918 at commit
|
|
retest this please |
1 similar comment
|
retest this please |
|
Test build #84727 has finished for PR 19918 at commit
|
|
Test build #84729 has finished for PR 19918 at commit
|
|
Since the failure in SparkR is not related to this PR, I will merge this. Thanks! Merged to master. |
What changes were proposed in this pull request?
Before we deliver the Hive compatibility mode, we plan to write a set of test cases that can be easily run in both Spark and Hive sides. We can easily compare whether they are the same or not. When new typeCoercion rules are added, we also can easily track the changes. These test cases can also be backported to the previous Spark versions for determining the changes we made.
This PR is the first attempt for improving the test coverage for type coercion compatibility. We generate these test cases for our binary comparison and ImplicitTypeCasts based on the Apache Derby test cases in https://github.com/apache/derby/blob/10.14/java/testing/org/apache/derbyTesting/functionTests/tests/lang/implicitConversions.sql
How was this patch tested?
N/A