HIVE-27995: Fix inconsistent behavior of LOAD DATA command for partitoned and non-partitioned tables #5000
Conversation
There was a problem hiding this comment.
you need to add a test, there are NegativeCLIDrivers which can be used.
There are bunch of return statements below the current line & the place where it is there right now, So, if that returned was called earlier, this applyConstraintsAndGetFiles(fromURI, ts.tableHandle); must not have been called, but now you would be calling that, right?. So, that is some additional cost just for error message?
asf-ci-hive
added
tests pending
tests failed
and removed
tests unstable
tests pending
labels
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
|
@ayushtkn Thanks for the review, i have added the test case as mentioned above.please review once |
|
There is a test failure in Jenkins, i ran the test in in local and that test is passing which can be viewed in the below screenshot: |
asf-ci-hive
added
tests failed
tests pending
and removed
tests pending
tests failed
labels
asf-ci-hive
added
tests pending
tests unstable
and removed
tests passed
tests pending
labels
asf-ci-hive
added
tests pending
tests failed
tests unstable
tests passed
and removed
tests unstable
tests pending
tests failed
labels
| CREATE TABLE validate_load_data(key int, value string) partitioned by (hr int) STORED AS TEXTFILE; | ||
| LOAD DATA INPATH 'hdfs:///validateload/filedoesnt.txt' INTO TABLE validate_load_data partition (hr); | ||
| SELECT * FROM validate_load_data; | ||
| DROP TABLE validate_load_data; |
There was a problem hiding this comment.
Can you check this, without fix also it is throwing error.
There was a problem hiding this comment.
@chinnaraolalam the following scenario is now handled, can you review this once
asf-ci-hive
added
tests pending
tests unstable
and removed
tests passed
tests pending
labels
…oned and non-partitioned tables
|
What changes were proposed in this pull request?
Earlier, the code flows for partitioned and non-partitioned tables were different. The partitioned tables skipped constraints checks before submitting the job for execution. This Pull request ensures that both, the partitioned and non-partitioned tables go through the constraints validations in
applyConstraintsAndGetFilesfunction.Why are the changes needed?
For partitioned tables, while executing LOAD DATA/ LOAD DATA LOCAL commands, the check for file existence is not executed on HiveServer2, and this in turn throws
java.io.FileNotFoundExceptionduring Runtime once the job is launched.This PR prevents such cases at compile time.
Does this PR introduce any user-facing change?
No
Is the change a dependency upgrade?
No
How was this patch tested?
The test cases already exist. The error messages prompted back to the user are now consistent if the file is not found at HiveServer2
Load Data Error (Non Partitioned Tables)
File Not Found Exception (Partitioned Tables)
Fixed: For partitioned tables
