[SPARK-9876] Revert "[SPARK-9876][SQL] Update Parquet to 1.8.1." #13445
Conversation
This reverts commit 776d183.
|
Sounds reasonable, but the "regression" wasn't located or even confirmed to exist after this change was reverted the last time. There was also no follow-up on it. If we revert the change I suggest that we apply it to master as soon as 2.0 is branched. That way we don't adversely affect 2.0, but we do get this addressed. Is that a reasonable path forward? |
|
@rdblue Since 2.0 branch has been cut, I am little bit concerned about potential merge conflicts when we cherry-pick bug fixes into 2.0 branch before the release if we do not revert it from the master. We can revisit the upgrade after the release and resolve the regression issue (We will run our benchmark again to check this issue) |
|
btw, we observed an error when filter pushdown is enabled. Unfortunately, we missed the exception... |
|
Test build #59746 has finished for PR 13445 at commit
|
|
@rdblue How about we merge this to master and branch 2.0? Feel free to open your PR again. We can figure out the perf thing with @liancheng together. |
|
OK. Let me create another PR for branch 2.0. We will merge that one first. |
|
I am closing this for now. |
What changes were proposed in this pull request?
Since we are pretty late in the 2.0 release cycle, it is not clear if this upgrade can be tested thoroughly and if we can resolve the regression issue that we observed before. This PR temporarily reverts #13280. Let's revisit it after the release of 2.0.
How was this patch tested?
Existing tests
This reverts commit 776d183.