PARQUET-1115: Warn users when misusing parquet-tools merge #433
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zivanfi
reviewed
Nov 2, 2017
| public String getCommandDescription() { | ||
| return "Merges multiple Parquet files into one. " + | ||
| "The command doesn't merge row groups, just places one after the other. " + | ||
| "When used to merge many small files, the resulting file will still contain small row groups."; |
There was a problem hiding this comment.
I would add ", which usually leads to bad query performance."
|
|
||
| if (tooSmallFilesMerged) { | ||
| out.println("Warning: you merged too small files. " + | ||
| "Despite the size of the merged file is bigger, it STILL contains small row groups, thus you don't have the advantage of big row groups!"); |
There was a problem hiding this comment.
I would use stronger wording after the last comma, like "which usually leads to bad query performance." (Additionally, I'm not a native speaker either, but I believe you either need a "that" after "Despite" or use some other word, for example "Although".)
|
@zivanfi thanks for you review, changed the wording accordingly. |
zivanfi
approved these changes
Nov 3, 2017
zivanfi
requested changes
Nov 7, 2017
|
@zivanfi removed the whitespace-only changes |
hubot
pushed a commit
that referenced
this pull request
Nov 7, 2017
Author: Nandor Kollar <nkollar@cloudera.com> Closes #433 from nandorKollar/PARQUET-1115 and squashes the following commits: 5504a39 [Nandor Kollar] PARQUET-1115: Warn users when misusing parquet-tools merge f2ece26 [Nandor Kollar] PARQUET-1115: Warn users when misusing parquet-tools merge 4f3ec99 [Nandor Kollar] PARQUET-1115: Warn users when misusing parquet-tools merge f97e620 [Nandor Kollar] PARQUET-1115: Prevent users from misusing parquet-tools merge
zivanfi
approved these changes
Nov 7, 2017
|
It might be nice if we actually suggested an alternative instead of just saying "don't do this." You can see my solution at https://gist.github.com/NickCrews/7a47ef4083160011e8e533531d73428c. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.