HBASE-24577 Doc WALSplitter classes #1913
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
infraio
reviewed
Jun 17, 2020
| @@ -233,7 +247,7 @@ public static boolean splitLogFile(Path walDir, FileStatus logfile, FileSystem w | |||
| } | |||
|
|
|||
| /** | |||
| * log splitting implementation, splits one log file. | |||
| * Log splitting implementation, splits one log file. | |||
anoopsjohn
approved these changes
Jun 17, 2020
| import org.apache.hbase.thirdparty.com.google.common.collect.Lists; | ||
|
|
||
| /** | ||
| * Class that manages the output streams from the log splitting process. | ||
| * Every region only has one recovered edits. | ||
| * Every region only has one recovered edits file. |
There was a problem hiding this comment.
Per one WAL file split, every region will have one recovered edits file. If another WAL file also get split, we will have one more right?
There was a problem hiding this comment.
Let me add qualification here.
| /** | ||
| * True if we are to run with bounded amount of writers rather than let the count blossom. | ||
| * Bounded writing tends to have higher throughput. | ||
| */ | ||
| public final static String SPLIT_WRITER_CREATION_BOUNDED = "hbase.split.writer.creation.bounded"; |
There was a problem hiding this comment.
This defaults to false right?
There was a problem hiding this comment.
Let me add this.
There was a problem hiding this comment.
I note that it defaults to 'false' but man, this should be 'true'. Let me test.
virajjasani
reviewed
Jun 17, 2020
| String regionName = Bytes.toString(buffer.encodedRegionName); | ||
| for (Map.Entry<String, CellSet> cellsEntry : familyCells.entrySet()) { | ||
| String familyName = cellsEntry.getKey(); | ||
| StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName, regionName, | ||
| familySeqIds.get(familyName), familyName, isMetaTable); | ||
| LOG.trace("Created {}", writer.getPath()); |
There was a problem hiding this comment.
Good to guard with LOG.isTraceEnabled()?
There was a problem hiding this comment.
Why you say? The slf4j doesn't do any work creating log string if we are not at log TRACE level.
There was a problem hiding this comment.
This is to avoid the cost of parameter construction: http://logging.apache.org/log4j/1.2/manual.html#performance
There was a problem hiding this comment.
My bad, this is old doc for apache log4j, not slf4j
There was a problem hiding this comment.
You are right, I just looked into one of the implementors of slf4j (Log4jLoggerAdapter) and it already covers isTraceEnabled check:
public void trace(String format, Object arg) {
if (isTraceEnabled()) {
FormattingTuple ft = MessageFormatter.format(format, arg);
logger.log(FQCN, traceCapable ? Level.TRACE : Level.DEBUG, ft.getMessage(), ft.getThrowable());
}
}
We are following explicit guarding for the cases of Trace and Debug level log at multiple places, all are redundant checks since library already does it for us.
There was a problem hiding this comment.
Thanks @virajjasani
There was a problem hiding this comment.
IIRC, this extra check could be used only if the parameter self will cause a heavy call.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
Show resolved
Hide resolved
|
🎊 +1 overall
This message was automatically generated. |
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Guanghao Zhang <zghao@apache.org>
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
saintstack
added a commit
that referenced
this pull request
Jun 17, 2020
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Guanghao Zhang <zghao@apache.org>
saintstack
added a commit
that referenced
this pull request
Jun 17, 2020
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Guanghao Zhang <zghao@apache.org>
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
ramkrish86
reviewed
Jun 18, 2020
| @@ -81,6 +83,7 @@ private RecoveredEditsWriter getRecoveredEditsWriter(TableName tableName, byte[] | |||
| if (ret == null) { | |||
| return null; | |||
| } | |||
| LOG.trace("Created {}", ret.path); | |||
There was a problem hiding this comment.
Just asking. this can be in DEBUG mode right rather than trace?
clarax
pushed a commit
to clarax/hbase
that referenced
this pull request
Nov 15, 2020
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Guanghao Zhang <zghao@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.