Skip to content

Commit

Permalink
HBASE-23106 [HBCK2/hbase-operator-tools] Add a WAL verifier
Browse files Browse the repository at this point in the history
Adds simple WALReader.
  • Loading branch information
saintstack committed Oct 3, 2019
1 parent 7ee6d59 commit 7a94401
Show file tree
Hide file tree
Showing 3 changed files with 77 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,7 @@ public boolean progress() {
// and transition the node back to FAILED_OPEN. If that fails,
// we rely on the Timeout Monitor in the master to reassign.
LOG.error(
"Failed open of region=" + this.regionInfo.getRegionNameAsString(), t);
"Failed open of {}", this.regionInfo.getRegionNameAsString(), t);
}
return region;
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/*
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hadoop.hbase.wal;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HBaseInterfaceAudience;
import org.apache.yetus.audience.InterfaceAudience;

import java.io.IOException;

/**
* Read the passed in WAL passed on command-line (or recovered.edits file since it has same
* format). Stamps out offset, key, and value size of edit seen. Use to verify WAL or
* recovered.edits files.
*/
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
public class WALReader {
private static void read(WALFactory factory, FileSystem fs, Path path) throws IOException {
System.out.println(path.toString());
try (WAL.Reader reader = factory.createReader(fs, path)) {
WAL.Entry entry = new WAL.Entry();
while((entry = reader.next(entry)) != null) {
System.out.println(reader.getPosition() + " " + entry.getKey().toString() +
" " + entry.getEdit().estimatedSerializedSizeOf());
}
}
}

public static void main(String [] args) throws IOException {
if (args.length < 1) {
System.err.println("Usage: WALReader <FILE> [<FILE>...]");
return;
}
Configuration configuration = HBaseConfiguration.create();
FileSystem fs = FileSystem.get(configuration);
WALFactory factory = null;
try {
factory = WALFactory.getInstance(configuration);
for (String p: args) {
read(factory, fs, new Path(p));
}
} finally {
if (factory != null) {
factory.close();
}
}
}
}
9 changes: 9 additions & 0 deletions src/main/asciidoc/_chapters/ops_mgt.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -879,6 +879,15 @@ The output can optionally be mapped to another set of tables.

WALPlayer can also generate HFiles for later bulk importing, in that case only a single table and no mapping can be specified.

.WALReader
[NOTE]
====
Since hbase-2.3.0/2.2.2/2.1.8, a WALReader tool has been added. Pass it WALs or recovered.edits files,
which are of the same format, and it will read the content spitting out offset, key names, and size
of edit. Useful looking at content or just verifying a file readable.
====


Invoke via:

----
Expand Down

0 comments on commit 7a94401

Please sign in to comment.