build index for hbase

[at15]

for instance we have a htable like this

rowkey  info:name info:email
1            jack          jack@123.com
2            dick          dick@123.com

we need a table to index this table by info:name

rowkey row
dick       2
jack      1

use mapreduce to do this

• setup hbase with existing hadoop
• write fake data
• write the mapper
• see the result
• send mail to TA

[at15]

why I only knew this today .... e ....

[at15]

some refs

http://www.informit.com/articles/article.aspx?p=2262143&seqNum=2

[at15]

Exception in thread "main" org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1 action: contacts: 1 time, 
    at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
    at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
    at org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
    at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
    at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
    at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1449)
    at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1040)
    at cn.edu.sjtu.stu.at15.hbase.Seeder.main(Seeder.java:33)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

this is caused by

• not creating table/column family

the class won't create the table automatically I guess ....

[at15]

https://books.google.com/books?id=nUhiQxUXVpMC HBase: The Definitive Guide

[at15]

the exact example .... https://github.com/apache/hbase/blob/master/hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/IndexBuilder.java

[at15]

finished in 60c1b83