HBase support, anyone interested ?

[amimimor]

Hi Scalding,

I was wondering if something like:

class TestHBaseTable(args: Args) extends Job(args) {
  val table = new HBaseTable("hostname", "tablename", "columnForKey", Array("familyName"), Array("columnForData"))
  table.read
  .map[Array[Byte], Double]('columnForData -> 'times3) { _ * 3 }
  .project('times3)
  .write(Tsv("hdfs://localhost:8020/user/hdfs/data/out/debug/HbaseTable")) 
}

... is of interest.
I made a very naive implementation of Source I called HBaseSource (with just createTap that uses Maple's HBaseTap) and trait HBaseSchemeSource that inits the Maple HBaseScheme. Just a few lines of code and 2 implicits i added in FieldConversions for the Array[Byte] Hbase uses.

Let me know if you want me to add my implementation to Scalding through the fork-pull thing.

Amit

[azymnis]

Yeah we are using something like this internally. Feel free to do a pull request, or I could also try to copy our code over.

[amimimor]

body p { margin-bottom: 0cm; margin-top: 0pt; }
Well, I have also added an HBase sink,
which is currently not implemented in Maple, in addition to the
aforementioned HBaseSource impl.
Most of what I made is just copy-paste of other stuff your
fellows/you have made.
My current version (finally ... it took me a day - don't tell my
boss) can both read and write to/from HBase but I had to make
changes in Maple's HBaseTap and HBaseScheme because of a few bugs
I found. I got a few questions and it would be nice if you could
give me an email of somebody who can help me understand some of
the things in Maple's HBase library.
My private email is:  amit <a/T/> stalbetan . com
All the best,
Amit
On 3/7/2012 21:05, Argyris Zymnis wrote:

  Yeah we are using something like this internally. Feel free to do a pull request, or I could also try to copy our code over.

---

Reply to this email directly or view it on GitHub:
#120 (comment)

[azymnis]

You can send me an email at argyris at twitter.com. I was the one who ported that code from some version of an HBase tap for cascading v1. Did not test it extensively except to make sure that it works for the use case I needed it for. If you found any bugs, send a pull request on maple.

[ehartford]

+1 bump for interest!

[johnynek]

https://github.com/twitter/scalding/wiki/Scalding-HBase

There are many options now. Let's close this issue and start another if the existing HBase options don't work.