Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scanner supports multiple family/qualifiers #37

Merged
merged 0 commits into from
Nov 13, 2012
Merged

Conversation

octo47
Copy link
Contributor

@octo47 octo47 commented Sep 23, 2012

Some quick implementation of multiple family/qualifiers passed to scanner.
If you have some complains or designs thoughts, feel free to write them. I'll fix any weird pieces of code.

Thank you.

@octo47
Copy link
Contributor Author

octo47 commented Sep 23, 2012

wow, last commit actually should go as separate pull request.

in case of empty values, exceptions looks like this

01:00:46.321 [New I/O  worker #1] ERROR org.hbase.async.RegionClient - Unexpected exception from downstream on [id: 0x6c91f005, /84.201.150.175:58795 => /95.108.139.59:60020]
java.lang.IndexOutOfBoundsException: Invalid index: 2968, maximum: 4
    at org.jboss.netty.buffer.CompositeChannelBuffer.componentId(CompositeChannelBuffer.java:677) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.buffer.CompositeChannelBuffer.getBytes(CompositeChannelBuffer.java:231) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:339) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:344) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.handler.codec.replay.ReplayingDecoderBuffer.readBytes(ReplayingDecoderBuffer.java:323) ~[netty-3.5.2.Final.jar:na]
    at org.hbase.async.KeyValue.fromBuffer(KeyValue.java:265) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.hbase.async.RegionClient.parseResults(RegionClient.java:1319) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.hbase.async.RegionClient.deserializeObject(RegionClient.java:1212) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.hbase.async.RegionClient.deserialize(RegionClient.java:1164) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.hbase.async.RegionClient.decode(RegionClient.java:1082) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.hbase.async.RegionClient.decode(RegionClient.java:82) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:502) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:487) ~[netty-3.5.2.Final.jar:na]
    at org.hbase.async.RegionClient.handleUpstream(RegionClient.java:938) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.hbase.async.HBaseClient$RegionClientPipeline.sendUpstream(HBaseClient.java:2408) ~[asynchbase-1.4.0-SNAPSHOT.jar:na]
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:91) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:373) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:247) ~[netty-3.5.2.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35) ~[netty-3.5.2.Final.jar:na]
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_33]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_33]
    at java.lang.Thread.run(Thread.java:680) [na:1.6.0_33]

@tsuna
Copy link
Member

tsuna commented Oct 12, 2012

It would be nice if we could provide this functionality with a simpler implementation. I'm especially not fond of the ColumnSpec class. I'd be tempted to do something along the lines of:

  private byte[][] families;
  private byte[][][] qualifiers;

  // ...

  /**
  * Specifies one more column qualifiers to scan within a given family.
  * @param family ...
  * @param qualifiers ...
  * @since 1.4
  */
  public void setQualifiers(final byte[] family, final byte[][] qualifiers) {
    add family to this.families[i];
    add qualifiers to this.qualifiers[i];
  }

Where families[i] would scan the qualifiers specified in qualifiers[i].

I think if we merge this change in, then it would override the one in #36, because it would be more general.

@octo47
Copy link
Contributor Author

octo47 commented Oct 12, 2012

Ok.
I thought about md arrays, but decided to make mo OOP decision.
I'll refactor it and remove this object.

@octo47
Copy link
Contributor Author

octo47 commented Oct 14, 2012

ColumnSpec class was removed.

@ghost ghost assigned tsuna Nov 4, 2012
@octo47 octo47 merged commit 93dcc81 into OpenTSDB:master Nov 13, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants