HBase counters

[pinifaran]

added support for using hbase counters as the underlying implementation of opentsdb
added support for summing up detected duplicate counters (instead of discarding them)

[johann8384]

This still isn't rebased off the latest Next, did you forget to do a git pull or something?

Do a "git fetch --all; git rebase origin/next" (assuming origin is what you named the OpenTSBD fork, you might have called it something else)

[pinifaran]

I did everything (fork, create pull request...) from github.com, not from command line.

anyway, i just did what you asked from command line (Git Shell), and now it appears I'm synched:

C:\Users\pini\Documents\GitHub\opentsdb [next]> git fetch --all; git rebase origin/next
Fetching origin
Fetching upstream
Fetching OpenTSDB
First, rewinding head to replay your work on top of it...
Fast-forwarded next to origin/next.

However, i see that there are still a few conflicts:

C:\Users\pini\Documents\GitHub\opentsdb [next]> git merge upstream/next
Removing third_party/hbase/asynchbase-1.7.0-20150910.030815-3.jar.md5
Auto-merging src/utils/Config.java
CONFLICT (content): Merge conflict in src/utils/Config.java
Auto-merging src/tools/TextImporter.java
Auto-merging src/core/TSDB.java
CONFLICT (content): Merge conflict in src/core/TSDB.java
Auto-merging src/core/IncomingDataPoints.java
Auto-merging src/core/CompactionQueue.java
Automatic merge failed; fix conflicts and then commit the result.

[pinifaran]

Ok i fixed the 2 conflicts and made git commit.

[johann8384]

Assigning to @manolama as he wanted to review this.

[pinifaran]

ok, great.

[manolama]

Jdoc and formatting.

[manolama]

Thanks for the attempt!
Atomic increments are actually pretty expensive (locking, modifying plus they store 8 bytes). Therefore we don't want to penalize all TSD integers when the vast majority of metrics are going to be write-once. Instead of checking the data point type in the addPointInternal() method, we need a different API like for telnet, "putctr" and for HTTP you can add a flag that indicates they're counters. Then add a method write method that performs the increment. That way only metrics that need it can use the expensive path.

Otherwise also watch out for:

• TSD variable naming styles (no camel case, yeah Cish)
• Don't remove or modify public APIs, add overloads
• JDoc all of your new functions.

Thanks!

[pinifaran]

Hi Chris\Jonathan,

I've committed today the relevant fixes as you suggested:

1. reverted back most of my previous changes (modified back the original Deferred signature, etc...).
2. added a new command "putctr" which writes the data point value as HBase counters.
3. relevant files for the new command:
   TSDB.java: added new method addCounter
   PutDataPointCounterRpc.java: a new file which implements writing the value as HBase counter
   RpcManager.java: registers PutDataPointCounterRpc instance for the "putctr" command.
4. Also added comments (JDoc style) where applicable.

I would be happy if you could review and send comments.

Thanks in advance!

[Pinif]

I just saw 2.4.0 was released ~ 6 months ago. was this feature merged in? why is it still open?

[amariniml]

hi, this is will be merged?

[Pinif]

hi, this is will be merged?

My question exactly... I've committed this code ~ 3 years ago, i don't know why it was not merged at the time.

[johann8384]

@Pinif Because the active maintainers got other jobs where we don't get to use OpenTSDB.