Skip to content
Commits on Mar 19, 2012
  1. Properly unescape the format and label when parsing them back from th…

    …e url
    Will Moss committed Mar 19, 2012
Commits on Mar 18, 2012
  1. Merge remote-tracking branch 'stumbleupon/master'

    * stumbleupon/master: (36 commits)
      Upgrade to asynchbase 1.2.0.
      Add parameters to change the foreground and background colors.
      Don't throw unnecessary exceptions when parsing timestamps.
      Javadoc fixes.
      Simplify computation of standard deviations.
      Add one more unit test for stddev aggregator.
      Standard deviation aggregator.
      Fix helper functions when using varying UniqueId sizes.
      Licensing adjustment: allow LGPLv2.1+ instead of LGPLv3+.
      Start version 1.1.0.
      Add a bit of README and some NEWS.
      Fix the knob to enable/disable TSD compactions.
      Use semantic versioning.
      Upgrade to GWT 2.4.0.
      Upgrade to Netty 3.2.7.Final.
      Upgrade to ZooKeeper 3.3.4.
      Upgrade to SLF4J 1.6.4.
      Upgrade to logback 1.0.0.
      Upgrade to PowerMock 1.4.10.
      Upgrade to Mockito 1.9.0.
    Will Moss committed Mar 18, 2012
Commits on Mar 4, 2012
  1. @tsuna

    Upgrade to asynchbase 1.2.0.

    Change-Id: Ibb0d17f7eb77809893a2009052454969183fc2cc
    tsuna committed Mar 3, 2012
Commits on Feb 27, 2012
  1. @tsuna

    Add parameters to change the foreground and background colors.

    The query API now supports two more query string parameters,
    `fgcolor' and `bgcolor' to set (respectively) the foreground
    and background colors.  Acceptable values are of the form
    `xRRGGBB' where RGB values are in hexadecimal.  For `bgcolor'
    the special value `transparent' can also be used.
    This closes #43 and #94.
    Change-Id: I9b451fb1b340fc80d4169a0c8b6c1b37af8f5339
    tsuna committed Feb 23, 2012
Commits on Feb 23, 2012
  1. @tsuna

    Don't throw unnecessary exceptions when parsing timestamps.

    Change-Id: Ib766c8e197f42331f8620c05280757aa10f3d591
    tsuna committed Feb 22, 2012
  2. @ryang-sfdc @tsuna

    Javadoc fixes.

    Change-Id: Ibb0c006374ba1ca9ee3c65f6fe04a708ad6d2d7f
    Signed-off-by: Benoit Sigoure <>
    ryang-sfdc committed with tsuna Jan 12, 2012
Commits on Feb 22, 2012
  1. @tsuna

    Simplify computation of standard deviations.

    Change-Id: Icbf7aba4103c029bd858701df82904b21ef0b64c
    tsuna committed Feb 22, 2012
  2. @tsuna

    Add one more unit test for stddev aggregator.

    Change-Id: Icdc510af28e6f6f35b7dd9fc6970d77ff1520a82
    tsuna committed Feb 22, 2012
  3. @deusaquilus @tsuna

    Standard deviation aggregator.

    Change-Id: I35aecd7433495fd7bfd99ebb2803416bae5fe0b1
    Signed-off-by: Benoit Sigoure <>
    deusaquilus committed with tsuna Jan 22, 2012
  4. @tsuna

    Fix helper functions when using varying UniqueId sizes.

    By default all UniqueId instances created by OpenTSDB encode the UIDs
    on 3 bytes (for metrics, tag names and tag values).  There were 2 bugs
    that would crop up if one configured tag names and tag values to use
    a different number of bytes to encode the UIDs.
    Change-Id: I15a01c513a18e75c69c1ade1abeb3e1602419fa9
    tsuna committed Feb 22, 2012
Commits on Feb 4, 2012
  1. @tsuna

    Licensing adjustment: allow LGPLv2.1+ instead of LGPLv3+.

    Until now, OpenTSDB was available under the terms of the LGPL,
    version 3 or any later version.  This change also allows the
    version 2.1 of the LGPL license.  Those who already accepted
    the terms of the LGPLv3 can keep doing so, because LGPLv2.1
    also states that one can use any later version.
    The net result of this change is that one can now use OpenTSDB
    under the terms of LGPLv2.1 or GPLv2, if they so chose.
    Change-Id: I67e0d47206db6a5544fdadc9c5fb9cd9466c1ad3
    tsuna committed Feb 4, 2012
Commits on Dec 23, 2011
  1. @tsuna

    Start version 1.1.0.

    Change-Id: If507ed9fedad75da306a24a9abf93d62013c5510
    tsuna committed Dec 23, 2011
  2. @tsuna

    Add a bit of README and some NEWS.

    Change-Id: Ic2bd5f3f1946edbe148a8b15165861a561013f5e
    tsuna committed Dec 23, 2011
  3. @tsuna

    Fix the knob to enable/disable TSD compactions.

    Compactions are enabled if the system property
    `tsd.feature.compactions' is set and its value
    is not the string "false".
    Previously the code would enable compactions
    as long as the system property was set, which
    makes it hard to disable them in the future
    once they're enabled by default.
    Change-Id: If58a3c269c1b6340c0176fbc793eb9d3fb3dea8e
    tsuna committed Dec 23, 2011
  4. @tsuna

    Use semantic versioning.

    Change-Id: I050ce5a7e6f1919c013d71a8b35dc83a227b4a51
    tsuna committed Dec 23, 2011
  5. @tsuna

    Upgrade to GWT 2.4.0.

    Change-Id: Ib50f47de57523c9ac4125b2eb153b3ee3a6003d1
    tsuna committed Dec 23, 2011
  6. @tsuna

    Upgrade to Netty 3.2.7.Final.

    Change-Id: I1293c3a7d95017e8a4279e0b28d8c4c31bb180fc
    tsuna committed Dec 23, 2011
  7. @tsuna

    Upgrade to ZooKeeper 3.3.4.

    Change-Id: I87141ebbfdc143749644c9bc5d4997e46904b061
    tsuna committed Dec 23, 2011
  8. @tsuna

    Upgrade to SLF4J 1.6.4.

    Change-Id: Ie6acf8c253ab9cad26f86756f5557a7268f2fa20
    tsuna committed Dec 23, 2011
  9. @tsuna

    Upgrade to logback 1.0.0.

    Change-Id: Ifd3d2e81646039e435531047b7a036dd8c19b816
    tsuna committed Dec 23, 2011
  10. @tsuna

    Upgrade to PowerMock 1.4.10.

    Change-Id: Ic49c7e5af251e9a68e4b922b49fed02ffe2198c6
    tsuna committed Dec 23, 2011
  11. @tsuna

    Upgrade to Mockito 1.9.0.

    Change-Id: I422549714e3b464e0dc70ff559db84811c84b92d
    tsuna committed Dec 23, 2011
  12. @tsuna

    Update to JUnit 4.10.

    Change-Id: I46502c17b05a6581f32da9ab8a8b256bc522aa50
    tsuna committed Dec 23, 2011
  13. @tsuna

    Properly handle malformed query strings.

    Change-Id: Iab42fa9429fb32156a22fcf7cdab974a5a2c14db
    tsuna committed Nov 16, 2011
  14. @tsuna

    Add support for compacted cells in fsck.

    Change-Id: Iceeb1ab40026087a3307f24dd38bebf9e21a5198
    tsuna committed Oct 2, 2011
  15. @tsuna

    Stop (ab)using reflection in tools.

    Instead create an `Internal' module that exposes some internal API.
    Although this breaks encapsulation, both the name of the module and
    its Javadoc comment make it clear that this module isn't part of the
    public API.  This makes the code easier to program / maintain than
    with reflection, which is a PITA in Java (and that's not necessarily
    a bad thing).
    Change-Id: I6bf27c0b45524f605838db8b4c7b9d721f435a68
    tsuna committed Sep 29, 2011
  16. @tsuna

    Don't flush the entire compaction queue on shutdown.

    It's just too much work and most of it is unnecessary.  Here's why.
    In a typical setup, you'd have at least 2 TSD to be N + 1.  If you
    shutdown the first one, all the traffic will shift to the other one
    by virtue of tcollectors reconnecting elsewhere.  This is going to
    "fault in" almost all the rows that were in the compaction queue of
    the TSD that got shutdown, so the other TSD will eventually compact
    them.  It's possible that few rows will be left behind, and it's OK
    since they can always be compacted later on subsequent reads.
    Flushing the entire queue simply takes too long.  If there are, say,
    600k rows sitting in the queue, and the TSD is able to flush 2500/s,
    it would take 4 minutes to flush them all.  So instead we only flush
    the ones we were really supposed to flush anyway, that is the rows
    that are over 1h old.
    Change-Id: I4594b714ebf0d831c9fb8d314a38154439166ec0
    tsuna committed Sep 27, 2011
  17. @tsuna

    Limit the number of concurrent flushes that can happen.

    When the CompactionQueue contains, say, 1M rows, and flush() is called,
    the code was attempting to compact all these rows concurrently.  This
    is unrealistic and can either kill HBase and/or run the TSD out of RAM.
    With this change, a single call to flush() can only initiate up to 10k
    concurrent compactions, after which the code asynchronously waits until
    the batch of 10k is done before flushing more.
    Change-Id: I05ec3839e08f13455f80b3aba31acf5f42c7ac38
    tsuna committed Sep 26, 2011
  18. @tsuna

    Make the compaction thread more robust.

    A Throwable that isn't an Exception could kill the thread.  This is bad
    as without this thread the TSD will slowly but surely run out of memory.
    We can handle OOME by dropping the queue on the floor to free up memory.
    Other Throwables will just cause the thread to die after 1s, but a new
    one will be created to replace it.
    Change-Id: I2bdb65338da43a0f366cdaa6b2b2e7aeabcd503e
    tsuna committed Sep 19, 2011
  19. @tsuna

    Dynamically compute how many rows to flush per iteration.

    This ensures that we flush more aggressively when under pressure.
    Change-Id: Ie82f93b080e1841b560e6a34a288cda9e755f65e
    tsuna committed Sep 14, 2011
  20. @tsuna

    Properly handle throttling requests from asynchbase.

    Compactions rewrite so much data that they're very likely to cause
    region splits in HBase.  When they do, they keep sending tons of
    requests to the HBase client, which eventually makes them all fail
    with a PleaseThrottleException, due to the nature of its completely
    non-blocking asynchronous implementation.  When we get such this
    exception, we simply re-queue the row for compaction again, to try
    to compact it later.
    Change-Id: Ie7f5c4291c82ce9485243b7265763a399150f3d1
    tsuna committed Sep 1, 2011
  21. @tsuna

    Add 2 more stats on compaction activities.

    Change-Id: Ieb1961516fb96f11257f99d3c1d07efbd325a660
    tsuna committed Aug 11, 2011
  22. @tsuna

    Make the read path work in O(N) again.

    After the changes to `RowSeq' to support reading compacted rows, the
    code kept using naive `DataPointsIterator', which does random access
    into the data structure.  With variable length encoding, we can no
    longer locate the n-th value in O(1), so the code needed a O(N) walk
    to find each data point, leading to an quadratic loop when iterating
    on all the data points in a `RowSeq'.
    With this change, `RowSeq' uses its own custom iterator implementation
    instead of `DataPointsIterator' in order to work in O(N) [single pass].
    Change-Id: I98d00e1b30463542134e087243c6243e4c03ac3a
    tsuna committed Jun 27, 2011
  23. @tsuna

    Improved schema: compact the data in HBase.

    This is a major change.  New data points written by this code aren't
    readable by older versions of the code.  The new behavior is turned
    off by default, however.  After turning on the new behavior, instead
    of rolling back, you can turn it off again if you observe problems.
    In order to turn on the new compaction code, the JVM system property
    "tsd.feature.compactions" needs to be set.  You can set this by in
    your `tsdb.local' file like so:
    With this change the TSD remembers every row it's writing to in HBase.
    Periodically, it will read back old rows from HBase, and rewrite them
    in a more efficient format.  The compacted form consists mostly in
    concatenating together all the qualifiers and values inside that row.
    When the TSD reads a row that isn't compacted properly (or at all)
    it will fix it by overwriting the row again.  So the code is handling
    properly weird cases such as data points coming in with very old
    timestamps or out of order data etc.  It also handles properly races
    in case multiple concurrent TSDs attempt to compact the same row at
    the same time.
    In order for this change to be truly effective, it's important to have
    HBase compactions turned on, this way HBase can purge the individual
    data points and keep the compacted form only.  If you wish to leave
    HBase compactions turned off, consider triggering them periodically
    (either via a cron job, or manually).
    Note: `fsck' currently doesn't support the compacted rows, and will
    emit a warning when it sees one.  Another commit will add support for
    compacted rows in `fsck' in the future.
    Change-Id: Ibc8f0154f7dd79d5337fa7effb645a297f9f94b2
    tsuna committed Jun 26, 2011
  24. @tsuna

    Don't log "Connection reset by peer" errors.

    They just spam the logs unnecessarily.
    Change-Id: I6fb9081474c97142888e85b89e8fe130e0aebcac
    tsuna committed Aug 31, 2011
Something went wrong with that request. Please try again.