rawDatapoints are no longer reported in query summary #784

Closed
bobrik opened this Issue Apr 24, 2016 · 4 comments

Projects

None yet

2 participants

@bobrik
bobrik commented Apr 24, 2016

Before (v2.2.0RC3):

  {
    "statsSummary": {
      "datapoints": 192,
      "rawDatapoints": 6176,
      "aggregationTime": 3,
      "serializationTime": 5,
      "storageTime": 5719,
      "timeTotal": 5697
    }
  }

Currently (v2.2.0):

  {
    "statsSummary": {
      "avgAggregationTime": 1.088369,
      "avgHBaseTime": 295.162437,
      "avgQueryScanTime": 5904.48002,
      "avgScannerTime": 295.223705,
      "avgScannerUidToStringTime": 0,
      "avgSerializationTime": 1.106604,
      "emittedDPs": 182,
      "maxAggregationTime": 1.088369,
      "maxHBaseTime": 5903.248743,
      "maxQueryScanTime": 5904.48002,
      "maxScannerUidtoStringTime": 0,
      "maxSerializationTime": 1.106604,
      "maxUidToStringTime": 0.048007,
      "processingPreWriteTime": 5905.98901,
      "queryIdx_00": {
        "aggregationTime": 1.088369,
        "avgHBaseTime": 5903.248743,
        "avgScannerTime": 5904.474104,
        "avgScannerUidToStringTime": 0,
        "emittedDPs": 182,
        "groupByTime": 0,
        "maxHBaseTime": 5903.248743,
        "maxScannerUidtoStringTime": 0,
        "queryIndex": 0,
        "queryScanTime": 5904.48002,
        "scannerStats": {
          "scannerIdx_00": {
            "compactionTime": 0.661153,
            "hbaseTime": 5903.248743,
            "scannerId": null,
            "scannerTime": 5904.474104,
            "scannerUidToStringTime": 0,
            "successfulScan": 1,
            "uidPairsResolved": 0
          }
        },
        "serializationTime": 1.106604,
        "successfulScan": 1,
        "uidPairsResolved": 0,
        "uidToStringTime": 0.048007
      },
      "successfulScan": 1,
      "uidPairsResolved": 0
    }
  }
@bobrik
bobrik commented Apr 24, 2016

Having the number of distinct timeseries would be nice too.

@bobrik
bobrik commented Apr 24, 2016

Probably deserves another issue, but overall avg doesn't look right:

  • "avgHBaseTime": 295.162437
  • "maxHBaseTime": 5903.248743
  • "queryIdx_00"
    • "avgHBaseTime": 5903.248743
    • "maxHBaseTime": 5903.248743
@manolama manolama added the bug label May 1, 2016
@manolama
Member
manolama commented May 1, 2016

I'll restore the raw data points here. It's an estimate now as it's pretty expensive to calculate (as I had removed it before because it doubled query time). I'll also look at the avg, that does seem fishy.

@manolama manolama added a commit to manolama/opentsdb that referenced this issue May 1, 2016
@manolama manolama Fix issue #784 by adding an estimate for the number of data points
from storage. Also fix the average calculation when salting is
not enabled.
050d507
@manolama manolama added a commit that referenced this issue May 1, 2016
@manolama manolama Fix issue #784 by adding an estimate for the number of data points
from storage. Also fix the average calculation when salting is
not enabled.

Signed-off-by: Chris Larsen <clarsen@yahoo-inc.com>
4b14b2d
@manolama manolama added a commit that referenced this issue May 1, 2016
@manolama manolama Fix issue #784 by adding an estimate for the number of data points
from storage. Also fix the average calculation when salting is
not enabled.

Signed-off-by: Chris Larsen <clarsen@yahoo-inc.com>
d667a92
@manolama
Member
manolama commented May 1, 2016

Merged in 4b14b2d. Give it a shot and let us know how it looks.

@manolama manolama closed this May 1, 2016
@johann8384 johann8384 added a commit to johann8384/opentsdb that referenced this issue Dec 5, 2016
@manolama @johann8384 manolama + johann8384 Fix issue #784 by adding an estimate for the number of data points
from storage. Also fix the average calculation when salting is
not enabled.

Signed-off-by: Chris Larsen <clarsen@yahoo-inc.com>
3d2cd0c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment