Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tenant by time #68

Closed
ghost opened this issue Jan 28, 2016 · 14 comments
Closed

tenant by time #68

ghost opened this issue Jan 28, 2016 · 14 comments

Comments

@ghost
Copy link

ghost commented Jan 28, 2016

Hello @mwringe
Could you please provide me info on what tenant by time table in cassandra represent? Can this be used somehow to get the uptime of a project and then be used to compute % cpu usage at namespace level?

Regards,
Shilpa

@mwringe
Copy link
Contributor

mwringe commented Jan 28, 2016

@jsanda can you answer the question about what this table is suppose to represent?

@jsanda
Copy link
Contributor

jsanda commented Jan 28, 2016

The table is obsolete. It was intended to facilitate basic tenant queries, but a simpler solution was implemented. The table will be removed in a subsequent release.

@ghost
Copy link
Author

ghost commented Jan 28, 2016

Is there a way to get project uptime just like how one can get project level cpu usage and memory usage?

@mwringe
Copy link
Contributor

mwringe commented Jan 28, 2016

@shilpapadgaonkar How are you calculating project level cpu and memory usage?

@mwringe
Copy link
Contributor

mwringe commented Jan 28, 2016

@shilpapadgaonkar Each project does have a 'creationTimestamp' that tells you when the project was created [oc get -o json project $PROJECTNAME]. Not sure if that helps you or not.

@ghost
Copy link
Author

ghost commented Jan 28, 2016

To get memory usage from time A(start) to time B(end) for project test I would use

curl -H "Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
-H "Hawkular-tenant: test"
-X GET https://hawkular-metrics.example.com/hawkular/metrics/gauges/data?tags=descriptor_name:memory/usage&buckets=1&start='+ str(start) + '&end=' + str(end)

@mwringe
Copy link
Contributor

mwringe commented Jan 28, 2016

Note that without the 'stacked=true' query parameter in there, you are only getting the average usage and not the total usage.

Is there any reason why you couldn't do the same with the uptime metric? This should give the uptime for all the containers running during this time period.

@ghost
Copy link
Author

ghost commented Jan 28, 2016

I actually need average..

True.. sounds like a good try.. let me see if the cpu % with this test look ok..

@ghost
Copy link
Author

ghost commented Jan 29, 2016

I am trying to get some stacked usage and I get nan records..

First example is without using stacked which gives good result and the next one just adds parameter stacked=true and this gives me nan

@mwringe : Could you please take a look if the query looks ok?

curl -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxx" -H "Hawkular-tenant: atest" -X GET https://172.30.28.145/hawkular/metrics/gauges/data?"tags=descriptor_name:memory/usage&buckets=1&start=1454056894799&end=1454058701234" --insecure | python -m json.tool
[
{
"avg": 458849065.7622114,
"empty": false,
"end": 1454058701234,
"max": 636641280.0,
"median": 476952378.46956563,
"min": 105230336.0,
"percentile95th": 636641261.8292253,
"samples": 1085,
"start": 1454056894799
}
]

curl -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxx" -H "Hawkular-tenant: atest" -X GET https://172.30.28.145/hawkular/metrics/gauges/data?"tags=descriptor_name:memory/usage&buckets=1&start=1454056894799&end=1454058701234&stacked=true" --insecure | python -m json.tool
[
{
"avg": "NaN",
"empty": true,
"end": 1454058701234,
"max": "NaN",
"median": "NaN",
"min": "NaN",
"percentile95th": "NaN",
"samples": 23,
"start": 1454056894799
}
]

@mwringe
Copy link
Contributor

mwringe commented Jan 29, 2016

@stefannegrea any idea why stacked=true would return empty values here?

@stefannegrea
Copy link

The non-stacked code collects all the data points from all the matching metrics as originating from a single metric. Thus the bucket has 1085 samples, all looking from a single metric, hence the computed values.

The stacked version first buckets individual metrics. From the number of samples, I can infer there are 23 metrics that match the query. If one the metrics did not have data for the time request then resulting individual bucket for that metric will have a bunch of NaN. When the rest of the buckets are added up, they are still NaN (NaN + any number = NaN).

Note: 1085 is not divisible by 23, that means there are matching metrics without data in the selected interval.

@ghost
Copy link
Author

ghost commented Jan 29, 2016

@stefannegrea : If (NaN + any number = NaN), then the stacked search can only work when none of the individual buckets have a NaN. Is this intentional?

@mwringe
Copy link
Contributor

mwringe commented Jan 29, 2016

@shilpapadgaonkar is this running the OSE images or the origin-metric ones? We will need to know what versions need to be updated and patched.

The current behaviour with the stacked options is that if any metric returns an NaN value, then the resulting stacked metric will also return an NaN value.

The logic being that if we know the value of A but not the value of B, then the sum of A plus B would also unknown.

In your case, it looks like your query is returning some metrics which didn't collect any data in the time specified. This would be the expected behaviour when a particular container is not running, but it could also potentially be the case that, for whatever reason, metrics were not gathered during this time. Which is why its listed as empty and not as zero.

In some situations it may make sense to consider that if no metrics were gathers (eg the container is not running) then to consider it as zero instead of being empty. This would fix the NaN issue you are seeing when using stacked. We could add a flag to the query parameter to enable this behaviour

@mwringe
Copy link
Contributor

mwringe commented Jun 27, 2016

Housekeeping to close older issues. If you think this issue is not resolved, please reopen it.

@mwringe mwringe closed this as completed Jun 27, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants