This repository was archived by the owner on May 12, 2021. It is now read-only.
[EAGLE-849] Refactor system metric collector scripts#763
Closed
haoch wants to merge 4 commits intoapache:masterfrom
Closed
[EAGLE-849] Refactor system metric collector scripts#763haoch wants to merge 4 commits intoapache:masterfrom
haoch wants to merge 4 commits intoapache:masterfrom
Conversation
jhsenjaliya
reviewed
Jan 4, 2017
|
|
||
| items = re.split("\s+", item.strip()) | ||
| demens = min(len(demension), len(items)) | ||
| tuple = dict() |
Contributor
There was a problem hiding this comment.
since tuple is data structure in python, should you name this something different ? like metric_info = dict() ?
jhsenjaliya
reviewed
Jan 4, 2017
| total_cpu_usage = 0 | ||
| cpu_stat_pre = None | ||
|
|
||
| data_dir = "/tmp/eagle_cpu_stat_previous" |
Contributor
There was a problem hiding this comment.
can we make this configurable ?
jhsenjaliya
suggested changes
Jan 4, 2017
| cpu_metric['value'] = str(round(per_cpu_usage * 100.0 / per_cpu_total, 2)) | ||
| self.collect(cpu_metric) | ||
|
|
||
| cup_stat_current = str(total_cpu_usage) + " " + str(total_cpu) |
Contributor
There was a problem hiding this comment.
do you meant cpu_stat_current ?
Member
Author
There was a problem hiding this comment.
In fact, it's current CPU usage:
Sum(Single CPU Usage) / Sum (Single CPU Total)
Member
Author
There was a problem hiding this comment.
It's used to compare with previous cpu usage to get system.cpu.totalusage:
Total CPU Usage (system.cpu.totalusage) = (total_cpu_usage - pre_total_cpu_usage) / (total_cpu - pre_total_cpu) %
Contributor
There was a problem hiding this comment.
sure, i was trying to point at spelling, should it be cpu_stat_current instead of cup_stat_current ?
jhsenjaliya
reviewed
Jan 4, 2017
| "irq"] + \ | ||
| tuple["softirq"] + tuple["steal"] + tuple["guest"] | ||
| total_cpu += per_cpu_total | ||
| total_cpu_usage += per_cpu_usage |
Contributor
There was a problem hiding this comment.
what is the difference between per_cpu_total and per_cpu_usage ?
Member
Author
There was a problem hiding this comment.
Data Source is:
[root@sandbox hadoop_jmx_collector]# cat /proc/stat
cpu 328118 17 134709 9869983 3745 3100 23742 0 0
cpu0 42170 1 17838 1224218 625 3058 3162 0 0
cpu1 41435 1 16803 1233688 403 4 2772 0 0
cpu2 42173 1 17059 1232358 582 6 2706 0 0
cpu3 40543 1 16920 1234035 471 6 2879 0 0
cpu4 41876 2 16936 1233251 456 6 2816 0 0
cpu5 41248 1 16654 1234433 551 5 2664 0 0
cpu6 41576 1 17046 1233823 335 5 2773 0 0
cpu7 37094 5 15448 1244174 318 5 3965 0 0
intr 24785791 145 7 0 0 0 0 0 0 0 0 0 0 112 0 473861 0 0 0 0 138240 350522 0 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ctxt 34107972
btime 1483582077
processes 154160
procs_running 2
procs_blocked 0
softirq 26040264 0 8770210 51530 909146 474103 0 2 1651574 154427 14029272
Metric Definition:
Single CPU Used (per_cpu_usage) = user + nice + system + wait + irq + softirq + steal + guest
Single CPU Total (per_cpu_total) = user + nice + system + idle + wait + irq + softirq + steal + guest
Single CPU Usage (%) = Single CPU Used (per_cpu_usage) / Single CPU Total (per_cpu_total)
Member
Author
|
@jhsenjaliya I have resolved most parts as commented, please help continue review. |
Member
Author
|
@anyway1021 please help review as well. |
Contributor
|
+1, LGTM. |
Member
Author
|
Thanks @jhsenjaliya and @anyway1021 for review, and now it's merged. |
jhsenjaliya
pushed a commit
to jhsenjaliya/eagle
that referenced
this pull request
Jul 12, 2017
Refactor System metric collector python script following similar framework as existing jmx metric collector. https://issues.apache.org/jira/browse/EAGLE-849 Author: Hao Chen <hao@apache.org> Closes apache#763 from haoch/SystemMetricCollector.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refactor System metric collector python script following similar framework as existing jmx metric collector.
https://issues.apache.org/jira/browse/EAGLE-849