[JAV-539] & [SCB-9] Implement CallCount and TPS Metrics #451

zhengyangyong · 2017-12-21T01:12:11Z

Follow this checklist to help us incorporate your contribution quickly and easily:

Make sure there is a JIRA issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
Each commit in the pull request should have a meaningful subject line and body.
Format the pull request title like [JAV-XXX] Fixes bug in ApproximateQuantiles, where you replace JAV-XXX with the appropriate JIRA issue.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Run mvn clean install to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

JAV-539 & SC-9 Implement CallCount and TPS Metrics

In core module we need add more build-in Metrics,include:
1.consumer total call count
2.provider total call count
3.consumer call tps
4.provider call tps

WillemJiang · 2017-12-21T03:49:34Z

metrics/metrics-core/src/main/java/io/servicecomb/metrics/core/metric/InstanceMetric.java

@@ -19,7 +19,8 @@

 public class InstanceMetric extends ModelMetric {
  public InstanceMetric(long waitInQueue, TimerMetric lifeTimeInQueue,
-      TimerMetric executionTime, TimerMetric consumerLatency, TimerMetric producerLatency) {


We should know if the invocation is consumer or producer to avoid storing other two metrics here.

Done,Separate into tow metric named "ConsumerInvocationMetric" and "ProducerInvocationMetric"

WillemJiang · 2017-12-21T03:50:54Z

metrics/metrics-sample/metrics-performance-test/src/main/resources/microservice.yaml

@@ -31,5 +32,5 @@ servicecomb:
  metrics:
    #polling setting
    polling:
-      #Time Unit is MILLISECONDS
+      #Support Muti Polling Time (MILLISECONDS) like 10000,60000 (10s and 60s)


Which polling time does metric use?

Both , support muti polling time ,use pollerIndex make a selection
RegistryMetric getRegistryMetric(int pollerIndex)

WillemJiang · 2017-12-21T22:29:58Z

metrics/metrics-core/src/main/java/io/servicecomb/metrics/core/monitor/InvocationMonitor.java

  private final TimerMonitor consumerLatency;

-  private final TimerMonitor producerLatency;
+  private final CallMonitor producerCall;


We still have the prducerXXX and consumerXXX here!

Because can't known invocation type before starting process ,all monitor temp keep in this class; when it convert to metric while polling ,will be separate into ConsumerMetric and ProducerMetric.
After Java Chassis improved invocation generation logic, will be refactor this code immediately

coveralls · 2017-12-22T03:18:23Z

Changes Unknown when pulling f6332fa on zhengyangyong:JAV-539&SCB-9 into ** on apache:master**.

zhengyangyong · 2017-12-23T01:59:20Z

https://lists.apache.org/thread.html/0b13c98f2066368fefe217040df4eecb0a86add08f31f7addfe0ee9e@%3Cdev.servicecomb.apache.org%3E

coveralls · 2017-12-23T02:16:07Z

Changes Unknown when pulling 7511c98 on zhengyangyong:JAV-539&SCB-9 into ** on apache:master**.

coveralls · 2017-12-23T02:46:04Z

Changes Unknown when pulling 5e76ae1 on zhengyangyong:JAV-539&SCB-9 into ** on apache:master**.

seanyinx · 2017-12-23T08:13:05Z

metrics/metrics-core/src/main/java/io/servicecomb/metrics/core/metric/RegistryMetric.java

-      consumerLatency = consumerLatency.merge(metric.getConsumerLatency());
-      producerLatency = producerLatency.merge(metric.getProducerLatency());
+      if (metric != null) {
+        if (metric instanceof ConsumerInvocationMetric) {


think of open closed principle

seanyinx · 2017-12-23T08:14:33Z

metrics/metrics-core/src/main/java/io/servicecomb/metrics/core/monitor/BasicMonitor.java

+
+  //Counting use System.nano get more precise time
+  //so we need change unit to millisecond when ouput
+  public double convertNanosecondToMillisecond(long nanoValue){


why double for milliseconds?

zhengyangyong · 2017-12-23T08:18:50Z

Rebase onto master and resolve conflicts

coveralls · 2017-12-23T08:33:21Z

Coverage decreased (-0.01%) to 87.137% when pulling 7722848 on zhengyangyong:JAV-539&SCB-9 into 7fb4030 on apache:master.

coveralls · 2017-12-23T10:21:11Z

Coverage decreased (-0.03%) to 87.121% when pulling dafe6fb on zhengyangyong:JAV-539&SCB-9 into 7fb4030 on apache:master.

coveralls · 2017-12-23T10:22:07Z

Coverage increased (+0.02%) to 87.169% when pulling dafe6fb on zhengyangyong:JAV-539&SCB-9 into 7fb4030 on apache:master.

wujimin · 2017-12-24T02:50:32Z

.../src/main/java/io/servicecomb/metrics/core/event/InvocationStartProcessingEventListener.java

@@ -41,9 +41,14 @@ public InvocationStartProcessingEventListener(RegistryMonitor registryMonitor) {
  public void process(Event data) {
    InvocationStartProcessingEvent event = (InvocationStartProcessingEvent) data;
    InvocationMonitor monitor = registryMonitor.getInvocationMonitor(event.getOperationName());
+    //TODO:current java chassis unable know invocation type before starting process,so all type WaitInQueue increment(-1) (decrement)


why?
event.getInvocationType is not you want?

and in fact, we only trigger InvocationStartProcessingEvent in producer invocation
missed it in consumer invocation.

Great,it's a important info for me ,thank a lot

Done,Consumer Side Event will be trigger in VertxHttpClient and HighwayClient,https://issues.apache.org/jira/browse/SCB-126

coveralls · 2017-12-25T08:12:26Z

Coverage increased (+0.004%) to 87.153% when pulling 2269965 on zhengyangyong:JAV-539&SCB-9 into 7fb4030 on apache:master.

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

…ion metric SCB-91 All Monitor can convert to Map with plain-key when polling data Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

…nymore) Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

…cond Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

…temMonitor" Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

zhengyangyong · 2017-12-26T00:54:51Z

resolve conflict

coveralls · 2017-12-26T01:32:21Z

Coverage decreased (-0.01%) to 87.106% when pulling 2a0f1b8 on zhengyangyong:JAV-539&SCB-9 into a3e96ec on apache:master.

zhengyangyong changed the title ~~JAV-539 & SCB-9~~ JAV-539 & SCB-9 Implement CallCount and TPS Metrics Dec 21, 2017

zhengyangyong changed the title ~~JAV-539 & SCB-9 Implement CallCount and TPS Metrics~~ [JAV-539] & [SCB-9] Implement CallCount and TPS Metrics Dec 21, 2017

WillemJiang reviewed Dec 21, 2017

View reviewed changes

zhengyangyong mentioned this pull request Dec 22, 2017

[JAV-540] & [SCB-85] Upgrade Metrics File Output from 0.5.0 to 0.6.0 #457

Closed

6 tasks

liubao68 approved these changes Dec 22, 2017

View reviewed changes

zhengyangyong mentioned this pull request Dec 23, 2017

[JAV-508] & [SCB-11] Metrics Auto Publish #453

Closed

6 tasks

zhengyangyong force-pushed the JAV-539&SCB-9 branch from 5e76ae1 to 7722848 Compare December 23, 2017 08:14

seanyinx reviewed Dec 23, 2017

View reviewed changes

wujimin reviewed Dec 24, 2017

View reviewed changes

seanyinx approved these changes Dec 25, 2017

View reviewed changes

zhengyangyong added 9 commits December 26, 2017 08:48

JAV-539 & SC-9 Implement CallCount and TPS Metrics

36057c2

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 update asf license

af1a884

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 fix pr comment separate Consumer and Producer invocat…

cc59ed0

…ion metric SCB-91 All Monitor can convert to Map with plain-key when polling data Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 add InvocationMonitorType comment

4bdb5c8

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 direct return servo monitor collect data (not cache a…

89abf6b

…nymore) Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 change time-related output from nanosecond to millise…

b1a4e48

…cond Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 add code comment for windowtime

7a1cddb

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 resolve conflicts and change "SystemResource" -> "Sys…

15905af

…temMonitor" Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 fix pr comment

17a2931

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

zhengyangyong added 3 commits December 26, 2017 08:48

JAV-539 & SCB-9 fix pr comment

de0ff1b

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 & SCB-9 minor refactor

350b2db

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

JAV-539 SCB-9 resolve conflict

2a0f1b8

Signed-off-by: zhengyangyong <yangyong.zheng@huawei.com>

zhengyangyong force-pushed the JAV-539&SCB-9 branch from 2269965 to 2a0f1b8 Compare December 26, 2017 00:54

wujimin approved these changes Dec 26, 2017

View reviewed changes

WillemJiang merged commit fca6aeb into apache:master Dec 27, 2017

zhengyangyong deleted the JAV-539&SCB-9 branch March 27, 2018 06:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JAV-539] & [SCB-9] Implement CallCount and TPS Metrics #451

[JAV-539] & [SCB-9] Implement CallCount and TPS Metrics #451

zhengyangyong commented Dec 21, 2017 •

edited

WillemJiang Dec 21, 2017

zhengyangyong Dec 21, 2017

WillemJiang Dec 21, 2017

zhengyangyong Dec 21, 2017

WillemJiang Dec 21, 2017

zhengyangyong Dec 22, 2017

coveralls commented Dec 22, 2017

zhengyangyong commented Dec 23, 2017

coveralls commented Dec 23, 2017

coveralls commented Dec 23, 2017

seanyinx Dec 23, 2017

seanyinx Dec 23, 2017

zhengyangyong commented Dec 23, 2017

coveralls commented Dec 23, 2017

coveralls commented Dec 23, 2017

coveralls commented Dec 23, 2017

wujimin Dec 24, 2017

wujimin Dec 24, 2017

zhengyangyong Dec 25, 2017

zhengyangyong Dec 25, 2017

coveralls commented Dec 25, 2017

zhengyangyong commented Dec 26, 2017

coveralls commented Dec 26, 2017

[JAV-539] & [SCB-9] Implement CallCount and TPS Metrics #451

[JAV-539] & [SCB-9] Implement CallCount and TPS Metrics #451

Conversation

zhengyangyong commented Dec 21, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Dec 22, 2017

zhengyangyong commented Dec 23, 2017

coveralls commented Dec 23, 2017

coveralls commented Dec 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhengyangyong commented Dec 23, 2017

coveralls commented Dec 23, 2017

coveralls commented Dec 23, 2017

coveralls commented Dec 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Dec 25, 2017

zhengyangyong commented Dec 26, 2017

coveralls commented Dec 26, 2017

zhengyangyong commented Dec 21, 2017 •

edited