New feature provided : new plugin for influxdb-java client #4846

dagmom · 2020-05-31T09:29:26Z

Please answer these questions before submitting pull request

Why submit this pull request?
New feature provided

add new java sdk plugin : InfluxDB

i develop a new plugin for influxdb-java client
i‘ve done the unit test and access influxdb can display on the topology and trace history

codecov · 2020-05-31T09:35:14Z

Codecov Report

Merging #4846 into master will decrease coverage by 0.25%.
The diff coverage is 33.33%.

@@             Coverage Diff              @@
##             master    #4846      +/-   ##
============================================
- Coverage     50.22%   49.97%   -0.26%     
+ Complexity     2751     2737      -14     
============================================
  Files           763      763              
  Lines         18872    18875       +3     
  Branches       1851     1851              
============================================
- Hits           9478     9432      -46     
- Misses         8635     8687      +52     
+ Partials        759      756       -3

Impacted Files	Coverage Δ	Complexity Δ
.../apache/skywalking/apm/agent/core/conf/Config.java	`47.05% <0.00%> (-1.14%)`	`0.00 <0.00> (ø)`
.../apm/network/trace/component/ComponentsDefine.java	`98.59% <100.00%> (+0.02%)`	`1.00 <0.00> (ø)`
.../core/logging/core/coverts/ThrowableConverter.java	`18.18% <0.00%> (-63.64%)`	`2.00% <0.00%> (-2.00%)`
...r/cluster/plugin/standalone/StandaloneManager.java	`80.00% <0.00%> (-20.00%)`	`3.00% <0.00%> (-1.00%)`
.../server/storage/plugin/jdbc/h2/dao/H2BatchDAO.java	`64.10% <0.00%> (-10.26%)`	`5.00% <0.00%> (ø%)`
...alking/apm/agent/core/commands/CommandService.java	`26.08% <0.00%> (-8.70%)`	`7.00% <0.00%> (-1.00%)`
...ache/skywalking/apm/agent/core/jvm/JVMService.java	`74.57% <0.00%> (-8.48%)`	`6.00% <0.00%> (-1.00%)`
...erver/library/server/grpc/CustomThreadFactory.java	`69.23% <0.00%> (-7.70%)`	`2.00% <0.00%> (-1.00%)`
...pm/agent/core/commands/CommandExecutorService.java	`46.15% <0.00%> (-7.70%)`	`4.00% <0.00%> (-1.00%)`
.../skywalking/apm/agent/core/remote/GRPCChannel.java	`80.64% <0.00%> (-6.46%)`	`5.00% <0.00%> (-1.00%)`
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8648f0d...03ac123. Read the comment docs.

kezhenxu94 · 2020-05-31T09:37:17Z

@dagmom please add the InfluxDB logo/icon to http://github.com/apache/skywalking-rocketbot-ui

dmsolr · 2020-05-31T09:52:41Z

Thanks! The plugin-test is required. :)

wu-sheng · 2020-05-31T12:39:01Z

@dagmom Read this doc, https://github.com/apache/skywalking/blob/master/docs/en/guides/Plugin-test.md, and you could find many test cases in our repo, https://github.com/apache/skywalking/tree/master/test/plugin/scenarios

dagmom · 2020-06-02T13:25:52Z

@dagmom Read this doc, https://github.com/apache/skywalking/blob/master/docs/en/guides/Plugin-test.md, and you could find many test cases in our repo, https://github.com/apache/skywalking/tree/master/test/plugin/scenarios

ok , I've read the doc.
I need to spend some time on it
and will finish the plugin-test as soon as possible

support-version.list modify form old version and healthCheck use ping() api

… ~ 2.17

wu-sheng · 2020-06-03T15:06:22Z

Hi, I will be on vacation for next three days. Feel free to review this PR, and other PRs. Just be carefully, we are closing the 8.0 release.
@dmsolr @kezhenxu94 @arugal

arugal · 2020-06-04T01:31:20Z

@dagmom rat check failed, please recheck.

…lugin scenarios

kezhenxu94 · 2020-06-06T05:50:33Z

@dagmom there is TODO in the new codes, when it's ready, please ping us

dagmom · 2020-06-06T06:48:16Z

@dagmom there is TODO in the new codes, when it's ready, please ping us

The first step ,I completed found influxDB as a component.
I left the TODO in order to complete the ‘db statement’ content later.
Can you merge the PR first, The second point will be completed later

kezhenxu94

Some review comments should be addressed before merging, and please rebase your branch on the latest master branch since your branch is out of date

kezhenxu94 · 2020-06-06T06:53:38Z

apm-sniffer/apm-sdk-plugin/influxdb-2.x-plugin/pom.xml

+  <artifactId>apm-influxdb-2.x-plugin</artifactId>
+  <description>This plugin is for use with InfluxDB 1.x.</description>


The artifactId and the description don't match, one says it's for 2.x while the other says it's for 1.x

apm-influxdb-2.x-plugin means influxdb client lib version
This plugin is for use with InfluxDB 1.x. means influxdb version , then i'll change to client version

kezhenxu94 · 2020-06-06T06:54:50Z

.../src/main/java/org/apache/skywalking/apm/plugin/influxdb/define/InfluxDBInstrumentation.java

+ * enhance InfluxDB InfluxDBFactory
+ * really impl class {@link org.influxdb.impl.InfluxDBImpl}
+ *
+ * @since  2020/05/22


Capitalize the first letter of the sentence please

What does since date mean? Quiet unusual, from my understanding. We coulf know when be added through git log directly

ok,I'll removesince

kezhenxu94 · 2020-06-06T06:55:55Z

.../src/main/java/org/apache/skywalking/apm/plugin/influxdb/define/InfluxDBInstrumentation.java

+
+    private static final String ENHANCE_CLASS = "org.influxdb.impl.InfluxDBImpl";
+    private static final String INTERCEPTOR_CLASS = "org.apache.skywalking.apm.plugin.influxdb.interceptor.InfluxDBConstructorInterceptor";
+  private static final String INFLUXDB_METHOD_INTERCEPT_CLASS = "org.apache.skywalking.apm.plugin.influxdb.interceptor.InfluxDBMethodInterceptor";


Suggested change

private static final String INFLUXDB_METHOD_INTERCEPT_CLASS = "org.apache.skywalking.apm.plugin.influxdb.interceptor.InfluxDBMethodInterceptor";

private static final String INFLUXDB_METHOD_INTERCEPT_CLASS = "org.apache.skywalking.apm.plugin.influxdb.interceptor.InfluxDBMethodInterceptor";

kezhenxu94 · 2020-06-06T06:56:55Z

...n/src/test/java/org/apache/skywalking/apm/plugin/influxdb/InfluxDBMethodInterceptorTest.java

+        assertThat(SpanHelper.getComponentId(span), is(ComponentsDefine.INFLUXDB_JAVA.getId()));
+        List<TagValuePair> tags = SpanHelper.getTags(span);
+        assertThat(tags.get(0).getValue(), is("InfluxDB"));
+//        assertThat(tags.get(1).getValue(), is("write ".concat(allArgument[0].toString())));


Remove if useless, or fix it if assertion is failed

kezhenxu94 · 2020-06-06T06:59:06Z

...rio/src/main/java/org/apache/skywalking/apm/testcase/influxdb/executor/InfluxDBExecutor.java

+ * InfluxDBExecutor
+ *
+ * @author guhao
+ * @since 2020/6/3


Remove author tag

kezhenxu94 · 2020-06-06T07:00:05Z

...src/test/java/org/apache/skywalking/apm/testcase/influxdb/executor/InfluxDBExecutorTest.java

+/**
+ * InfluxDBExecutorTest
+ *
+ * @author guhao
+ * @since 2020/6/3
+ */
+public class InfluxDBExecutorTest {
+
+//  @Test
+//  public void testPing(){
+//    InfluxDBExecutor executor = new InfluxDBExecutor("http://localhost:8086");
+//    Pong pong = executor.ping();
+//    System.out.println(pong.getVersion());
+//    Assert.assertNotNull(pong.getVersion());
+//  }
+
+}


Remove this if it's useless

kezhenxu94 · 2020-06-06T07:09:05Z

.../src/main/java/org/apache/skywalking/apm/plugin/influxdb/define/InfluxDBInstrumentation.java

+          new InstanceMethodsInterceptPoint() {
+            @Override
+            public ElementMatcher<MethodDescription> getMethodsMatcher() {
+              return InfluxDBMethodMatch.INSTANCE.getInfluxDBMethodMatcher();


This is good for encapsulation, but it's a pitfall that we check third-party classes in classes whose name end with *Instrumentation

skywalking/apm-checkstyle/checkStyle.xml

Lines 122 to 129 in f3d907b

<module name="ImportControl">

<property name="file" value="${import.control}"/>

<property name="path" value="apm-sniffer/(apm-sdk-plugin|bootstrap-plugins|optional-plugins)/.+/src/main/.+Instrumentation.java$"/>

</module>

<module name="ImportControl">

<property name="file" value="${import.control}"/>

<property name="path" value="apm-sniffer/apm-toolkit-activation/.+/src/main/.+Activation.java$"/>

to avoid issues like this #2871 , but this breaks the checks, although there is no third-party class in the InfluxDBMethodMatch for now, other reviewers should pay attention to this class in the future, FYI @wu-sheng

Question, why need this singleton. This method is not called in high frequently, and nothing cached there.

I did it the other way

wu-sheng · 2020-06-20T13:48:38Z

This is what I mean submodule unexpected update.

Please make sure the submodule commit id is as same as the master branch.

This reverts commit 8744795.

wu-sheng · 2020-06-20T14:31:47Z

Another Quasar plugin got merged first, so, please change the component id, #4951. And update the codes.

# Conflicts: # apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java # oap-server/server-bootstrap/src/main/resources/component-libraries.yml # oap-server/server-core/src/test/resources/component-libraries.yml

wu-sheng

Agent setup document, Table of Agent Configuration Properties should be updated as you could collect parameters. And this feature should be tested plugin test tool, I haven't seen the parameters in the tags for now.

dagmom · 2020-06-21T03:57:46Z

Agent setup document, Table of Agent Configuration Properties should be updated as you could collect parameters. And this feature should be tested plugin test tool, I haven't seen the parameters in the tags for now.

@wu-sheng As i known, influxdb-java is unlike jdbc api, it has no parameters.

Just use Query,Point and lineprotocol , none of these use parameters

wu-sheng · 2020-06-21T04:46:49Z

If just influxql, why need a config item?

dagmom · 2020-06-21T04:54:56Z

If just influxql, why need a config item?

@wu-sheng I looked at other plugins, like plugin.elasticsearch.trace_dsl.
Maybe just want to show component in some scenarios.

wu-sheng · 2020-06-21T05:57:16Z

OK. Your test case and CI fails somehow, please fix them.

…that in component-libraries.yml

dagmom · 2020-06-21T11:58:47Z

@dmsolr @wu-sheng
This time it seems to be VM crash, can you rerun CI ?

wu-sheng

Merged your other PR, but this PR still doesn't allow me to update automatically(you closed that option). Please update the branch.

wu-sheng · 2020-06-21T13:10:33Z

apm-sniffer/config/agent.config

@@ -78,3 +78,6 @@ logging.level=${SW_LOGGING_LEVEL:INFO}

 # mysql plugin configuration
 # plugin.mysql.trace_sql_parameters=${SW_MYSQL_TRACE_SQL_PARAMETERS:false}
+
+# influxdb plugin configuration
+# plugin.influxdb.trace_influxql=${SW_INFLUXDB_TRACE_INFLUXQL:true}


This is not a widely used config, don't need to add it into the default agent.config. Putting it in the document should be enough.

…xql" This reverts commit d6b32df

dagmom · 2020-06-21T13:42:30Z

Merged your other PR, but this PR still doesn't allow me to update automatically(you closed that option). Please update the branch.

@wu-sheng Where is this option? I can't find it, or invite to this fork repository

wu-sheng · 2020-06-21T13:45:52Z

I think at the PR page, there is a selector to provide upstream repo owner to edit your fork repo or branch. I can't see this at my side.

dagmom and others added 4 commits May 31, 2020 13:14

Bulk executed with failures，then log detail message

fafcf29

Merge branch 'master' into master

81d07c0

add new java sdk plugin : InfluxDB

dcc9a71

Merge branch 'master' into master

d28d344

kezhenxu94 added agent Language agent related. plugin Plugin for agent or collector. Be used to extend the capabilities of default implementor. labels May 31, 2020

wu-sheng requested review from wu-sheng, dmsolr and kezhenxu94 May 31, 2020 12:38

dagmom mentioned this pull request Jun 2, 2020

add influxdb-java plugin apache/skywalking-rocketbot-ui#303

Merged

dagmom and others added 4 commits June 3, 2020 21:33

Merge branch 'master' into master

d7a5d8a

plugin-test : influxdb-scenario

7121812

plugin-test : influxdb-scenario

2f99381

support-version.list modify form old version and healthCheck use ping() api

plugin-test : upgrade influxdb-scenario support-version.list form 2.5…

2e0c634

… ~ 2.17

wu-sheng requested a review from arugal June 3, 2020 15:04

dagmom and others added 2 commits June 3, 2020 23:12

plugin-test : add influxdb-scenario to Github Actions Job.

1296890

Merge branch 'master' into master

9caa661

dagmom added 2 commits June 5, 2020 00:33

raf check : exclude [jacoco] dir --> generated file from test agent p…

b151ad3

…lugin scenarios

influxdb plugin : resolve raf problem and unit test case

ca855b4

kezhenxu94 reviewed Jun 6, 2020

View reviewed changes

[PHP-E2E] Support v8 (apache#4862)

5818b7e

dagmom and others added 2 commits June 20, 2020 21:30

Merge branch 'master' into master

4d9e137

Mischange when merge remote-tracing branch

0838ef5

dagmom added 3 commits June 20, 2020 22:06

Revert "update submodule"

f77f8f3

This reverts commit 8744795.

make submodule commit id is as same as the master branch.

d4c4515

exclude scenario test gen files

15884f1

wu-sheng previously approved these changes Jun 20, 2020

View reviewed changes

dagmom added 2 commits June 21, 2020 10:50

change componentId

da55955

dagmom dismissed wu-sheng’s stale review via da55955 June 21, 2020 03:28

wu-sheng reviewed Jun 21, 2020

View reviewed changes

add agent/config/agent.config : plugin.influxdb.trace_influxql

d6b32df

Component id defined in class ComponentsDefine should be the same as …

43af626

…that in component-libraries.yml

wu-sheng reviewed Jun 21, 2020

View reviewed changes

dagmom and others added 2 commits June 21, 2020 21:38

Merge branch 'master' into master

d285424

Revert "add agent/config/agent.config : plugin.influxdb.trace_influ…

03ac123

…xql" This reverts commit d6b32df

wu-sheng approved these changes Jun 21, 2020

View reviewed changes

wu-sheng added this to the 8.1.0 milestone Jun 21, 2020

wu-sheng added the feature New feature label Jun 21, 2020

wu-sheng merged commit 08897ce into apache:master Jun 21, 2020

		<artifactId>apm-influxdb-2.x-plugin</artifactId>
		<description>This plugin is for use with InfluxDB 1.x.</description>

	private static final String INFLUXDB_METHOD_INTERCEPT_CLASS = "org.apache.skywalking.apm.plugin.influxdb.interceptor.InfluxDBMethodInterceptor";
	private static final String INFLUXDB_METHOD_INTERCEPT_CLASS = "org.apache.skywalking.apm.plugin.influxdb.interceptor.InfluxDBMethodInterceptor";

	<module name="ImportControl">
	<property name="file" value="${import.control}"/>
	<property name="path" value="apm-sniffer/(apm-sdk-plugin\|bootstrap-plugins\|optional-plugins)/.+/src/main/.+Instrumentation.java$"/>
	</module>

	<module name="ImportControl">
	<property name="file" value="${import.control}"/>
	<property name="path" value="apm-sniffer/apm-toolkit-activation/.+/src/main/.+Activation.java$"/>

New feature provided : new plugin for influxdb-java client #4846

New feature provided : new plugin for influxdb-java client #4846

Conversation

dagmom commented May 31, 2020

add new java sdk plugin : InfluxDB

codecov bot commented May 31, 2020 • edited

Codecov Report

kezhenxu94 commented May 31, 2020

dmsolr commented May 31, 2020

wu-sheng commented May 31, 2020

dagmom commented Jun 2, 2020

wu-sheng commented Jun 3, 2020

arugal commented Jun 4, 2020

kezhenxu94 commented Jun 6, 2020

dagmom commented Jun 6, 2020

kezhenxu94 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wu-sheng commented Jun 20, 2020

wu-sheng commented Jun 20, 2020

wu-sheng left a comment

Choose a reason for hiding this comment

dagmom commented Jun 21, 2020

wu-sheng commented Jun 21, 2020

dagmom commented Jun 21, 2020

wu-sheng commented Jun 21, 2020

dagmom commented Jun 21, 2020

wu-sheng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dagmom commented Jun 21, 2020

wu-sheng commented Jun 21, 2020

codecov bot commented May 31, 2020 •

edited