Add escaping for new lines in AbstractInfluxdbMetricsSender by dgetzlaf · Pull Request #645 · apache/jmeter

dgetzlaf · 2021-02-25T10:41:53Z

Description

Add escaping for new lines in AbstractInfluxdbMetricsSender.tagToStringValue()

Motivation and Context

For load- and performance testing (distributed testing) we send metrics to the InfluxDB.
From time to time, no metrics occur in the dashboard while testing:

In the log file of the JMeter controller, I find this line pretty often:

ERROR o.a.j.v.b.i.HttpMetricsSender: Error writing metrics to influxDB Url: https://<URL>, responseCode: 400, responseBody: {"code":"invalid","message":"unable to parse 'jmeter,application=<application name>,transaction=<transaction name>,responseCode=500,responseMessage=javax.script.ScriptException:\\ org.openqa.selenium.ElementClickInterceptedException:\\ element\\ click\\ intercepted:\\ Element\\ \u003cbutton\\ type\\=\"submit\"\\ class\\=\"btn\\ btn-primary\\ btn-next\"\u003e...\u003c/button\u003e\\ is\\ not\\ clickable\\ at\\ point\\ (313\\,\\ 620).\\ Other\\ element\\ would\\ receive\\ the\\ click:\\ \u003cdiv\\ class\\=\"dhl\\ redesign\\ frankieren-loading-overlay\"\u003e...\u003c/div\u003e': missing fields\nunable to parse '\\ \\ (Session\\ info:\\ headless\\ chrome\\=88.0.4324.150)': missing fields\nunable to parse 'Build\\ info:\\ version:\\ '3.14.0'\\,\\ revision:\\ 'aacccce0'\\,\\ time:\\ '2018-08-02T20:19:58.91Z'': missing fields\nunable to parse 'System\\ info:\\ host:\\ 'ip-172-31-45-153.internal'\\,\\ ip:\\ '172.31.45.153'\\,\\ os.name:\\ 'Linux'\\,\\ os.arch:\\ 'amd64'\\,\\ os.version:\\ '4.14.214-160.339.amzn2.x86_64'\\,\\ java.version:\\ '11.0.7'': missing fields\nunable to parse 'Driver\\ info:\\ org.openqa.selenium.chrome.ChromeDriver': missing fields\nunable to parse 'Capabilities\\ {acceptInsecureCerts:\\ false\\,\\ browserName:\\ chrome\\,\\ browserVersion:\\ 88.0.4324.150\\,\\ chrome:\\ {chromedriverVersion:\\ 88.0.4324.96\\ (68dba2d8a0b14...\\,\\ userDataDir:\\ /tmp/.com.google.Chrome.L3v1yY}\\,\\ goog:chromeOptions:\\ {debuggerAddress:\\ localhost:46619}\\,\\ javascriptEnabled:\\ true\\,\\ networkConnectionEnabled:\\ false\\,\\ pageLoadStrategy:\\ normal\\,\\ platform:\\ LINUX\\,\\ platformName:\\ LINUX\\,\\ proxy:\\ Proxy(manual\\,\\ http\\=localhos...\\,\\ setWindowRect:\\ true\\,\\ strictFileInteractability:\\ false\\,\\ timeouts:\\ {implicit:\\ 0\\,\\ pageLoad:\\ 300000\\,\\ script:\\ 30000}\\,\\ unhandledPromptBehavior:\\ dismiss\\ and\\ notify\\,\\ webauthn:extension:largeBlob:\\ true\\,\\ webauthn:virtualAuthenticators:\\ true}': missing fields"}

It seems that the escaping of the line breaks is not working properly. By that, invalid requests are sent to the InfluxDB, resulting in 'missing fields'.

How Has This Been Tested?

Configure InfluxDB connection
Add a new JSR223 with the body

throw new Exception('''Line 0
Line 1
Line 2
End
''')

Current JMeter 5.4.1 Release:
3. Looking at the console output / jmeter log file --> The error occurs
4. influxDB : No metrics received

JMeter 5.4.1 Release + this fix:
3. Looking at the console output / jmeter log file --> No error occurs
4. influxDB : metrics are received

Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist:

My code follows the code style of this project.
I have updated the documentation accordingly.

…ngValue()

codecov-io · 2021-02-25T11:06:40Z

Codecov Report

Merging #645 (429d67f) into master (b89c454) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master     #645      +/-   ##
============================================
+ Coverage     55.43%   55.46%   +0.02%     
- Complexity    10157    10180      +23     
============================================
  Files          1044     1046       +2     
  Lines         64082    64151      +69     
  Branches       7250     7265      +15     
============================================
+ Hits          35527    35579      +52     
- Misses        26048    26069      +21     
+ Partials       2507     2503       -4

Impacted Files	Coverage Δ	Complexity Δ
...ackend/influxdb/AbstractInfluxdbMetricsSender.java	`100.00% <100.00%> (ø)`	`3.00 <1.00> (ø)`
...ter/protocol/http/proxy/SamplerCreatorFactory.java	`27.90% <0.00%> (-13.96%)`	`4.00% <0.00%> (-1.00%)`
.../apache/jmeter/protocol/http/util/HTTPFileArg.java	`87.93% <0.00%> (-2.55%)`	`25.00% <0.00%> (+6.00%)`	⬇️
...n/java/org/apache/jmeter/sampler/DebugSampler.java	`77.27% <0.00%> (-1.46%)`	`13.00% <0.00%> (-1.00%)`
...rg/apache/jmeter/extractor/DebugPostProcessor.java	`80.00% <0.00%> (-0.96%)`	`14.00% <0.00%> (-1.00%)`
...n/java/org/apache/jmeter/reporters/Summariser.java	`85.49% <0.00%> (-0.77%)`	`18.00% <0.00%> (-1.00%)`
...ter/protocol/http/proxy/DefaultSamplerCreator.java	`54.12% <0.00%> (-0.52%)`	`28.00% <0.00%> (-1.00%)`
...in/java/org/apache/jmeter/save/CSVSaveService.java	`44.53% <0.00%> (-0.05%)`	`61.00% <0.00%> (+1.00%)`	⬇️
...pache/jmeter/protocol/http/sampler/PostWriter.java	`86.70% <0.00%> (ø)`	`31.00% <0.00%> (-1.00%)`
...ache/jmeter/protocol/http/sampler/HTTPHC4Impl.java	`60.46% <0.00%> (ø)`	`89.00% <0.00%> (-1.00%)`
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b89c454...429d67f. Read the comment docs.

ham1 · 2021-02-25T18:47:40Z

.../main/java/org/apache/jmeter/visualizers/backend/influxdb/AbstractInfluxdbMetricsSender.java

+                .replaceAll(" ", "\\\\ ")
+                .replaceAll(",", "\\\\,")
+                .replaceAll("=", "\\\\=")
+                .replaceAll("\n", "\\\\n");


This looks ok to me, but while we are here, I believe we should use replace as replaceAll is only needed when using a regex as the pattern to find (it should also perform better).

@ham1 thanks for your feedback. I've updated this code block according to your recommendation.

…luxdbMetricsSender

FSchumacher · 2021-02-27T16:06:07Z

Thanks for the PR. Could you test next nightly or build from trunk and report back, if it solves your problem?

Contributed by David Getzlaff (david.getzlaff at t-systems.com>) Closes apache#645 on github

dgetzlaf · 2021-05-28T11:24:45Z

Hi @FSchumacher,
I'm sorry for the late answer. I've just verified this fix with the SNAPSHOT from https://ci-builds.apache.org/job/JMeter/job/JMeter-trunk/266/

coeyc · 2021-12-06T11:16:04Z

has this fix made its way into a downloadable build of JMeter?

dgetzlaf · 2021-12-06T11:22:20Z

@coeyc this fix was approved and merged. Sadly JMeter wasn't released yet 😿
Maybe you can use a Nightly build instead https://ci-builds.apache.org/job/JMeter/job/JMeter-trunk/

vlsi · 2021-12-06T11:24:31Z

AFAIK it should be included in v5.4.1+. Am I mistaken?

vlsi · 2021-12-06T11:26:27Z

Oh, there were no releases since the fix has been merged. That is quite some time now.

coeyc · 2021-12-06T11:26:38Z

so, i guess the next build, we will look at the nightly build

vlsi · 2021-12-06T11:31:28Z

Just in case, the nightly would include Open Model Thread Group, and I would be glad to hear your feedback :)

dgetzlaf · 2021-12-06T16:10:22Z

@vlsi do you know someone, who could initiate a JMeter 5.5 release?

Your Open Model Thread Group looks good to me; sadly I had no chance in testing it yet.

vlsi · 2021-12-06T19:39:58Z

A mail to dev@jmeter.apache.org would do

Add escaping for new lines in AbstractInfluxdbMetricsSender.tagToStri…

429d67f

…ngValue()

ham1 reviewed Feb 25, 2021

View reviewed changes

Use string replacement instead of regex replacement in in AbstractInf…

6fb8ecf

…luxdbMetricsSender

asfgit closed this in 2c2f611 Feb 27, 2021

kkalinin pushed a commit to kkalinin/jmeter that referenced this pull request Mar 11, 2021

Add escaping for new lines in AbstractInfluxdbMetricsSender

7e495f4

Contributed by David Getzlaff (david.getzlaff at t-systems.com>) Closes apache#645 on github

dgetzlaf deleted the fix-linebreaks-in-influxdb branch March 16, 2023 11:46

Conversation

dgetzlaf commented Feb 25, 2021

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

codecov-io commented Feb 25, 2021

Codecov Report

Uh oh!

ham1 Feb 25, 2021

Choose a reason for hiding this comment

Uh oh!

dgetzlaf Feb 26, 2021

Choose a reason for hiding this comment

Uh oh!

FSchumacher commented Feb 27, 2021

Uh oh!

dgetzlaf commented May 28, 2021

Uh oh!

coeyc commented Dec 6, 2021

Uh oh!

dgetzlaf commented Dec 6, 2021

Uh oh!

vlsi commented Dec 6, 2021

Uh oh!

vlsi commented Dec 6, 2021

Uh oh!

coeyc commented Dec 6, 2021

Uh oh!

vlsi commented Dec 6, 2021

Uh oh!

dgetzlaf commented Dec 6, 2021

Uh oh!

vlsi commented Dec 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants