Doesn't respect quoted characters

[andsel]

Logstash information:

Please include the following information:

1. Logstash version (e.g. bin/logstash --version) 8.5.1 but also tested with 7.17.8
2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
4. How was the Logstash Plugin installed

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

1. JVM version (java -version)
2. JVM installation source (e.g. from the Operating System's package manager, from source, etc).
3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system): any

Description of the problem including expected versus actual behavior:
When an field of the event contains quoted characters, like \" the payload sent to the server should also contains the same quoting.

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

1. Run an http mock server

wget https://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-jre8-standalone/2.35.0/wiremock-jre8-standalone-2.35.0.jar

java -jar wiremock-jre8-standalone-2.35.0.jar --verbose 

2. run the following pipeline

input {
generator {
  count => 1
  message => '{"Field1": "22222wwwww\"www\"wwwwww"}'
  codec => "json"
}
}

filter {
http {
  target_body => "response_body"
  verb => "POST"
  url => "http://localhost:8080"
  body => "%{Field1}"
}
}

output {
 stdout{
   codec => rubydebug
 }
}

3. check the WireMock console
   It receives

22222wwwww"www"wwwwww

while we would have expected

22222wwwww\"www\"wwwwww

Provide logs (if relevant):

Apache http client wire logs

[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> POST / HTTP/1.1
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Connection: Keep-Alive
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> content-type: text/plain
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Content-Length: 21
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Host: localhost:8080
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> User-Agent: Manticore 0.9.1
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Accept-Encoding: gzip,deflate
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "POST / HTTP/1.1[\r][\n]"
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "content-type: text/plain[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Content-Length: 21[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Host: localhost:8080[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "User-Agent: Manticore 0.9.1[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "22222wwwww"www"wwwwww"
[2023-02-28T16:09:20,475][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "HTTP/1.1 404 Not Found[\r][\n]"
[2023-02-28T16:09:20,475][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "Content-Type: text/plain[\r][\n]"
[2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "Transfer-Encoding: chunked[\r][\n]"
[2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "[\r][\n]"
[2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "54[\r][\n]"
[2023-02-28T16:09:20,477][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << HTTP/1.1 404 Not Found

WireMock console

2023-02-28 16:09:20.472 Request received:
127.0.0.1 - POST /

Connection: [keep-alive]
Content-Type: [text/plain]
Content-Length: [21]
Host: [localhost:8080]
User-Agent: [Manticore 0.9.1]
Accept-Encoding: [gzip,deflate]
22222wwwww"www"wwwwww


Matched response definition:
(no response definition configured)

Response:
HTTP/1.1 404
(no headers)

[MikeKemmerer]

This is expected behavior. The slash you’re using is escaping the doublequote inside of the doubleqoute encapsulated string. Try using 3 \ characters (one more to escape the slash and another to escape the quote).

[andsel]

@MikeKemmerer I would agree with you if the generator used the the double quotes:

message => "{\"Field1\": \"22222wwwww\"www\"wwwwww\"}"

but however to have to be right it should have quoted also the quote character like in:

message => "{\"Field1\": \"22222wwwww\\\"www\\\"wwwwww\"}"

but is uses the single quote:

message => '{"Field1": "22222wwwww\"www\"wwwwww"}'

Replacing the HTTP filter with HTTP output

output {
  http {
    ecs_compatibility => "disabled"
    http_method => "post"
    url => "http://localhost:8080"
  }
}

Behaves like expected on WireMock

Connection: [keep-alive]
Content-Type: [application/json]
Content-Length: [206]
Host: [localhost:8080]
User-Agent: [Manticore 0.9.1]
Accept-Encoding: [gzip,deflate]
{"Field1":"22222wwwww\"www\"wwwwww","@version":"1","event":{"original":"{\"Field1\": \"22222wwwww\\\"www\\\"wwwwww\"}","sequence":0},"host":{"name":"kalimera"},"@timestamp":"2023-02-28T15:25:13.743720606Z"}

[andsel]

@MikeKemmerer you are right the json

{"Field1": "22222wwwww\"www\"wwwwww"}

is a valid json, on which the Field1 contains the string with a couple of double quotes, and those double quotes needs to be quoted to be represented in json: 22222wwwww"www"wwwwww