Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: HTTP OUTPUT - Add configuration option to not group messages into JSON array #680

Closed
chicken231 opened this issue Jul 12, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@chicken231
Copy link

commented Jul 12, 2018

Is your feature request related to a problem? Please describe.
Not a problem, more of a limitation.
I'm trying to use the HTTP Output plugin to POST log events formatted as JSON to an endpoint that can't handle a JSON array.

This works:

$ echo "{\"iamkey\":\"iamvalue\"},{\"who\":\"you\"}" | curl -X POST "http://example.com" -v --data-binary @-
...
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
...

This doesn't:

$ echo "[{\"iamkey\":\"iamvalue\"},{\"who\":\"you\"}]" | curl -X POST "http://example.com" -v --data-binary @-
...
< HTTP/1.1 400 Bad Request
< Content-Type: application/json; charset=UTF-8
< 
* Connection #0 to host example.com left intact
{"malformedLines":1,"oversizedLines":0,"successfulLines":0}

Describe the solution you'd like
Add a configuration option to not group log events into an array, and instead, every HTTP POST includes just one log event.

Describe alternatives you've considered
The logging SaaS does not have plans to accept a JSON array.

Obviously, grouping these into an array is more efficient, but I don't have control over the behavior of the SaaS.

@chicken231 chicken231 changed the title HTTP OUTPUT - Add configuration option to not group messages into JSON array Feature Request: HTTP OUTPUT - Add configuration option to not group messages into JSON array Jul 12, 2018

@michiel

This comment has been minimized.

Copy link
Contributor

commented Jul 13, 2018

@aa23 For the HTTP output, have you tried setting format json_stream? This outputs Concatenated JSON, not a JSON Array and it might be a format your SaaS accepts.

See #477

@chicken231

This comment has been minimized.

Copy link
Author

commented Jul 13, 2018

Thank you @michiel , using the Format json_stream resolved the issue for me.
I saw that in the docs and tried it before, but I must have had something else mis-configured at the time.

For those curious, this should be how the message body is structured with the json_stream option:

$ echo "{\"iamkey\":\"iamvalue\"},{\"who\":\"you\"},{\"k\":\"v\"},{\"what\":\"yeah\"}" | curl -X POST "http://example.com" -v --data-binary @-
...
< HTTP/1.1 200 OK
...

Closing issue - my fault.

@chicken231 chicken231 closed this Jul 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.