Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing a null character at the end of TCP payload #206

Closed
coulas opened this issue Aug 15, 2014 · 5 comments
Closed

Writing a null character at the end of TCP payload #206

coulas opened this issue Aug 15, 2014 · 5 comments
Assignees

Comments

@coulas
Copy link

coulas commented Aug 15, 2014

I need to send a null character at the end of a message on a tcp output...
But this null is considered to be an end of string.

My use case is sending a GELF message to a graylog's tcp input.
GELF message is a json syntax quite simple to comply with thanks to format-json.
Graylog needs a null character at the end of a GELF message in TCP but not in UDP.

I tried many combination in the template with \x0, echo \x0, global_definition_of_null, etc...
The matter is that null character when interpreted is an end of string, and is not written in the tcp message.
I see truncated messages with ngrep.

Gelf Specification is not clear about it, but I have tested it
After finding a clue in Graylog's issues

In syslog-ng's mailling list, we have seen three options with Balazs Scheidler and Gergely Nagy :

  1. the creation of $(format-gelf) which would solve both the NUL issue and makes it easier for someone else to create GELF output
  2. add an option to $(format-json) to add a suffix (and perhaps a prefix) in front of and after the resulting string
  3. $(char 0) as a template function? That can be reused outside
    of format-json

A format-gelf as in 1 is missing, but my case and other may be satisfied with a template function that writes raw characters as in solution 3

@algernon algernon added this to the syslog-ng 3.6.1 milestone Aug 15, 2014
@algernon
Copy link
Contributor

Instead of $(char 0), we should name it $(literal), and it would take a string, not just a char.

@algernon
Copy link
Contributor

In the long run, we'll have proper gelf support (which would be more than just $(format-gelf), since we want to support Chunked GELF too). Until then, I'll go with $(literal), which would be option 3, but a little bit more flexible.

@coulas
Copy link
Author

coulas commented Aug 18, 2014

I agree with $(litteral "x0"), it seems good to me.
Thank you. 👍

@algernon
Copy link
Contributor

Right. This is going to be harder than it first seemed, because even if we manage to append a NULL character to the string, and the GString's length is correct too, we end up using result->str somewhere down the line, and that does not handle embedded null-chars.

So $(char 0) is not implementable at the moment, neither is $(format-json --append-suffix) for the same reason.

To be able to implement this feature, we need to teach templates to handle embedded NULLs.

@juhaszviktor
Copy link
Contributor

As far as I saw, the problem is that the parser eliminates the NULL character. If the compiled template somehow contains the NULL character it will be contained in the formatted template's result.
So I think the $(literal "x0") or the $(char 0) should be work.

@juhaszviktor juhaszviktor self-assigned this Jun 22, 2015
juhaszviktor added a commit to juhaszviktor/syslog-ng that referenced this issue Jun 23, 2015
Custom byte sequences can be appended to the formatted template with this function
the parameter of this template function is a space separated byte sequence.
The bytes can be hexadecimal (0x..) octal (0...) and decimal formatted.
e.g.: $(literal "0xFE 0222 42").
NULL character can be also set.
Fixes syslog-ng#206

Signed-off-by: Juhász Viktor <viktor.juhasz@balabit.com>
@juhaszviktor juhaszviktor mentioned this issue Dec 14, 2015
bazsi pushed a commit that referenced this issue Sep 16, 2017
This new template function makes it possible to inject binary data
to the output of a template.

Parameters are a list of unsigned 8 bit numbers that are converted to
characters and produced as output.

The bytes can be hexadecimal (0x prefix) octal (0 prefix) and
decimal formatted (no prefix).

For example:

  $(binary 0xFE 0222 42).

Please note that binary characters, especially the NUL character (0x00)
may not be handled properly by the driver that uses the produced
output. LogWriter based destinations (such as file, tcp and other socket
socket based destinations) should be OK though.

Fixes #206

Signed-off-by: Juhász Viktor <viktor.juhasz@balabit.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
bazsi added a commit that referenced this issue Sep 18, 2017
This new template function makes it possible to inject binary data
to the output of a template.

Parameters are a list of unsigned 8 bit numbers that are converted to
characters and produced as output.

The bytes can be hexadecimal (0x prefix) octal (0 prefix) and
decimal formatted (no prefix).

For example:

  $(binary 0xFE 0222 42).

Please note that binary characters, especially the NUL character (0x00)
may not be handled properly by the driver that uses the produced
output. LogWriter based destinations (such as file, tcp and other socket
socket based destinations) should be OK though.

Fixes #206

Signed-off-by: Juhász Viktor <viktor.juhasz@balabit.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
@lbudai lbudai closed this as completed in c62b345 Sep 27, 2017
@lbudai lbudai removed the blocked label Sep 27, 2017
kovgeri01 pushed a commit to kovgeri01/syslog-ng that referenced this issue Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants