add-context bug with type hints

[ryanfaircloth]

syslog-ng

Version of syslog-ng

3.38.1 with beta features enabled

[2022-09-27T14:11:46.579838] add-contextual-data(): the failing line is; input='"tcp_13",name,"Daytime (RFC 867)"', filename='/etc/syslog-ng/core/lib/parsers/p_well_known_ports/p_well_known_ports.csv:14'

it seems use of () in the value field is problematic

[bazsi]

You are right. As a workaround try enclosing the value into string() that should satisfy the parser and force it to the right type.

I am not sure if this should become the final solution, as this is a slight incompatibility with older versions. I think I'll replace that error with a warning and a recommendation to add the explicit type cast to string.

I think that would work. Alternatively I could introduce a 4 column format but a very similar issue exists in db-parser() where I extended the syntax the same way and there's no separate column (although I could add the type information as an XML attribute in that case).

[ryanfaircloth]

Could we assume string for values that are quoted ?

…

On Wed, Sep 28, 2022 at 6:24 AM Balazs Scheidler ***@***.***> wrote: You are right. As a workaround try enclosing the value into string() that should satisfy the parser and force it to the right type. I am not sure if this should become the final solution, as this is a slight incompatibility with older versions. I think I'll replace that error with a warning and a recommendation to add the explicit type cast to string. I think that would work. Alternatively I could introduce a 4 column format but a very similar issue exists in db-parser() where I extended the syntax the same way and there's no separate column (although I could add the type information as an XML attribute in that case). — Reply to this email directly, view it on GitHub <#4155 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIN6WOGUA4M7P3XMXKTZ573WAQMGTANCNFSM6AAAAAAQW3PCGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bazsi]

Unfortunately that quotation is processed by the CSV parser. add-contextual-data() does not know if a column was escaped or not.

…

On Wed, Sep 28, 2022, 13:28 Ryan Faircloth ***@***.***> wrote: Could we assume string for values that are quoted ? On Wed, Sep 28, 2022 at 6:24 AM Balazs Scheidler ***@***.***> wrote: > You are right. As a workaround try enclosing the value into string() that > should satisfy the parser and force it to the right type. > > I am not sure if this should become the final solution, as this is a > slight incompatibility with older versions. I think I'll replace that error > with a warning and a recommendation to add the explicit type cast to string. > > I think that would work. Alternatively I could introduce a 4 column format > but a very similar issue exists in db-parser() where I extended the syntax > the same way and there's no separate column (although I could add the type > information as an XML attribute in that case). > > — > Reply to this email directly, view it on GitHub > < #4155 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AIN6WOGUA4M7P3XMXKTZ573WAQMGTANCNFSM6AAAAAAQW3PCGQ > > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#4155 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFOK5T3OUD3JGDVA7ATVBLWAQTWXANCNFSM6AAAAAAQW3PCGQ> . You are receiving this because you commented.Message ID: ***@***.***>

[ryanfaircloth]

The work around seems to work, just removing () from the string also works

[ryanfaircloth]

I think a minor update in docs to better explain how the column is parser would help future users of the feature but I don't think a change is needed once proper use is understood thanks for your help

[bazsi]

I ended up adding warnings that get emitted in case you are using an old config version and template expressions that would conflict with the new syntax. This is a sample log message:

WARNING: the value field in add-contextual-data() CSV files has been changed to support typing from syslog-ng 4.0. You are using an older config version and your CSV file contains an unrecognized type-cast, probably a parenthesis in the value field. This will be interpreted in the `type(value)' format in future versions. Please add an explicit string() cast as shown in the 'fixed-value' tag of this log message or remove the parenthesis; config-version='3.38', selector='bzorp/mail', name='pid', value='foobar(1224)', fixed-value='string(foobar(1224))'