With an <url> tag in patterndb.xml db-parser did not parse

[mitzkia]

After a recent patterndb changes syslog-ng can start with an old patterndb.xml which contains an tag, but db-parsing did not work.

Reproduction steps:
1, use the following patterndb.xml

<?xml version='1.0' encoding='UTF-8'?>
<patterndb version='4' pub_date='2015-10-08'>
  <ruleset name='test ruleset' id='480de478-d4a6-4a7f-bea4-0c0245d361e3'>
    <description>syslog-ng-testcase</description>
     <url>www.test.com</url>
      <pattern>matched_program</pattern>
        <rules>
          <rule id='09944c71-95eb-4bc0-8575-936931d85713' provider='syslog-ng' class='system'>
            <patterns>
              <pattern>ipv4 address: @IPv4:IPV4_MACRO@</pattern>
              </patterns>
              <values>
                <value name='value_filled_by_patterndb'>OK</value>
              </values>
          </rule>
        </rules>
  </ruleset>
</patterndb>

2, use the following syslog-ng.conf

@version: 3.7
source s_network_622bf2002224427f8f2ced7aa509cac4 {
       file("input");
};
parser p_db_parser_685467d8585a4c59a78f4a05a4d2dd58 {
    db_parser(file(patterndb.xml));
};
destination d_file_74efb8c47bfd41e89eda046f17da8f8f {
    file("output.txt" template("Parsed program: $PROGRAM , Parsed ipv4: $IPV4_MACRO
"));
};
log {
source(s_network_622bf2002224427f8f2ced7aa509cac4);
parser(p_db_parser_685467d8585a4c59a78f4a05a4d2dd58);
destination(d_file_74efb8c47bfd41e89eda046f17da8f8f);

flags(flow-control);
};

3, syslog-ng console log:

[2016-05-25T13:43:44.130495] Error parsing pattern database file; filename='patterndb.xml', error='patterndb.xml:5:12: Unexpected <url> tag, expected a <rules>, <patterns> or <pattern>'
[2016-05-25T13:43:44.130524] Error reloading pattern database, no automatic reload will be performed;
...
[2016-05-25T13:43:45.131297] Incoming log entry; line='<13>Feb 22 11:16:54 matched_program: ipv4 address: 10.30.255.254'
[2016-05-25T13:43:45.131494] Requesting flow control; location='s.conf:9:5'
[2016-05-25T13:43:45.131556] Message parsing complete; result='1', rule='p_db_parser_685467d8585a4c59a78f4a05a4d2dd58', location='s.conf:9:5'
[2016-05-25T13:43:45.131695] Initializing destination file writer; template='7.txt', filename='7.txt'
[2016-05-25T13:43:45.131897] Incoming log entry; line='<13>Feb 22 11:16:54 matched_program: ipv4 address: 10.30.255.254'
[2016-05-25T13:43:45.131964] Requesting flow control; location='s.conf:9:5'
[2016-05-25T13:43:45.131996] Message parsing complete; result='1', rule='p_db_parser_685467d8585a4c59a78f4a05a4d2dd58', location='s.conf:9:5'
[2016-05-25T13:43:45.132192] Outgoing message; message='Parsed program: matched_program , Parsed ipv4: \x0a'

[mitzkia]

syslog-ng should accept an old version (v4) xml with an "url" tag.

[bazsi]

Hi,

Can you pls post an example? I dont get it from the subject.

Thanks
On May 25, 2016 4:00 PM, "mitzkia" notifications@github.com wrote:

syslog-ng should accept an old version (v4) xml with an tag.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#1063 (comment)

[mitzkia]

Hi,

In my case syslog-ng can not parsed the $IPV4_MACRO from the incoming log.
Incoming log was:
<13>Feb 22 11:16:54 matched_program: ipv4 address: 10.30.255.254
Outgoing log was:
Parsed program: matched_program , Parsed ipv4: \x0a
Used pattern for the incoming log:
<pattern>ipv4 address: @IPv4:IPV4_MACRO@</pattern>

[mitzkia]

Belongs to the example:
My patterndb.xml also contains an "url" tag. On syslog-ng start we got the following log,:
Unexpected <url> tag, expected a <rules>, <patterns> or <pattern>, and the starting process continuous.

[mitzkia]

Sorry you mean an example for accepting the v4 patterndb.xml?
In this case it would be good if syslog-ng can accept a previous version of patterndb.xmls (which can contains an "old" xml tags) and work with them as a backward compatible mode.

[bazsi]

sorry, I have not seen your initial report, which does contain an example
patterndb which does not work. Of course syslog-ng should support old
versions and the intent was this, simply "url" tags were completely ignored
beforehand, so the new code does not parse it. But as we report errors now
when we encounter an invalid syntax, we might get issues. I'll get to the
bottom of this in a minute.

Bazsi

On Thu, May 26, 2016 at 7:01 AM, mitzkia notifications@github.com wrote:

Sorry you mean an example for accepting the v4 patterndb.xml?
In this case it would be good if syslog-ng can accept a previous version
of patterndb.xmls (which can contains an "old" xml tags) and work with them
as a backward compatible mode.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1063 (comment)

[mitzkia]

I have tested the fix. It is working correctly. I have used the configurations from the 1st comment.

syslog-ng console log

[2016-05-30T11:53:20.280104] Incoming log entry; line='<13>Feb 22 11:16:54 matched_program: ipv4 address: 10.30.255.254'
[2016-05-30T11:53:20.280188] Requesting flow control; location='syslog-ng.conf:6:5'
[2016-05-30T11:53:20.280229] patterndb rule matches; rule_id='09944c71-95eb-4bc0-8575-936931d85713'
[2016-05-30T11:53:20.280252] Advancing patterndb current time because of an incoming message; utc='1456139814'
[2016-05-30T11:53:20.280273] Message parsing complete; result='1', rule='p_db_parser_685467d8585a4c59a78f4a05a4d2dd58', location='syslog-ng.conf:6:5'
[2016-05-30T11:53:20.280333] Initializing destination file writer; template='output.txt', filename='output.txt'
[2016-05-30T11:53:20.280507] Outgoing message; message='Parsed program: matched_program , Parsed ipv4: 10.30.255.254\x0a'