Afsql threaded dest driver

[bazsi]

This branch converts our afsql driver to use the LogThreadedDestDriver framework, instead of its home-grewn implementation (that has originally been the basis of LogThreadedDestDriver).

This should go in after #2315 because without that some of our stats counters would be missing.

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[bazsi]

@kira-syslogng retest this please;

[kira-syslogng]

Build FAILURE

[bazsi]

@kira-syslogng retest this please

[kira-syslogng]

Build SUCCESS

[Ruha0309i]

👍👍👍👍

[furiel]

@bazsi I checked with valgrind, and I found an invalid read:

Config:

@version: 3.18

options {
  time-reopen(1);
  stats-level(2);
};

destination d_sql {
    sql(type(sqlite3)
    database("/run/user/1000/testdb.db")
    table("test")
    columns("datetime", "host", "program", "pid", "message")
    values("{$R_DATE}", "${HOST}", "${PROGRAM}", "${PID}", "${MSGONLY}")
    indexes("datetime", "host", "program", "pid", "message")
    flush-lines(2)
    flags(explicit-commits)
    );
};

log {
    source { stdin(flags(no-parse)); };
    destination(d_sql);
    flags(flow-control);
};

The invalid read happens right after syslog-ng starts, without messages.

==8475== Invalid read of size 1
==8475==    at 0x94BC3FB: ??? (in /usr/lib/x86_64-linux-gnu/dbd/libdbdsqlite3.so)
==8475==    by 0x799A85E: dbi_conn_connect (in /usr/lib/x86_64-linux-gnu/libdbi.so.1.1.0)
==8475==    by 0x7789B85: afsql_dd_connect (afsql.c:691)
==8475==    by 0x4E873D1: _compat_connect (logthrdestdrv.c:744)
==8475==    by 0x4E85A79: log_threaded_dest_worker_connect (logthrdestdrv.h:153)
==8475==    by 0x4E85FE9: _connect (logthrdestdrv.c:177)
==8475==    by 0x4E86967: _perform_work (logthrdestdrv.c:440)
==8475==    by 0x603C056: ??? (in /usr/lib/x86_64-linux-gnu/libivykis.so.0.5.4)
==8475==    by 0x603F38D: iv_main (in /usr/lib/x86_64-linux-gnu/libivykis.so.0.5.4)
==8475==    by 0x4E87051: _worker_thread (logthrdestdrv.c:640)
==8475==    by 0x4E8E1FE: _worker_thread_func (mainloop-worker.c:339)
==8475==    by 0x51A5C54: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4800.2)
==8475==  Address 0x7339e4f is 1 bytes before a block of size 26 alloc'd
==8475==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8475==    by 0x94BC394: ??? (in /usr/lib/x86_64-linux-gnu/dbd/libdbdsqlite3.so)
==8475==    by 0x799A85E: dbi_conn_connect (in /usr/lib/x86_64-linux-gnu/libdbi.so.1.1.0)
==8475==    by 0x7789B85: afsql_dd_connect (afsql.c:691)
==8475==    by 0x4E873D1: _compat_connect (logthrdestdrv.c:744)
==8475==    by 0x4E85A79: log_threaded_dest_worker_connect (logthrdestdrv.h:153)
==8475==    by 0x4E85FE9: _connect (logthrdestdrv.c:177)
==8475==    by 0x4E86967: _perform_work (logthrdestdrv.c:440)
==8475==    by 0x603C056: ??? (in /usr/lib/x86_64-linux-gnu/libivykis.so.0.5.4)
==8475==    by 0x603F38D: iv_main (in /usr/lib/x86_64-linux-gnu/libivykis.so.0.5.4)
==8475==    by 0x4E87051: _worker_thread (logthrdestdrv.c:640)
==8475==    by 0x4E8E1FE: _worker_thread_func (mainloop-worker.c:339)
==8475== 

[bazsi]

This error is in dbd_sqlite3 and is caused by the fact that we set sqlite3_dbdir to an empty string, which that code does not handle properly.

I am thinking about a possible workaround, however this is clearly not the result of this refactor, but has been in the code for quite some time now.

[bazsi]

I have added a workaround on top of the existing branch.

[kira-syslogng]

Build SUCCESS

[kira-syslogng]

Build SUCCESS

[furiel]

I found another leak, but as the previous one, it is not caused by the refactor. _is_table_present receives a db_res parameter, that is only freed in the true branch. As a result: we leak a leak a dbi_result once, when the table is created. We can handle this independently from this patch. Approve from my side.

[pzoleex]

@kira-syslogng retest this please

[kira-syslogng]

Build SUCCESS