rsyslog crash when using omelasticsearch

[snaix]

Expected behavior

A normal omelasticsearch behavior

Actual behavior

Crash in computeMessageSize() at omelasticsearch.c:650, because of an illegal action param.

The omelasticsearch config like: dynSearchIndex = "on", searchIndex ="foo(template name)", template = "bar"
so we have two templates in one action.

the call stack is:

#1  0x00007f7c60304650 in computeMessageSize (pWrkrData=0x7f7ad4002e10, tpls=0x7f7ae77fd950, message=<optimized out>) at omelasticsearch.c:650
#2  doAction (pMsgData=0x7f7ae77fd950, pWrkrData=0x7f7ad4002e10) at omelasticsearch.c:1538
#3  0x0000000000458625 in actionCallDoAction (pWti=0x1b94cc0, iparams=<optimized out>, pThis=0x141a9e0) at ../action.c:1138
#4  actionProcessMessage (pThis=0x141a9e0, actParams=0x7f7ad43f3780, pWti=0x1b94cc0) at ../action.c:1180
#5  0x0000000000458984 in doTransaction (nparams=3, iparams=0x7f7ad43f3740, pWti=0x1b94cc0, pThis=0x141a9e0) at ../action.c:1208
#6  actionTryCommit (pThis=0x141a9e0, pWti=0x1b94cc0, iparams=0x7f7ad43f3740, nparams=3) at ../action.c:1242
#7  0x0000000000458f21 in actionCommit (pThis=pThis@entry=0x141a9e0, pWti=pWti@entry=0x1b94cc0) at ../action.c:1449
#8  0x000000000045a300 in actionCommitAllDirect (pWti=pWti@entry=0x1b94cc0) at ../action.c:1500
#9  0x000000000045369c in processBatch (pBatch=0x1b94cf0, pWti=0x1b94cc0) at ruleset.c:673
#10 0x000000000040e95c in msgConsumer (notNeeded=<optimized out>, pBatch=0x1b94cf0, pWti=0x1b94cc0) at rsyslogd.c:702
#11 0x000000000044f10c in ConsumerReg (pThis=0x141c1d0, pWti=0x1b94cc0) at queue.c:2044
#12 0x000000000044c22e in wtiWorker (pThis=pThis@entry=0x1b94cc0) at wti.c:374
#13 0x000000000044b1c8 in wtpWorker (arg=0x1b94cc0) at wtp.c:422
#14 0x00007f7c6466cdf5 in start_thread () from /usr/lib64/libpthread.so.0
#15 0x00007f7c637491ad in clone () from /usr/lib64/libc.so.6

In coredump file, I saw that a wrong iparam content in actionCommit() at action.c:1449.
In this case, wrkrInfo->p.tx.currIParam == 185, so a temporary memory would be allocated in line 1432; call actionTryRemoveHardErrorsFromBatch() for retrying; finally, got iparams because action returned RS_RET_SUSPENDED.

the iparams, like:

(gdb) p iparams[0]
$1 = {param = 0x7f86ac0d93f0 "xxxxx...(a message content generated by template)"..., lenBuf = 1664, lenS
tr = 249}
(gdb) p iparams[1]
$2 = {param = 0x7f86ac66b970 "yyyyy...(a message content generated by template)"..., lenBuf = 2944, lenS
tr = 222}
(gdb) p iparams[2]
$3 = {param = 0x7f86ac07f720 "es_index...(a index generated by searchIndex)", lenBuf = 128, lenStr = 21}
(gdb) p iparams[3]
$4 = {param = 0x657322202c223030 <Address 0x657322202c223030 out of bounds>, lenBuf = 1919252082, lenStr = 574833737}

Follow the normal process, it should be like this:

(gdb) p iparams[0]
$1 = {param = 0x7f86ac0d93f0 "xxxxx...(a message content generated by template)"..., lenBuf = 1664, lenS
tr = 249}
(gdb) p iparams[1]
$3 = {param = 0x7f86ac07f720 "es_index...(a index generated by searchIndex)", lenBuf = 128, lenStr = 21}
(gdb) p iparams[2]
$2 = {param = 0x7f86ac66b970 "yyyyy...(a message content generated by template)"..., lenBuf = 2944, lenS
tr = 222}
(gdb) p iparams[3]
$3 = {param = 0x7f86ac07f720 "es_index...(a index generated by searchIndex)", lenBuf = 128, lenStr = 21}

A completed param pair is a message generated by template then a index generated by searchIndex followed.

In this case there are 2 param pairs for 2 log messages.

So omelasticsearch picks out of order parameters and illegal index param, and crash happend in computeMessageSize().

It seems a wild pointer exists: some code freed a pointer but still holding, so though actionCommit() allocated memory for temporary params, modification still happened.
I have read the code of this part, no problem was found ;(

Steps to reproduce the behavior

Sorry I can't, but it have happened a few times

Environment

• rsyslog version: 8.37.0
• platform: CentOS Linux release 7.3.1611 (Core) x86_64