sigtool --list-sigs functionality breaks on signatures over 8k characters in length

[mjbroekman]

Describe the bug

When a signature is over 8k characters in length (excluding the name), sigtool's listdb functionality will give incorrect output or report a pattern error (code highlight where the error is occurring: https://github.com/Cisco-Talos/clamav/blob/main/sigtool/sigtool.c#:~:text=cli_strbcasestr(filename%2C%20%22.cdb%22))%20%7B-,while%20(fgets(buffer%2C%20CLI_DEFAULT_LSIG_BUFSIZE%2C%20fh))%20%7B,%7D,-%7D%20else%20if%20(cli_strbcasestr )

How to reproduce the problem

$ sigtool -l

$ sigtool -l | tail
Doc.Malware.Valyria-6923115-0
Xls.Malware.Generic-6923116-0
Doc.Malware.00536d-6923117-0
Doc.Malware.Valyria-6923118-0
Xls.Malware.Sload-6923119-0
Xls.Downloader.Powload-6923120-0
ERROR: listdb: Malformed pattern line 32300 (file /var/folders/hm/f2875_6n5vg67s795z5ht8tr0000gn/T//clamav-8bee4b9314b9f94989115e7828b3c11e.tmp/main.ldb)
ERROR: listdb: Error listing database /var/folders/hm/f2875_6n5vg67s795z5ht8tr0000gn/T//clamav-8bee4b9314b9f94989115e7828b3c11e.tmp/main.ldb
ERROR: listdb: Can't list directory /Users/mbroekman/clamdb/main.cld
ERROR: listdb: Error listing database /Users/mbroekman/clamdb/main.cld

Xls.Downloader.Powload-6923120-0 is 8228 characters long and does not contain a semi-colon after the 8k mark. This is important because Doc.Dropper.Generic-6922945-0 is almost 16k in size and does contain a semi-colon after the 8k mark, but sigtool -l produces incorrect output for it also.

This occurs with both 0.103.3 and 0.103.4 (see clamav-users mailing list as well)

$ sigtool -l./test.ldb    
Doc.Dropper.Generic-6922945-0
6c652e577269746520223466343735323431376533323266343332643436353234353435376533313266366436393665363737373266366336393632326636373633363332663664363936653637373733333332326633333334333834323339333237653331326533353266363936653633366337353634363532663733373436343631373236373265363830303566356636373665373536333566373636313566366336393733373433613734323833353263333132393364323833303263333233303239303035663639366636323735363633613534373432383331326333313239336437333333333235663730373437323361323833313263333232393364326132383330326333313339323932633330326333333332336235663633366537343361323833303263333332393263333333323263333333323362356636323631373336353361323833313263333232393263333633343263333333323362356636363663363136373361323833303263333332393263333933363263333333323362356636363639366336353361323833303263333332393263333133323338326333333332336235663633363836313732363237353636336132383330326333333239326333313336333032633333333233623566363237353636373336393761336132383330326333333239326333313339333232633333333233623566373436643730363636653631366436353361323833313263333232393263333233323334326333333332336236663730363537323631373436663732336433613361323833313263333332393364323332383331326333313239326332383331326333343239336432363238333132633331323932633238333132633335323933643261323833313263333132393263323833313263333632393364323632383331326333373239336436623238333132633331323932633238333032633336323933623361356635613465333635663639366636323735363636313533343535323462353335663362333234313265336235663566363236313733363535663633373436663732336133613238333132633338323933643233323833313263333132393263323833303263333632393263323833313263333532393263323833313263333632393263323833303263333632393362336135663561346533363566363936663632373536363433333234353532346235333566336233323431326533623566356636333666366437303566363337343666373233613361323833313263333832393361356635613465333635663639366636323735363634333331343535323462353335663362333234313265336235663566363236313733363535663633373436663732336122
Win.Adware.Linkury-16152
00720063006800550072006c004300680072006f006d0065002c0020003a00500072006f0074006500630074006f007200730048006f006d0065005000610067006500550052004c004300680072006f006d0065002c0020003a00500072006f0074006500630074006f00720073004e0065007700540061006200550052004c004300680072006f006d0065002c0020003a00500072006f0074006500630074006f0072007300530065006100720063006800550072006c00460046002c0020003a00500072006f0074006500630074006f007200730048006f006d0065005000610067006500550052004c00460046002c0020003a00500072006f0074006500630074006f00720073004e0065007700540061006200550052004c00460046002c0020003a00500072006f0074006500630074006f007200730053006500610072006300680044006f006d00610069006e002c0020003a00500072006f0074006500630074006f007200730048006f006d0065005000610067006500550052004c00490045002c0020003a00500072006f0074006500630074006f0072007300530065006100720063006800550072006c00490045002c0020003a0048006f00730074007300460069006c0065004d006f006e00690074006f0072004c0069006e006b0075007200790044006f006d00610069006e0073002c0020003a00420061007300690063004c0069006e006b0054006f004f00660066006500720073004d0061006e00610067006500720043006c006f007500640053006500720076006900630065002c0020003a00470065007400420075006e0064006c0069006e0067004100700070006c00690063006100740069006f006e00730054006f0049006e007300740061006c006c00460075006e006300740069006f006e002c0020003a0047006500740041007000700072006f0076006500640049006e00730074006c006c006100740069006f006e00460075006e006300740069006f006e002c0020003a005400610073006b006200610072004e006f0074006900660069006500720045007800650050006100740068002c0020003a00440065006600610075006c0074004d00610078004f00720064006500720073002c0020003a00440065006600610075006c00740049006e00740065007200760061006c002c0020003a0043006800650063006b0069006e0067004f006600660065007200730049006e00740065007200760061006c0049006e004d0069006e0075007400650073002c0020003a00500072006900760061007400650049006e007600650073007400690067006100740069006f006e0045006e00640070006f0069006e0074002c0020003a00530068006f007200740049006e00740065007200760061006c0054006f0055007000640061007400650046006c0061006700730049006e004d0069006e0075007400650073002c0020003a004c006f006e00670049006e00740065007200760061006c0054006f0055007000640061007400650046006c0061006700730049006e004d0069006e0075007400650073002c0020003a0052004f00540058006d006c004f007000650072006100740069006f006e007300550072006c002c0020003a0044006c006c0049006e006a0065006300740069006f006e00550072006c002c0020003a005400720061007900570069006e0064006f0077004800650061006400650072004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f0032004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f004e0054004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f00440053004c006100620065006c002c0020003a00540061006b006500440065006600610075006c0074005300650061007200630068002c0020003a00540061006b0065004e00650077005400610062002c0020003a00540061006b00650048006f006d0065005000610067006500
Win.Adware.Linkury-16148
00072006f0074006500630074006f007200730048006f006d0065005000610067006500550052004c004300680072006f006d0065002c0020003a00500072006f0074006500630074006f00720073004e0065007700540061006200550052004c004300680072006f006d0065002c0020003a00500072006f0074006500630074006f0072007300530065006100720063006800550072006c00460046002c0020003a00500072006f0074006500630074006f007200730048006f006d0065005000610067006500550052004c00460046002c0020003a00500072006f0074006500630074006f00720073004e0065007700540061006200550052004c00460046002c0020003a00500072006f0074006500630074006f007200730053006500610072006300680044006f006d00610069006e002c0020003a00500072006f0074006500630074006f007200730048006f006d0065005000610067006500550052004c00490045002c0020003a00500072006f0074006500630074006f0072007300530065006100720063006800550072006c00490045002c0020003a0048006f00730074007300460069006c0065004d006f006e00690074006f0072004c0069006e006b0075007200790044006f006d00610069006e0073002c0020003a00420061007300690063004c0069006e006b0054006f004f00660066006500720073004d0061006e00610067006500720043006c006f007500640053006500720076006900630065002c0020003a00470065007400420075006e0064006c0069006e0067004100700070006c00690063006100740069006f006e00730054006f0049006e007300740061006c006c00460075006e006300740069006f006e002c0020003a0047006500740041007000700072006f0076006500640049006e00730074006c006c006100740069006f006e00460075006e006300740069006f006e002c0020003a005400610073006b006200610072004e006f0074006900660069006500720045007800650050006100740068002c0020003a00440065006600610075006c0074004d00610078004f00720064006500720073002c0020003a00440065006600610075006c00740049006e00740065007200760061006c002c0020003a0043006800650063006b0069006e0067004f006600660065007200730049006e00740065007200760061006c0049006e004d0069006e0075007400650073002c0020003a00500072006900760061007400650049006e007600650073007400690067006100740069006f006e0045006e00640070006f0069006e0074002c0020003a00530068006f007200740049006e00740065007200760061006c0054006f0055007000640061007400650046006c0061006700730049006e004d0069006e0075007400650073002c0020003a004c006f006e00670049006e00740065007200760061006c0054006f0055007000640061007400650046006c0061006700730049006e004d0069006e0075007400650073002c0020003a0052004f00540058006d006c004f007000650072006100740069006f006e007300550072006c002c0020003a0044006c006c0049006e006a0065006300740069006f006e00550072006c002c0020003a005400720061007900570069006e0064006f0077004800650061006400650072004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f0032004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f004e0054004c006100620065006c002c0020003a005400720061007900570069006e0064006f00770049006e0066006f00440053004c006100620065006c002c0020003a00540061006b006500440065006600610075006c0074005300650061007200630068002c0020003a00540061006b0065004e00650077005400610062002c0020003a00540061006b00650048006f006d0065005000610067006500
Xls.Downloader.Powload-6923120-0
ERROR: listdb: Malformed pattern line 8 (file ./test.ldb)
$ wc -l test.ldb
       4 test.ldb

Attachments

test.ldb with the 4 signatures over 8k in size: test.ldb.txt

[micahsnyder]

I neglected to update this ticket from our conversation in the mailing list. This is an issue that was fixed in 0.104 but still exists in 0.103: https://lists.clamav.net/pipermail/clamav-users/2021-November/012119.html

Fix: 13af789f4ed

[micahsnyder]

I just cherry-picked this fix to the dev/0.103.5 branch, verified that --list-sigs works now, and pushed it. The upcoming 0.103.5 release will include this fix.