Clam 2001 regex slash colon 0.105.2 #695
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There is an issue parsing PCRE patterns if the pattern contains a '/' in the middle, followed by a ':'. When splitting the subsignature (or yara regex string) by ':' delimiters to identify the offset, it will inadvertently think that the '/' in the middle of the sig is the end of the PCRE string and will therefore consider the ':' in the string as valid delimiter instead of ignoring it for being inside of the regex string. The solution I came up with is to ignore all content after a '/' when tokenizing rather than ignoring content between a matching pair of /'s. This works for LDB signatures because PCRE subsignatures are always the last subsignature and because a ':' never comes *after* the PCRE string. It works for YARA rules because the `cli_tokenize()` function is only ever used on the regex strings, never on the whole rule. Fixes: Cisco-Talos#594
|
Marked this as a draft until the fix-PR is reviewed, merged, and we can revise this to do whatever that does. That is, this PR is bad and must not be merged until it can be fixed. |
|
#696 is done. I'll fix up this backport PR next to include and squash those changes into this one. |
My recent fix for the issue where a '\' followed by ':' in a Yara regex string would fail to parse introduced a new issue that broke loading a signature in the current daily.ldb database. Unbeknownst to me at the time, you can have multiple PCRE subsignatures in a logical signature, so long as they're the last subsignatures. The previous fix made it so the signature parser muddled more than one PCRE subsignature into one messed up regex string. This commit essentially reverts the previous fix, while keeping some of the code readability improvements in that function. Instead, it addresses the problem a different way. To resolve the original problem, I'm simply checking if the signame starts with "YARA". If it does, we don't tokenize it by ':' delimiters.
I was unaware that while Yara rule regex strings may-or-may-not escape '/' characters in the regex string, Clam logical sigs MUST escape them. The Yara rule parser automatically removes the unnecessary '/': https://github.com/Cisco-Talos/clamav/blob/clamav-0.105.1/libclamav/yara_lexer.l#L509-L514 That's a good feature, we don't want to remove that. But the Clam logical sigs don't have an equivalent feature. So I changed the LDB version of the regex '/' + ':' test to include the escape '\/' character. This commit also adds some new tests to make sure we don't break support for LDB sigs with multiple PCRE subsignatures in the future, and to test that the offset feature and the case-insensitive feature work for PCRE subsignatures.
|
Okay this is updated. Leaving as a draft until version # is bumped in dev/0.105.2 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Cherry-pick of #653 for 0.105.2.
Do not merge until 0.105.2 version bump PR is merged.