Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clam 1288 Signature regex filter search optimization #837

Merged

Conversation

micahsnyder
Copy link
Contributor

PDB, WDB, and CDB signatures can use a regex feature to match domain names. At one time in the process we ran a filter search to speed up filtering out non-matching static patterns but were accidentally discarding the result. When tested, it turns out it wasn't working correctly anyways.

Since then, we've fixed some bugs and upgraded the regex implementation to the latest version. After re-testing, I have found that the filter_search() appears to be working correctly now, both in the unit tests and with the existing CDB signatures.

PDB, WDB, and CDB signatures can use a regex feature to match
domain names. At one time in the process we ran a filter search
to speed up filtering out non-matching static patterns but were
accidentally discarding the result. When tested, it turns out
it wasn't working correctly anyways.

Since then, we've fixed some bugs and upgraded the regex
implementation to the latest version. After re-testing, I have
found that the filter_search() appears to be working correctly
now, both in the unit tests and with the existing CDB signatures.
@micahsnyder
Copy link
Contributor Author

I went back to figure out when the test was fixed. I didn't spent a lot of time testing, but I did find that with this change version 0.104 has the following test failure:

2: Running suite(s): cl_suite
2:  cli
2:  jsnorm
2:  str
2:  regex
2:  disasm
2:  unique
2:  matchers
2:  htmlnorm
2:  bytecode
2: 99%: Checks: 1176, Failures: 1, Errors: 0
2: /home/micasnyd/workspace/clamav-micah/unit_tests/check_regex.c:295:F:regex_list:regex_list_match_test:13: regex_list_match

That failure is gone as of 0.105, and of course is not present in main either.

Outside of some expected failures in the CI pipelines, everything looks good.

@micahsnyder micahsnyder merged commit d7317ca into Cisco-Talos:main Feb 22, 2023
19 of 24 checks passed
@micahsnyder micahsnyder deleted the CLAM-1288-regex-filter_search branch February 22, 2023 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant