Return more meaningful info about placeholders

[mostafa]

Hey @thomaspatzke,

I've been working on placeholders and found out that if a placeholder is defined in the rule, and there's no matching transformation in any of the provided pipelines, it'll be regarded as an unknown type and a TypeError is raised. The TypeError is caught by the general Exception block and returned with a generic message:

TypeError: unhashable type: 'Placeholder' (while applying processing pipeline on rule /path/to/some_rule.yml)

This is not very useful, so I suggest we catch the exception and modify this line to consider Placeholder a special object to return NO_VALUE_PROVIDED instead. This way, the query is safely converted, yet the value of the placeholder will be NO_VALUE_PROVIDED. I made two fixes to address this, and they more or less do the same thing:

1. https://github.com/mostafa/pySigma/tree/meaningful-placeholders-proposal-1
2. https://github.com/mostafa/pySigma/tree/meaningful-placeholders-proposal-2

I'd be very happy to have your feedback on this. (This look more like an issue, rather than an idea for discussion. 😁)

CC: @kelnage

[thomaspatzke]

Hi @mostafa! Thanks for pointing this out, it's indeed an issue because the initial intention was to raise an exception for this case, perhaps a SigmaValueError or even something more specific like a SigmaPlaceholderError that could be derived from the SigmaValueError. It appears that I've forgot to implement this and it's now errors in an uncontrolled way.

A general decision I've made for pySigma is to not recover gracefully in such situations but to fail hard if the result will not work as expected by the rule writer. The reason is, that there are lots of users who convert the whole rule repository into queries and expect that these work without reviewing them.

For placeholders there's the WildcardPlaceholderTransformation. It's intention is to put it at the end of a processing pipeline and replace all remaining placeholders with *-wildcards. This will cause queries that will match on more events and therefore don't cause the harm of not detecting anything, as this would be the case for a placeholder value.

[thomaspatzke]

Created an issue #138

[mostafa]

@thomaspatzke
I think this is resolved, so I'll close this discussion.