Update dependencies

[mostafa]

This is a maintenance PR and updates all the dependencies to the latest versions. Running mypy produces these issues, which should be resolved:

poetry run mypy --explicit-package-bases .
sigma/backends/loki/loki.py:120: error: Argument 1 to "escape" of "SigmaRegularExpression" has incompatible type "str"; expected "tuple[str]"  [arg-type]
sigma/backends/loki/loki.py:165: error: Argument 1 to "escape" of "SigmaRegularExpression" has incompatible type "str"; expected "tuple[str]"  [arg-type]
sigma/backends/loki/loki.py:185: error: Cannot override class variable (previously declared on base class "Backend") with instance variable  [misc]
sigma/backends/loki/loki.py:193: error: Incompatible types in assignment (expression has type "tuple[ConditionItem, ConditionItem]", base class "TextQueryBackend" defined the type as "tuple[ConditionItem, ConditionItem, ConditionItem]")  [assignment]
sigma/backends/loki/loki.py:273: error: Incompatible types in assignment (expression has type "tuple[str] | None", base class "TextQueryBackend" defined the type as "tuple[str]")  [assignment]
sigma/backends/loki/loki.py:298: error: Incompatible types in assignment (expression has type "ProcessingPipeline | None", base class "Backend" defined the type as "ProcessingPipeline")  [assignment]
sigma/backends/loki/loki.py:430: error: "ParentChainMixin" has no attribute "args"  [attr-defined]
sigma/backends/loki/loki.py:621: error: Incompatible types in assignment (expression has type "ConditionAND", variable has type "ConditionOR")  [assignment]
sigma/backends/loki/loki.py:626: error: No overload variant of "__setitem__" of "list" matches argument types "int", "ParentChainMixin"  [call-overload]
sigma/backends/loki/loki.py:626: note: Possible overload variants:
sigma/backends/loki/loki.py:626: note:     def __setitem__(self, SupportsIndex, ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression, /) -> None
sigma/backends/loki/loki.py:626: note:     def __setitem__(self, slice, Iterable[ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression], /) -> None
sigma/backends/loki/loki.py:633: error: No overload variant of "__setitem__" of "list" matches argument types "int", "ParentChainMixin"  [call-overload]
sigma/backends/loki/loki.py:633: note: Possible overload variants:
sigma/backends/loki/loki.py:633: note:     def __setitem__(self, SupportsIndex, ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression, /) -> None
sigma/backends/loki/loki.py:633: note:     def __setitem__(self, slice, Iterable[ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression], /) -> None
sigma/backends/loki/loki.py:692: error: Argument 1 to "convert_condition" of "Backend" has incompatible type "ParentChainMixin"; expected "ConditionOR | ConditionAND | ConditionNOT | ConditionFieldEqualsValueExpression | ConditionValueExpression"  [arg-type]
sigma/backends/loki/loki.py:733: error: Argument 1 to "len" has incompatible type "str | DeferredQueryExpression"; expected "Sized"  [arg-type]
sigma/backends/loki/loki.py:735: error: Argument 1 to "append" of "list" has incompatible type "str | DeferredQueryExpression"; expected "str"  [arg-type]
sigma/backends/loki/loki.py:742: error: Argument 1 to "len" has incompatible type "str | DeferredQueryExpression"; expected "Sized"  [arg-type]
sigma/backends/loki/loki.py:748: error: Argument 1 to "append" of "list" has incompatible type "str | DeferredQueryExpression"; expected "str"  [arg-type]
sigma/backends/loki/loki.py:773: error: Argument 1 to "escape" of "SigmaRegularExpression" has incompatible type "str"; expected "tuple[str]"  [arg-type]
sigma/backends/loki/loki.py:807: error: Argument 1 to "convert_condition" of "Backend" has incompatible type "ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression"; expected "ConditionOR | ConditionAND | ConditionNOT | ConditionFieldEqualsValueExpression | ConditionValueExpression"  [arg-type]
sigma/backends/loki/loki.py:808: error: Argument 2 to "compare_precedence" of "TextQueryBackend" has incompatible type "ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression"; expected "ConditionItem"  [arg-type]
sigma/backends/loki/loki.py:809: error: Argument 1 to "convert_condition_group" of "TextQueryBackend" has incompatible type "ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression"; expected "ConditionItem"  [arg-type]
sigma/backends/loki/loki.py:836: error: Argument 1 to "convert_condition" of "Backend" has incompatible type "ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression"; expected "ConditionOR | ConditionAND | ConditionNOT | ConditionFieldEqualsValueExpression | ConditionValueExpression"  [arg-type]
sigma/backends/loki/loki.py:837: error: Argument 2 to "compare_precedence" of "TextQueryBackend" has incompatible type "ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression"; expected "ConditionItem"  [arg-type]
sigma/backends/loki/loki.py:838: error: Argument 1 to "convert_condition_group" of "TextQueryBackend" has incompatible type "ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression"; expected "ConditionItem"  [arg-type]
sigma/backends/loki/loki.py:859: error: Argument 1 to "len" has incompatible type "SigmaType"; expected "Sized"  [arg-type]
sigma/backends/loki/loki.py:860: error: Argument 1 to "convert_str_to_re" of "LogQLBackend" has incompatible type "SigmaType"; expected "SigmaString"  [arg-type]
sigma/backends/loki/loki.py:870: error: "SigmaType" has no attribute "contains_special"  [attr-defined]
sigma/backends/loki/loki.py:871: error: Argument 1 to "convert_str_to_re" of "LogQLBackend" has incompatible type "SigmaType"; expected "SigmaString"  [arg-type]
sigma/backends/loki/loki.py:881: error: Argument 1 to "convert_value_str" of "LogQLBackend" has incompatible type "SigmaType"; expected "SigmaString"  [arg-type]
sigma/backends/loki/loki.py:891: error: Argument 2 to "LogQLDeferredUnboundStrExpression" has incompatible type "SigmaType"; expected "str"  [arg-type]
sigma/backends/loki/loki.py:896: error: Return type "str | DeferredQueryExpression | None" of "convert_condition_val_re" incompatible with return type "str | DeferredQueryExpression" in supertype "TextQueryBackend"  [override]
sigma/backends/loki/loki.py:904: error: "SigmaType" has no attribute "regexp"  [attr-defined]
sigma/backends/loki/loki.py:913: error: "SigmaType" has no attribute "regexp"  [attr-defined]
sigma/backends/loki/loki.py:918: error: Argument 2 to "LogQLDeferredUnboundStrExpression" has incompatible type "str | DeferredQueryExpression"; expected "str"  [arg-type]
sigma/backends/loki/loki.py:918: error: Argument 1 to "convert_value_re" of "LogQLBackend" has incompatible type "SigmaType"; expected "SigmaRegularExpression"  [arg-type]
sigma/backends/loki/loki.py:933: error: "SigmaType" has no attribute "values"  [attr-defined]
sigma/backends/loki/loki.py:940: error: Incompatible types in assignment (expression has type "ConditionAND", variable has type "ConditionFieldEqualsValueExpression")  [assignment]
sigma/backends/loki/loki.py:940: error: Argument 1 to "ConditionAND" has incompatible type "list[ConditionFieldEqualsValueExpression]"; expected "list[ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression]"  [arg-type]
sigma/backends/loki/loki.py:940: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
sigma/backends/loki/loki.py:940: note: Consider using "Sequence" instead, which is covariant
sigma/backends/loki/loki.py:942: error: Incompatible types in assignment (expression has type "ConditionOR", variable has type "ConditionFieldEqualsValueExpression")  [assignment]
sigma/backends/loki/loki.py:942: error: Argument 1 to "ConditionOR" has incompatible type "list[ConditionFieldEqualsValueExpression]"; expected "list[ConditionItem | ConditionFieldEqualsValueExpression | ConditionValueExpression]"  [arg-type]
sigma/backends/loki/loki.py:942: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
sigma/backends/loki/loki.py:942: note: Consider using "Sequence" instead, which is covariant
sigma/backends/loki/loki.py:958: error: "Iterable[str | SpecialChars]" has no attribute "__next__"  [attr-defined]
sigma/backends/loki/loki.py:1026: error: Unsupported target for indexed assignment ("Collection[str]")  [index]
Found 40 errors in 1 file (checked 12 source files)

[github-actions]

Pull Request Test Coverage Report for Build 6732220181

• 38 of 41 (92.68%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.6%) to 99.149%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sigma/backends/loki/loki.py	37	40	92.5%

Totals
Change from base Build 6588815655:	-0.6%
Covered Lines:	466
Relevant Lines:	470

---

💛 - Coveralls

[kelnage]

PR ready, but excuse me whilst I rant for a little about mypy and type checking in Python in general... I've just spent 4+ hours "fixing" the "issues" mypy 1.6.1 identified. But in reality, basically none of these were issues - they seemed to be caused by limitations of the mypy tool itself.

To be fair, some of these issues were not entirely caused by mypy itself, but type decisions in pySigma. E.g., SigmaType is indirectly used in a number of places in pySigma when more specific typing would avoid the issue. I've raised SigmaError exceptions when those expectations are violated, but I am not adding tests for them as they are not (as far as I can tell) reachable from actual Sigma rules.

In general, I feel the changes I had to make to please mypy in a number of places here have made the code worse/less pythonic (such as introducing lots of unnecessary type checking). Going forward, I would suggest that updating mypy major/minor versions should be done only if required (a security vulnerability, keeping aligned with other dependencies, or significant improvements to the tool) and never without a strong justification.

[mostafa]

@kelnage
Thanks for looking into this! 🙏

Since I created this PR, I won't be able to approve it, but it LGTM! Also, I agree with your point about mypy and I am against updating such packages that causes (sometimes unnecessary) disruptions, but on the other hand, they help uncover a lot of issues one has never thought of. As you already mentioned, pySigma lacks a lot of type info, which subsequently causes disruptions in our work, which puts us in an unfortunate situation (and was kinda expected). This is exactly why I always add type info to any Python code I write, especially when I submit PRs to pySigma.