Skip to content

Conversation

antoinemzs
Copy link
Member

@antoinemzs antoinemzs commented Dec 19, 2024

Proposed changes

  • Refactor pyobas to expose more encapsulated utilities to accelerate building of OpenBAS collectors
  • Backwards compatibility with legacy helpers

Related issues

Checklist

  • I consider the submitted work as finished
  • I tested the code for its functionality
  • I wrote test cases for the relevant uses case
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

Main features:

  • Migrate configuration logic into its own submodule; refactor the legacy OpenBASConfigHelper to use this new module
  • Migrate collector setup and run logic into its own submodule; refactor the legacy OpenBASCollectorHelper to use the new modules
  • Create a new submodule specifically to handle Expectations coming from the API to give them a behaviour. Existing collectors have to manipulate data classes and be tightly coupled to various API endpoints to fetch or update them. With new encapsulation we can hide this behaviour and expose a minimal interface to clients.
  • Migrate matching logic to expectations. Not all existing mathcing logic was moved though as it is not used by our collectors (it's unclear whether undisclosed third party collectors have been relying on the part that wasn't migrated, although it had been written for an internal need that did not eventually manifest)

An example collector may be built this way:

from pyobas.daemons import CollectorDaemon
from pyobas.configuration import Configuration

def do_work(collector):
    # get expectations
    expectations = collector.api.inject_expectation.expectations_models_for_source(collector.get_id())
    # get some alerts
    alerts = get_alerts_from_integration_api()
    # do matching work
    match_alerts(alerts, expectations)


if __name__ == "__main__":
    config_hints={
        # API information
        "openbas_url": {"env": "OPENBAS_URL", "file_path": ["openbas", "url"]},
        "openbas_token": {
            "env": "OPENBAS_TOKEN",
            "file_path": ["openbas", "token"],
        },
        [...]
    }

    CollectorDaemon(
        configuration=Configuration(
            config_hints=config_hints
        ),
        callback=do_work
    ).start()

@antoinemzs antoinemzs changed the title Issue/68 cs siem [client] Refactor helpers to expose encapsulated and tested components for building collectors Dec 20, 2024
@antoinemzs antoinemzs marked this pull request as ready for review December 20, 2024 15:58
@antoinemzs antoinemzs added filigran team use to identify PR from the Filigran team do not merge Do not merge this PR until this tag will be removed labels Dec 21, 2024
@antoinemzs antoinemzs changed the base branch from main to release/1.11.0 December 23, 2024 11:56
Copy link
Member

@Dimfacion Dimfacion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I'm no python expert so if someone else wants to chime in, it might be better ...

Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
Signed-off-by: Antoine MAZEAS <antoine.mazeas@filigran.io>
@antoinemzs antoinemzs merged commit b41ab06 into release/1.11.0 Jan 9, 2025
3 checks passed
@antoinemzs antoinemzs deleted the issue/68_cs_siem branch January 9, 2025 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Do not merge this PR until this tag will be removed filigran team use to identify PR from the Filigran team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants