Introduce machinery to handle multiple Handler streams, and parsing handler configurations from OKS

[emmuhamm]

Description

Close #35

Needs to go in with DUNE-DAQ/erskafka#43

To fully implement ERS into drunc, some machinery is required to be introduced to the logging infrastructure in the Python implementation of the DUNEDAQ.

This PR adds several things

• New handlers are added
  • ERSKafkaLogHandler imported from erskafka
  • Several dummy handlers to serve as placeholders
• HandleIDFilter is a logging filter that is added to every Handler by default. It will let that Handler fire if and only if the same handler is in the list of allowed handlers that must be parsed by a log message
• Enums:
  • HandlerType to classify set of Handlers
  • StreamType to classify the different streams that exist
• Configuration dataclasses
  • ERSHandlerConf to hold the relevant ERS configuration from OKS
  • LogHandlerConf to hold the set of relevant Handlers necessary for a given stream

Full information the logic behind this is described in the logging page in the drunc wiki

A full demonstration of how this works in practice is found in the demonstration python script that has been updated as part of this PR.

Testing instructions for review

• Setup latest nightly (tested on NFD_DEV_260109_A9) and check out this branch and Update handlers in preparation for introducing ERS to drunc erskafka#43.
• run daqpytools-logging-demonstrator -r -f logger.log --handlertypes
• Check terminal output
  • Everything should appear as expected, the log messages should be clear enough which needs to go where
• Check file output
  • See if only whats in the file output is expected
• Check Kakfa output
  • If unsure, check out instructions in Update handlers in preparation for introducing ERS to drunc erskafka#43

Type of change

• Documentation (non-breaking change that adds or improves the documentation)
• New feature or enhancement (non-breaking change which adds functionality)
• Optimization (non-breaking change that improves code/performance)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (whatever its nature)

Testing checklist

• Unit tests pass (e.g. dbt-build --unittest)
• Minimal system quicktest passes (pytest -s minimal_system_quick_test.py)
• Full set of integration tests pass (daqsystemtest_integtest_bundle.sh)
• Python tests pass if applicable (e.g. python -m pytest)
• Pre-commit hooks run successfully if applicable (e.g. pre-commit run --all-files)

Comments here on the testing

Further checks

• Code is commented where needed, particularly in hard-to-understand areas
• Code style is correct (dbt-build --lint, and/or see https://dune-daq-sw.readthedocs.io/en/latest/packages/styleguide/)
• If applicable, new tests have been added or an issue has been opened to tackle that in the future.
  (Indicate issue here: # (issue))

FOLLOW UP

• This PR does not handle multiple protobufstreams with different URL and port number. It is hardcoded to use the default one used in python so far (monkafka.cern.ch:30092, ers_stream). This should be implemented at some point (See [Feature]: Implement new set of Handlers #36)

TODOs:

• Run linting
• Merge develop in here

[emmuhamm]

@PawelPlesniak This, and the sister PR DUNE-DAQ/erskafka#43 is ready for review. Theres a few things I've highlighted here that I think we should discuss, but the general implementation should be ready for you to go through

Let me know what you think :)

[PawelPlesniak]

Once done with comments in code, I will re-read, will choose HandlerConf->PyLogHandlerConf?

[PawelPlesniak]

Also please do not forget to run the following prior to completing this

ruff check . --fix
ruff format .

The format will fix your indents, and the check will tell you things like missing args

[PawelPlesniak]

LGTM, structure is sound and the implementation works as intended, but OpMon requires removal

[PawelPlesniak]

Note for future reviewers

The OpMon implementation is correct to have a protobuf handler, but has not yet been finished - there will be a future PR associated with the listed issue.