log-processor-py

Simple Python processors to send an array of messages to a channel and to log an incoming stream to the RDF-Connect logging system.

Usage

To use the Python rdfc:LogProcessorPy or rdfc:SendProcessorPy in your RDF-Connect pipeline, you need to have a pipeline configuration that includes the rdfc:PyRunner (check out their documentation to find out how to install and configure it).

You can install the Python runner package using the following command:

uv add rdfc_log-processor

Next, you can add the processors to your pipeline configuration as follows:

@prefix rdfc: <https://w3id.org/rdf-connect#>.
@prefix owl: <http://www.w3.org/2002/07/owl#>.

# Import the processor
<> owl:imports <./.venv/lib/python3.13/site-packages/rdfc_log_processor/processor.ttl>.

### Define the channels your processor needs
<channel> a rdfc:Writer, rdfc:Reader.

# Attach the processor to the pipeline under the PyRunner
# Add the `rdfc:processor <send>, <log>` statement under the `rdfc:consistsOf` statement of the `rdfc:PyRunner`

### Define and configure the processors
<send> a rdfc:SendProcessorPy;
       rdfc:writer <channel>;
       rdfc:msg "Hello, World!", "Good afternoon, World!",
                "Good evening, World!", "Good night, World!".

<log> a rdfc:LogProcessorPy;
      rdfc:reader <channel>;
      rdfc:level "info";
      rdfc:label "test".

Configuration

Parameters of rdfc:SendProcessorPy:

• rdfc:writer: The channel to which the processor will write messages.
• rdfc:msg: The messages to be sent by the processor.

Parameters of rdfc:LogProcessorPy:

• rdfc:reader: The channel from which the processor will read messages.
• rdfc:writer: The channel to which the processor will write log messages (optional).
• rdfc:level: The log level to use (e.g., "debug", "info", "warning", "error", "critical"). Defaults to "info".
• rdfc:label: A label for the log messages, which can be used to filter or categorize logs. Defaults to "log".
• rdfc:raw: If set to true, the processor will log raw messages to stdout without any formatting instead of to the RDF-Connect logging system. Defaults to false.

Development

The Packaging Python Projects guide was used to set up this project. As build backend, the default Hatchling is used, for which the pyproject.toml file is configured. That file tells build frontend tools like pip which backend to use. This project uses uv as package manager.

First, make sure you have Hatch installed:

pip install hatch
# OR
brew install hatch
# OR another method of your choice

Then, create a virtual environment and spawn a shell. This will automatically install the project dependencies defined in pyproject.toml:

hatch env create
hatch shell

You can build the project with:

hatch build

You can run the tests using:

hatch test

Lastly, you can publish the package to PyPI with:

hatch publish

Project Structure

log-processor-py/             # Root directory of the project
├── src/                      # Source code directory
│   └── rdfc_log_processor/   # Package directory
│       ├── __init__.py       # Package initialization, allows importing as a regular package
│       ├── processor.py      # Contains the main logic for the Python processor
│       └── processor.ttl     # RDF schema for the processor, used for metadata and configuration
├── tests/                    # Directory for unit tests
├── pyproject.toml            # Project metadata and build configuration