Overview

The LLM‑Router project ships with a modular plugin system that lets you plug‑in anonymizers (also called maskers) and guardrails into request‑processing pipelines.
Each plugin implements a tiny, well‑defined interface (apply) and can be composed in an ordered list to form a pipeline. The pipelines are instantiated by the MaskerPipeline and GuardrailPipeline classes and are driven automatically by the endpoint logic in endpoint_i.py.

---

1. Anonymizers (Maskers)

1.1 What they do

• Goal – Remove or replace personally‑identifiable information (PII) from a payload before it reaches the LLM or external service.
• Typical strategy – Run a pipeline of maskers, to locate spans that correspond to IDs, etc., and replace each span with a placeholder such as {{MASKED_ITEM}}.

1.2 Built‑in anonymizer plugins

Full list of FastMaskerPlugin masking strategies is located in README.md file.

Plugin	Description	Technical notes
FastMaskerPlugin (fast_masker_plugin.py)	A thin wrapper around the FastMasker utility class. It receives a JSON‑compatible payload and returns the same payload with all detected PII masked.	Implements PluginInterface. The heavy lifting is delegated to FastMasker.mask_payload(payload). No extra I/O; the FastMasker instance is created once in __init__.

1.3 How a masker is used

1. The endpoint (e.g. EndpointI._do_masking_if_needed) checks the global flag FORCE_MASKING.
2. If enabled, it creates a MaskerPipeline with the list of masker plugin identifiers (e.g. ["fast_masker"]).
3. The pipeline calls each plugin’s apply method sequentially, feeding the output of one as the input to the next.
4. The final payload – now stripped of PII – proceeds to the rest of the request flow (guardrails, model dispatch, etc.).

---

2. Guardrails

2.1 What they do

• Goal – Verify that a request (or its response) complies with policy rules (e.g. no hateful, illegal, or unsafe content).
• Typical strategy – Split the payload into manageable text chunks, run a pipeline of guardrails, aggregate per‑chunk scores, and decide whether the overall request is safe.

2.2 Built‑in guardrail plugins

Plugin	Description	Technical notes
NASKGuardPlugin (nask_guard_plugin.py)	An HTTP‑based guardrail that forwards the payload to the external NASK guardrail service (/nask_guard endpoint) and returns a boolean safe flag together with the raw response.	Inherits from HttpPluginInterface. The apply method calls _request(payload) (provided by the base class) and extracts results["safe"]. Errors are caught and logged; on failure the plugin returns (False, {}).
(Implicit) GuardrailProcessor (processor.py)	Not a plugin per‑se, but the core logic used by the internal NASK guardrail Flask route (nask_guardrail). It tokenises the payload, creates overlapping chunks, runs a Hugging‑Face text‑classification pipeline, and produces a detailed safety report.	Handles model loading (AutoTokenizer, pipeline("text‑classification")), chunking (_chunk_text), and scoring thresholds (MIN_SCORE_FOR_SAFE, MIN_SCORE_FOR_NOT_SAFE). Returns a dict: {"safe": <bool>, "detailed": [...]}.

2.3 How a guardrail is used

1. The endpoint calls _is_request_guardrail_safe(payload) (or the analogous response guardrail).
2. If FORCE_GUARDRAIL_REQUEST is true, a GuardrailPipeline is built from the configured plugin IDs (e.g. ["nask_guard"]).
3. The pipeline iterates over each guardrail plugin; each apply returns (is_safe, message).
4. The first plugin that reports is_safe=False short‑circuits the pipeline and the request is rejected with a 400/500 error payload.

---

3. Pipelines

Both masker and guardrail pipelines share the same design pattern:

Class	Purpose
MaskerPipeline (pipeline.py – masker version)	Executes a list of masker plugins in order, transforming the payload step‑by‑step.
GuardrailPipeline (pipeline.py – guardrail version)	Executes guardrail plugins sequentially, stopping on the first failure.

3.1 Registration

• Plugins are registered lazily via MaskerRegistry.register(name, logger) or GuardrailRegistry.register(name, logger).
• The registry maps a string identifier (e.g. "fast_masker") to a concrete plugin class, allowing pipelines to resolve the classes at runtime.

3.2 Configuration

All plugin identifiers are stored in environment variables or constants such as:

MASKING_STRATEGY_PIPELINE = ["fast_masker"]
GUARDRAIL_STRATEGY_PIPELINE_REQUEST = ["nask_guard"]

These lists are consumed by the endpoint initialization (EndpointI._prepare_masker_pipeline, EndpointI._prepare_guardrails_pipeline).

---

4. Adding a New Plugin

1. Create a subclass of either PluginInterface (for maskers) or HttpPluginInterface / a custom guardrail base.
2. Define a name class attribute – this is the identifier used in pipeline configuration.
3. Implement apply(self, payload: Dict) -> Dict (masker) **or apply(self, payload: Dict) -> Tuple[bool, Dict] ** (guardrail).
4. Register the plugin – either automatically via the registry’s register call in the pipeline constructor, or manually by calling MaskerRegistry.register(name=MyPlugin.name, logger=logger).

Example stub for a new masker:

# my_custom_masker.py
from llm_router_plugins.maskers.plugin_interface import PluginInterface
import logging
from typing import Dict, Optional


class MyCustomMasker(PluginInterface):
    name = "my_custom_masker"

    def __init__(self, logger: Optional[logging.Logger] = None):
        super().__init__(logger=logger)
        # Load any heavy resources here (e.g., a spaCy model)

    def apply(self, payload: Dict) -> Dict:
        # Perform your masking logic and return the modified payload
        return payload

After placing the file in llm_router_plugins/maskers/plugins/, you can enable it by adding "my_custom_masker" to MASKING_STRATEGY_PIPELINE.

---

5. Summary

• Anonymizers (FastMaskerPlugin, BANonymizer) scrub PII from requests.
• Guardrails (NASKGuardPlugin, internal GuardrailProcessor) enforce safety policies.
• Pipelines (MaskerPipeline, GuardrailPipeline) orchestrate the sequential execution of these plugins, short‑circuiting on failure for guardrails.
• The system is extensible: new plugins are just classes that obey the tiny interface contract and can be referenced by name in the configuration.

These components together give the LLM‑Router a flexible, policy‑driven request‑processing stack that can be tailored to any deployment scenario.