Wiki 3: Label Studio Integration & Active Learning

Label Studio Annotation Backend & Active Learning Protocol

1. Technical Framework Architecture

The system utilizes Label Studio as an external human-in-the-loop validation layer. The code inside label_studio_backend/ serves as a bridge, allowing machine learning models to dynamically interact with the annotation UI.

sequenceDiagram participant LS as Label Studio UI participant BE as Custom Python Backend participant ML as PyTorch Model Inference LS->>BE: Request Pre-annotations (Webhook) BE->>ML: Pass Raw Text through Inference Pipeline ML-->>BE: Return Softmax Probabilities & Predicted Labels BE-->>LS: Send Standardized JSON Predictions

2. Active Learning Cycle (Model-Assisted Labeling)

Seniors utilize this setup to build an Active Learning Loop. Instead of humans labeling everything blindly, the system optimizes time metrics:

1. Low-confidence predictions from the PyTorch model are surfaced to human annotators first.
2. New human-curated data is saved back to database triggers.
3. The training pipeline fetches the newly annotated items using uv run and automatically triggers a refined training epoch via the Makefile.

3. Local Setup & Execution Guide

To deploy the synchronization service locally on macOS, ensure your environment flags are fully set:

# 1. Sync dependencies with the ultra-fast uv tool
uv sync --package label_studio_backend

# 2. Spin up the localized web server
uv run python -m label_studio_backend.main --port 9090