SigX

Implicit Feedback Signal Extraction for LLM Alignment

Extract preference signals from conversation logs — no human annotation required.

SigX Documentation | Quick Start | API Reference | Contributing

Why SigX?

Traditional LLM alignment relies on expensive human annotations or synthetic preferences from GPT-4. But every production chat log already contains rich preference signals:

What Users Do	What It Means	SigX Signal
Re-ask the same question	Previous answer was unhelpful	`rephrase`
Say "Actually I meant..."	Model misunderstood	`correction`
Say "That's wrong"	Explicit dissatisfaction	`negative`
Say "Thanks! Perfect!"	Explicit satisfaction	`positive`
Say "Never mind" and leave	User gave up	`abandon`

SigX mines these signals at zero marginal cost — turning your existing chat logs into DPO/KTO training data.

Inspired by WildFeedback (Microsoft Research, 2024) and the broader literature on learning from implicit preferences.

Comparison with Related Tools

Feature	SigX	TRL (HuggingFace)	VeRL
Extract signals from raw logs	Yes	No (expects labeled pairs)	No
Implicit feedback detection	Yes	No	No
DPO/KTO output format	Yes	Yes (training only)	Yes (training only)
Auto-infer chosen responses	Yes	No	No
Pluggable extractors	Yes	No	No
Built-in evaluation metrics	Yes	No	No
Zero GPU required	Yes	No	No

SigX is not a training framework — it sits upstream of TRL/VeRL, producing the preference data they consume.

Highlights

Core Capabilities

Pluggable Extractors — Mix regex, TF-IDF, ML, and LLM detectors
Smart DPO Conversion — Auto-infer chosen from subsequent positive turns
Multi-Format Output — DPO, KTO, rejection sampling for TRL/VeRL
Built-in Evaluation — Per-type precision/recall/F1 against benchmarks

Design Principles

Lightweight — Only numpy + scikit-learn. No GPU needed.
Extensible — Subclass BaseExtractor for custom detectors
Production-ready — Type annotations, 92 tests, CI/CD
Framework-agnostic — Output compatible with TRL, VeRL, and more

Quick Start

pip install sigx

1. Extract Signals from Conversations

from sigx import Pipeline, RephraseDetector, SentimentDetector

pipeline = Pipeline([
    RephraseDetector(similarity_threshold=0.6),
    SentimentDetector(min_confidence=0.6),
])

conversations = [
    {
        "conversation_id": "1",
        "conversation": [
            {"role": "user", "content": "What is Python?"},
            {"role": "assistant", "content": "Python is a type of snake."},
            {"role": "user", "content": "That's not what I asked. I meant the programming language."},
            {"role": "assistant", "content": "Python is a high-level programming language created by Guido van Rossum."},
            {"role": "user", "content": "Thanks! That's exactly what I needed."},
        ]
    }
]

signals = pipeline.run(conversations)
for s in signals:
    print(f"[{s.signal_type}] confidence={s.confidence:.2f} | {s.evidence[:60]}...")

Output:

[negative] confidence=0.95 | That's not what I asked. I meant the programming language....
[positive] confidence=0.80 | Thanks! That's exactly what I needed....

2. Convert to DPO Training Pairs

pairs = pipeline.to_dpo(conversations)
for p in pairs:
    print(f"Prompt:   {p.prompt[:80]}...")
    print(f"Rejected: {p.rejected[:80]}...")
    print(f"Chosen:   {p.chosen[:80] if p.chosen else 'None'}...")

Output:

Prompt:   User: What is Python?...
Rejected: Python is a type of snake....
Chosen:   Python is a high-level programming language created by Guido van Rossum....

3. Feed to TRL for Training

from trl import DPOTrainer

trainer = DPOTrainer(
    model=model,
    train_dataset=pairs,  # SigX output works directly with TRL
)
trainer.train()

Installation

Published on PyPI as sigx.

Install Command	Includes
`pip install sigx`	Core (numpy, scikit-learn)
`pip install "sigx[llm]"`	+ OpenAI API for LLMExtractor
`pip install "sigx[wildchat]"`	+ HuggingFace datasets for WildChat
`git clone https://github.com/Fengrru/sigx.git && pip install -e ".[dev]"`	+ pytest, ruff, mypy, pyright (from source)

Signal Types

SigX detects five categories of implicit feedback:

Signal	Trigger Pattern	Confidence Range	Training Implication
`rephrase`	User re-asks similar question (TF-IDF cosine sim)	0.60 – 1.00	Previous response → `rejected`
`correction`	"Actually I meant...", "No, that's not..."	0.60 – 0.95	Model output → `rejected`
`negative`	"That's wrong", "Not helpful"	0.65 – 0.95	Explicit dissatisfaction → `rejected`
`positive`	"Thanks!", "Exactly what I needed"	0.60 – 0.90	Explicit satisfaction → `chosen` / `label=True`
`abandon`	"Never mind", "I give up", trailing assistant	0.35 – 0.90	User gave up → `rejected`

Extractors

RephraseDetector

Detects when users rephrase or repeat a question using TF-IDF cosine similarity. For CJK text (Chinese/Japanese/Korean) it automatically switches to character n-grams, since whitespace tokenization does not apply to those languages.

from sigx import RephraseDetector

detector = RephraseDetector(
    similarity_threshold=0.6,   # Cosine similarity threshold
    min_turn_length=20,         # Skip user turns shorter than this (chars)
    skip_acknowledgments=True,  # Skip "thanks", "ok", etc.
)

SentimentDetector

Hybrid regex + ML detection with 41 built-in patterns for corrections, negatives, positives, and sarcasm.

from sigx import SentimentDetector

# Rule-only mode (default, no extra deps)
detector = SentimentDetector(min_confidence=0.6)

# ML-enhanced mode (broader coverage)
detector = SentimentDetector(use_ml=True, min_confidence=0.5)
detector.fit(
    texts=["that's wrong", "thanks!", "actually I meant..."],
    labels=["negative", "positive", "correction"],
)

AbandonDetector

Detects when users give up via explicit patterns or conversation structure analysis.

from sigx import AbandonDetector

detector = AbandonDetector(
    min_assistant_length=300,           # Min chars for trailing-assistant
    min_turns=3,                        # Min conversation length
    require_unanswered_question=True,   # Only flag if user asked a question
)

LLMExtractor

High-accuracy classification using any OpenAI-compatible LLM.

from sigx import LLMExtractor

detector = LLMExtractor(
    model="gpt-4o-mini",            # Or "qwen2.5-7b-instruct" via vLLM
    base_url=None,                  # Defaults to OpenAI; set for Ollama/vLLM
    api_key="sk-...",               # Or set OPENAI_API_KEY env var
    min_confidence=0.6,
)

Custom Extractors

from sigx.extractors import BaseExtractor
from sigx.types import Signal

class ToxicityExtractor(BaseExtractor):
    name = "toxicity"

    def extract(self, conversation: dict) -> list[Signal]:
        signals = []
        # Your detection logic here
        return signals

pipeline = Pipeline([ToxicityExtractor()])

DPO Chosen Strategies

A key innovation: SigX automatically infers the chosen response from subsequent conversation turns.

Strategy	Behavior	Use Case
`subsequent` (default)	Find positive turn after negative signal; use assistant response before it as `chosen`	Best quality
`last_assistant`	Use the final assistant response as `chosen`	Simple fallback
`none`	`chosen=None`	Backward compatible

from sigx import Pipeline, CHOSEN_SUBSEQUENT, CHOSEN_NONE

# Default: smart chosen inference
pipeline = Pipeline(extractors=[...], chosen_strategy=CHOSEN_SUBSEQUENT)

# Backward compatible: no chosen inference
pipeline = Pipeline(extractors=[...], chosen_strategy=CHOSEN_NONE)

Data Loading

SigX supports multiple conversation formats:

from sigx import load_conversations, load_wildchat, stream_wildchat

# ShareGPT format
convos = load_conversations("sharegpt_data.json", format="sharegpt")

# OpenAI chat format
convos = load_conversations("openai_data.json", format="openai")

# Generic JSONL
convos = load_conversations("logs.jsonl", format="jsonl")

# WildChat from HuggingFace (1M+ conversations)
convos = load_wildchat(n=1000)

# Stream large datasets (memory efficient)
for convo in stream_wildchat(n=10000):
    signals = pipeline.run([convo])

Output Formats

DPO (Direct Preference Optimization)

pairs = pipeline.to_dpo(conversations)
# List[PreferencePair] — compatible with TRL's DPOTrainer

KTO (Kahneman-Tversky Optimization)

examples = pipeline.to_kto(conversations)
# List[KTOExample] — binary labels: True (desirable), False (undesirable)

Rejection Sampling

pairs = pipeline.to_rejection(conversations)
# List[dict] with keys: prompt, rejected, signal_type, confidence

End-to-End Training Pipeline

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#e3f2fd', 'primaryTextColor': '#1565c0', 'primaryBorderColor': '#1565c0', 'lineColor': '#90a4ae', 'secondaryColor': '#fff3e0', 'tertiaryColor': '#e8f5e9'}}}%%

graph LR
    subgraph INPUT [" "]
        direction TB
        I1["JSONL"]
        I2["ShareGPT"]
        I3["WildChat"]
        I4["OpenAI"]
    end

    subgraph SIGX ["SigX"]
        direction LR
        S1["Extract"] --> S2["Filter"] --> S3["Convert"]
    end

    subgraph OUTPUT [" "]
        direction TB
        O1["DPO Pairs"]
        O2["KTO Examples"]
    end

    INPUT --> SIGX --> OUTPUT

    style INPUT fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#0d47a1
    style SIGX fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#bf360c
    style OUTPUT fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#1b5e20

from sigx import Pipeline, SentimentDetector, RephraseDetector, AbandonDetector
from sigx.io import load_wildchat

# 1. Configure pipeline
pipeline = Pipeline([
    SentimentDetector(min_confidence=0.6),
    RephraseDetector(similarity_threshold=0.6),
    AbandonDetector(min_turns=3),
])

# 2. Load conversation data
convos = load_wildchat(n=10000)

# 3. Convert to DPO training pairs
dpo_pairs = pipeline.to_dpo(convos)

# 4. Feed directly to TRL
from trl import DPOTrainer
trainer = DPOTrainer(model=model, train_dataset=dpo_pairs)
trainer.train()

Evaluation

Evaluate extraction quality against a labeled benchmark:

metrics = pipeline.evaluate("benchmark.json")

print(f"Overall F1: {metrics['overall']['f1']:.4f}")
# Per-type breakdown
for signal_type, m in metrics["per_type"].items():
    print(f"  {signal_type}: P={m['precision']:.4f} R={m['recall']:.4f} F1={m['f1']:.4f}")

Benchmark Results

Evaluated on a 20-conversation benchmark with regex-based extractors (default settings, no LLM):

Reproduce with pipeline.evaluate("tests/benchmark.json"); regenerate the chart with python scripts/generate_assets.py. Adding LLMExtractor significantly improves recall on negative and rephrase types.

Architecture

sigx/
├── pipeline.py          # Orchestration: extract → filter → convert
├── types.py             # Core dataclasses: Signal, PreferencePair, KTOExample
├── exceptions.py        # Custom exception hierarchy
├── extractors/
│   ├── base.py          # Abstract BaseExtractor
│   ├── rephrase.py      # TF-IDF cosine similarity rephrase detection
│   ├── sentiment.py     # Regex + optional ML sentiment classifier
│   ├── abandon.py       # Frustration pattern + trailing-assistant detection
│   └── llm.py           # OpenAI-compatible LLM classifier
├── filters/
│   └── quality.py       # Confidence threshold, dedup, per-conv limits
├── converters/
│   └── preference.py    # DPO / KTO / rejection-sampling converters
└── io/
    └── loader.py        # ShareGPT, OpenAI, WildChat, JSONL loaders

Research Background

SigX is built on growing evidence that implicit feedback from real user interactions produces better alignment data than synthetic alternatives:

Paper	Institution	Year	Key Finding
WildFeedback	Microsoft Research	2024	20K preference pairs from 148K ChatGPT conversations outperformed UltraFeedback on AlpacaEval 2, Arena-Hard, MT-Bench

The key insight: Real user feedback captures nuance that synthetic data misses, and the signals are already present in existing chat logs — they just need to be extracted.

How SigX Relates

Project	Focus	SigX Connection
WildFeedback	Extracting preference pairs from in-situ feedback	SigX implements the extraction pipeline in Python
TRL	Post-training (DPO/KTO/PPO)	SigX output is TRL-compatible
VeRL	RLHF framework	SigX output works with VeRL

Development

git clone https://github.com/fengrru/sigx.git
cd sigx
pip install -e ".[dev]"

# Run tests
pytest                    # 92 tests
pytest --cov=sigx         # With coverage

# Lint
ruff check .              # Lint
ruff format .             # Format

# Type check
mypy sigx/                # mypy
pyright                   # pyright

Citation

@software{sigx2026,
  title   = {{SigX: Implicit Feedback Signal Extraction for LLM Alignment}},
  author  = {fengrru},
  license = {MIT},
  url     = {https://github.com/fengrru/sigx},
  year    = {2026},
}

License

MIT License — see LICENSE for details.

Built with care for the LLM alignment community.

Report Bug · Request Feature · Contributing Guide

Name		Name	Last commit message	Last commit date
Latest commit History 19 Commits
.github		.github
docs		docs
examples		examples
scripts		scripts
sigx		sigx
tests		tests
.gitattributes		.gitattributes
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation