Skip to content

v0.9.0 — NLP Pipeline Complete

Choose a tag to compare

View all tags
@VforVitorio VforVitorio released this 17 Mar 10:27
· 851 commits to main since this release
v0.9.0
d072e75

All NLP notebooks (N17–N24) are complete. The radio analysis pipeline is fully operational
and integrated into the unified inference entry point used by the Strategy Agent.

New notebooks

N17 — Radio Labeling

  • 659 F1 team radio messages manually labeled (intent + sentiment + F1 entities)
  • 49 post-race messages removed after manual inspection; 610 clean samples

N18 — Whisper Transcription

  • Whisper turbo ASR pipeline for raw radio audio → text

N19 — VADER Sentiment Baseline

  • Rule-based VADER baseline established as benchmark for N20

N20 — RoBERTa Sentiment

  • RoBERTa-base fine-tuned on F1 radio dataset
  • 3 classes: negative / neutral / positive
  • Export: bert_sentiment_v1/best_roberta_sentiment_model.pt

N21 — Intent Classification (SetFit + ModernBERT)

  • SetFit + ModernBERT-base; 370 training examples
  • 5 classes: INFORMATION / PROBLEM / ORDER / WARNING / QUESTION
  • SetFit chosen over DeBERTa-large (insufficient data for full fine-tuning)
  • Export: intent_setfit_modernbert_v1/

N22 — NER Custom F1 Entities (BERT-large CoNLL-03 BIO)

  • BERT-large CoNLL-03 fine-tuned with BIO tagging; 9 F1 entity types
  • Token-level weighted F1 = 0.42 (legacy N05 = 0.43)
  • 399 annotated examples (319/40/40 split)
  • DeBERTa-v3-base tested and reverted (no NER priors → F1=0.10)
  • Synthetic augmentation tested and reverted (F1 0.42 → 0.41, too clean vs real annotations)
  • Export: ner_v1/bert_bio_v1/bert_bio_state_dict.pt

N23 — Race Control Messages Parser

  • Deterministic rule-based parser; 25 event types
  • Coverage: 100% Flag / DRS / SafetyCar, 92% Other
  • Validated on 6 races 2023–2025 (including China 2025 for retirement branch)
  • Export: rcm_parser_v1/rcm_parser_config_v1.json

N24 — Unified NLP Pipeline

  • Two entry points on a single JSON schema:
    • run_pipeline(text) → team radio (sentiment + intent + NER)
    • run_rcm_pipeline(rcm_row) → race control messages (rule-based)
  • End-to-end latency GPU: mean 47.8 ms · P95 59.4 ms (target <500 ms ✓)
  • Intent demo: 4/5 correct; WARNING→PROBLEM miss is known N21 limitation
  • Export: pipeline_config_v1.json

Model weights uploaded to HuggingFace (VforVitorio/f1-strategy-models)

  • models/nlp/bert_sentiment_v1/
  • models/nlp/intent_setfit_modernbert_v1/
  • models/nlp/ner_v1/bert_bio_v1/
  • models/nlp/pipeline_config_v1.json
  • models/nlp/rcm_parser_v1/
  • models/pit_prediction/lgbm_undercut_v1.pkl + calibrator + config (N16, missing from v0.8.1)

What's next — v1.0.0

Phase 6: Multi-Agent System (N25–N28)

  • N25 Telemetry Sub-Agent · N26 Radio Sub-Agent · N27 RAG Sub-Agent · N28 Strategy Orchestrator
Assets 2
Loading