bertfinetuner

Fine-tuning harness for ModernBERT-TR encoders. Each task is one TaskSpec; the backbone, training loop, metrics, and export are shared across all tasks.

Setup

uv sync                                  # CPU (laptop)
uv sync --no-group cpu --group cu126     # CUDA 12.6 GPU

Train and evaluate

uv run bertfinetuner tasks
uv run bertfinetuner train --task massive-intent --profile cpu --max-samples 64 --epochs 1
uv run bertfinetuner train --task massive-joint --profile h100 --seed 42
uv run bertfinetuner eval  --task massive-joint --checkpoint OUT [--fold-ascii] [--calibrate]
uv run bertfinetuner export-head --checkpoint OUT --head intent --output-dir EXPORT

train writes run.json, containing config plus validation and test metrics, to the output dir. export-head materializes one head of a multi-head checkpoint.

Options

--profile (cpu/rtx/h100) changes throughput.

Optional args: --learning-rate, --epochs, --batch-size, --head-lr, --warmup, --weight-decay, --lr-scheduler, --label-smoothing, --pooling (cls/mean), --layer-lr-decay, --ascii-fold-p.

For imbalanced tasks: --class-weight-beta (1 is full inverse-frequency, 0 is unweighted) and --protect-beta with --protect-labels to keep chosen rare classes weighted.

Modal

uv run modal run scripts/run_modal.py::run    --task massive-joint
uv run modal run scripts/run_modal.py::sweep  --task massive-joint --lrs 3e-5,5e-5 --seeds 42,43,44
uv run modal run scripts/run_modal.py::export --checkpoint outputs/RUN_DIR

sweep runs jobs in parallel across containers, capped by your Modal GPU quota. scripts/select.py ranks run.json results across seeds.

Layout

• spec.py: TaskSpec and Head, the only thing a new task defines.
• model.py, multihead.py: stock AutoModelFor* for single-head tasks; a shared-encoder multi-head model that exports each head as a plain checkpoint.
• data.py: tokenization, first-subword label alignment, Turkish casefold, ASCII-fold augmentation, collators.
• args.py: optimization defaults plus cpu/rtx/h100 profiles.
• trainer.py: class-weight and pos-weight losses, optional encoder/head LR split.
• metrics.py, calibration.py: per-head metrics, exact match, temperature scaling, and ECE.
• tasks/: registered tasks.