Repository navigation
FediAID artifact update: reproducibility improvements and five-seed MMR sensitivity
Pre-release
Pre-release
This release addresses artifact-review feedback and extends the optional MMR sensitivity analysis.
- Reproduction environment: pinned the validated core runtime and tokenizer dependencies and the DeBERTa encoder revision; added SentencePiece and the protobuf version constraint.
- Stricter verification: both evaluation workflows now reject missing references, incomplete coverage, duplicate results, non-finite metrics, and invalid tolerances.
- Clearer evaluation instructions: documented a complete raw-text reproduction route, aligned raw and embedded dataset names for reference comparison, and corrected command examples.
- Per-seed references: added expected F1/AUC values for all 60 dataset/seed combinations in the main evaluation.
- Five-seed sensitivity analysis: extended the optional sweep from seed 0 to seeds 0–4. Each platform is averaged over five seeds before computing the mean and population standard deviation across platforms.
- Documentation and licensing: documented the released checkpoint’s training data, split construction, settings, and training command; added an MIT source-code license.
The released checkpoint, memory bank, main expected metrics, and external evaluation configuration are unchanged.