Skip to content

ChiEngMixBench v0.2.0 — Audited benchmark and reproducibility release

Latest
Latest

Choose a tag to compare

View all tags
@LingLing0117 LingLing0117 released this 04 Aug 08:26
v0.2.0
e4ac79a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This release aligns the public artifacts with the substantially revised ChiEngMixBench manuscript and replaces the earlier broad framing with bounded, auditable paired and open-ended diagnostics of Chinese-English AI/CS terminology choice.

Highlights

  • Audited all 1,706 source-derived candidate pairs covering 1,344 non-empty normalized target-term types.
  • Materialized a 1,167-pair strict primary set that holds the Chinese prefix and syntactic position fixed while varying the terminology form.
  • Added a 1,289-pair localized-edit sensitivity set.
  • Repaired two missing target-term fields and excluded one duplicated strict item from strict-primary inference.
  • Recomputed paired-preference summaries and added frequency, length, and multiple-comparison controls.
  • Reconciled EDP item-level clipping and documented the canonical scoring rules, coefficients, thresholds, and aliases.
  • Added an English-ratio baseline, a pure-Chinese negative control, a bounded temporal audit, and implementation checks.
  • Released anonymized results and protocol summaries for the 100-item Likert study and 77-pair A/B study.
  • Added explicit data, provenance, licensing, privacy, schema, and reproducibility documentation.
  • Removed raw crawled pages, source URLs, timestamps, account identifiers, and workbook metadata from the public release.

Interpretation boundary

The benchmark evaluates source-attested Chinese-English technical terminology preferences. Reference-profile conformity is informative under the intended mixed-style rubric, but it is not a general measure of naturalness, cognitive alignment, or holistic response quality. The release does not support document-disjoint or author-disjoint evaluation because reliable upstream grouping identifiers are unavailable for the released items.

Compatibility notes

  • Use the 1,167-pair strict set for primary matched inference.
  • Use the full 1,706-pair set and the 1,289-pair localized-edit set for explicitly labeled sensitivity analyses.
  • The canonical entry points are metric_paired_preference.py and metric_edp.py.
  • Historical CSG names are retained only where needed for compatibility and are documented as aliases.
  • Code is licensed under Apache-2.0. Data and source-derived text are governed by the scoped terms in LICENSE_DATA.md and PROVENANCE_AND_LICENSE.md; this release does not purport to relicense third-party material beyond rights held by the authors.

Verification

The frozen distribution archive is:

  • File: ChiEngMixBench_Public_Release_v0.2.0.zip
  • Size: 2,196,173 bytes
  • SHA-256: D515BCCEE7AF2852FD64D52E5D6669800661822A17F1E01D8816BE6A3F278D28

The archive contains one top-level directory, ChiEngMixBench-Dataset-v0.2.0, with 100 files. The included manifest covers 99 release files, and validation on a fresh extraction completed successfully with 29,858 checks.

See CHANGELOG.md, README.md, DATA_CARD.md, DATA_SCHEMA.md, DATA_STATEMENT.md, REPRODUCIBILITY.md, and PROVENANCE_AND_LICENSE.md for details.

Assets 3
Loading