Skip to content

HIPE-2026 Person-Place Relation Data - v1.0 - Competition

Latest
Latest

Choose a tag to compare

View all tags
@e-maud e-maud released this 09 Jun 17:34
v1.0
5f4c2db

HIPE-2026 Data — v1.0

HIPE-2026 is a CLEF 2026 Evaluation Lab on the qualification of person–place relations in multilingual historical documents (Who was where, when?). This release contains the complete v1.0 dataset used during the official evaluation campaign.

DOI

📦 What's in this release

The HIPE-2026 dataset covers two evaluation domains across two test sets:

  • Domain A — Historical Newspapers (Test A): Articles in German, English, and French from the HIPE historical newspaper corpus (derived from HIPE-2022 impresso dataset). Entity annotations and Wikidata links were manually created in HIPE-2022. Person-Places relations were manually annotated for HIPE-2026. Evaluates both at and isAt relations.
  • Domain B — Literary Works (Test B, surprise): A held-out test set of French literature and history works from the 16th–18th centuries, included to assess out-of-domain generalization. Evaluates at only.
Split Domain Languages File(s)
Train Newspapers DE, EN, FR data/newspapers/v1.0/HIPE-2026-v1.0-impresso-train-*.jsonl
Test Newspapers DE, EN, FR data/newspapers/v1.0/HIPE-2026-v1.0-impresso-test-*.jsonl
Test (masked) Newspapers DE, EN, FR data/newspapers/v1.0/HIPE-2026-v1.0-impresso-test_masked-*.jsonl
Test Literary works FR data/litworks/v1.0/HIPE-2026-v1.0-surprise-test-fr.jsonl
Test (masked) Literary works FR data/litworks/v1.0/HIPE-2026-v1.0-surprise-test_masked-fr.jsonl

For detailed information on tasks, datasets, and evaluation protocol, refer to the CLEF HIPE-2026 Shared Task Participation Guidelines.

🔬 Reproducing the official evaluation

The full campaign evaluation — including reference data, participant submissions, and the evaluation orchestrator — is available at:
👉 https://github.com/hipe-eval/hipe-2026-eval

📐 Data format

Data is distributed as UTF-8 JSON Lines (.jsonl). Each line is one document with OCR text, document metadata, and sampled person–location pairs. Prediction targets are:

  • at — evidence that a person was at a location at any time before publication (TRUE, FALSE, PROBABLE, null)
  • isAt — evidence of presence within ~one month before publication (TRUE, FALSE, null)

The full schema is in schemas/hipe-2026-data.schema.json. See the README for format details, validation, and a prediction/evaluation walkthrough.

📖 How to cite

[Condensed Overview] Juri Opitz, Maud Ehrmann, Corina Raclé, Andrianos Michail, Matteo Romanello, Emanuela Boros, Simon Gabay, and Simon Clematide. 2026. Extended Overview of HIPE-2026: Evaluating Accurate and Efficient Person–Place Relation Extraction from Multilingual Historical Texts. In CLEF 2026 working notes, CEUR workshop proceedings, 2026. CEUR-WS. https://doi.org/10.5281/zenodo.20344461

@inproceedings{opitz_extended_2026,
  title = {Extended {{Overview}} of {{HIPE-2026}}: {{Evaluating Accurate}} and {{Efficient Person}}--{{Place Relation Extraction}} from {{Multilingual Historical Texts}}},
  booktitle = {{{CLEF}} 2026 Working Notes, {{CEUR}} Workshop Proceedings},
  author = {Opitz, Juri and Ehrmann, Maud and Racl{\'e}, Corina and Michail, Andrianos and Romanello, Matteo and Boros, Emanuela and Gabay, Simon and Clematide, Simon},
  editor = {S{\'a}nchez Salido, Eva and {Barr{\'o}n-Cede{\~n}o}, Alberto and {Garc{\'i}a Seco de Herrera}, Alba and MacAvaney, Sean and Stru{\ss}, Julia Maria},
  year = 2026,
  publisher = {CEUR-WS},
  doi = {10.5281/zenodo.20344461}
}

[Extended Overview] Juri Opitz, Maud Ehrmann, Corina Raclé, Andrianos Michail, Matteo Romanello, and Simon Clematide. 2026. Overview of HIPE-2026: Person–Place Relation Extraction from Multilingual Historical Texts. In Experimental IR meets multilinguality, multimodality, and interaction. Proceedings of the seventeenth international conference of the CLEF association (CLEF 2026) (Lecture notes in computer science (LNCS)), 2026. Springer.

@inproceedings{opitz_overview_2026,
  title = {Overview of {{HIPE-2026}}: {{Person}}--{{Place Relation Extraction}} from {{Multilingual Historical Texts}}},
  booktitle = {Experimental {{IR}} Meets Multilinguality, Multimodality, and Interaction. {{Proceedings}} of the Seventeenth International Conference of the {{CLEF}} Association ({{CLEF}} 2026)},
  author = {Opitz, Juri and Ehrmann, Maud and Racl{\'e}, Corina and Michail, Andrianos and Romanello, Matteo and Clematide, Simon},
  editor = {Hagen, Matthias and Potthast, Martin and Stein, Benno and Schaer, Philipp and Zangerle, Eva and MacAvaney, Sean and Stru{\ss}, Julia Maria and S{\'a}nchez Salido, Eva and {Barr{\'o}n-Cede{\~n}o}, Alberto and {Garc{\'i}a Seco de Herrera}, Alba},
  year = 2026,
  series = {Lecture Notes in Computer Science ({{LNCS}})},
  publisher = {Springer}
}

📜 License

Released under CC BY-NC-SA 4.0.

Update

3.07.2026: corrected HIPE-2026-v1.0-surprise-test-fr.jsonl, which had been inadvertently replaced with a later adjudication pass; restored to the version used to compute the officially reported Test B results. See commit 5f4c2db.

HIPE-2026 is organised within the Impresso project, funded by the Swiss National Science Foundation (grant CRSII5_213585) and the Luxembourg National Research Fund (grant 17498891).

Assets 2
Loading