Benchmark Analysis

ELEVANT provides not only statistics about entity linker results, but also about entity linking benchmarks.

ELEVANT provides statistics about the number of words, sentences, articles and ground truth labels in the benchmark, as well as detailed statistics about the mention types, entity types, and number of words, etc. of the ground truth labels.

In the web app, you can switch to the benchmark analysis by clicking on "Switch to Benchmark Analysis" in the page header.

To create statistics for a benchmark, run the following command:

python3 scripts/analyze_benchmark.py -b <benchmark_name>

where <benchmark_name> is the ELEVANT name of the benchmark, e.g., kore50. The computed statistics are written to benchmarks/<benchmark_name>.benchmark_statistics.jsonl and read by the evaluation web app. You can create statistics for all benchmarks listed in the benchmarks directory at once by using ALL as benchmark name.