This section examines coverage and performance across task types in SEB. The task types categories are derived from the MTEB benchmark.
The table show the performance across task types:
<iframe title="Task type SEB" aria-label="Table" id="datawrapper-chart-4jkip" src="https://datawrapper.dwcdn.net/4jkip/6/" scrolling="no" frameborder="0" style="width: 0; min-width: 100% !important; border: none;" height="1160" data-external="1"></iframe><script type="text/javascript">!function(){"use strict";window.addEventListener("message",(function(a){if(void 0!==a.data["datawrapper-height"]){var e=document.querySelectorAll("iframe");for(var t in a.data["datawrapper-height"])for(var r=0;rThe follows table give you and an overview of the coverage of the tasks:
Across | · Danish | Norwegian Bokmål | Norwegian Nynorsk | Swedish | ||
---|---|---|---|---|---|---|
Formalization | Task | |||||
Retrieval | Question answering | ✓ | ✓ | ✓ | ✓ | |
article retrieval | ✓ | ✓ | ✓ | ✓ | ||
bitext Mining | dialect pairing | ✓ | ✓ | ✓ | ✓ | |
Classification | Political | ✓ | ✓ | ✓ | ||
Language Identification | ✓ | ✓ | ✓ | ✓ | ✓ | |
Linguistic Acceptability | ✓ | ✓ | ✓ | ✓ | ✓ | |
Sentiment/Hate Speech | ✓ | ✓ | ✓ | ✓ | ||
Dialog Systems | ✓ | ✓ | ✓ | ✓ | ✓ | |
Clustering | Thematic Clustering | ✓ | ✓ | ✓ | ||
Reranking | ||||||
Pair Classification | ||||||
STS |