ko-mteb

MTEB for korean

Task 정리

Classification

KB-SentiNeg

• Predict the polarity of a negated sentence.

KLUE-YNAT

• The text discusses topic classification (TC) as the prediction of topics in text snippets,
  the inclusion of TC in the KLUE benchmark, the introduction of YNAT as a dataset for Korean TC,
  and the evaluation metric of Macro-F1 score used for assessing system performance.,

PairClassification

KB-BoolQ

• identify whether a given question is true or false considering a paragraph.

• kobest-wic 가능한가?

STS

Retrieval

할 것.

• 바로 추가할 수 있는 task 추가하기.
• 분석 결과 쉽게 aggregation할 수 있는 함수 만들기.
• openai와 같이 외부 API를 활용하는 경우에도 사용가능한 코드 만들기
• dev/BM-K/KoMiniLM