KoPolitic-Benchmark-Dataset

A New Korean Text Classification Benchmark for Recognizing the Politic Intents in Online Newspapers

• This repository provides KoPolitic Dataset and PyTorch implementations for classification models
  (KoPolitic, KoBigBird, KoBERT, KoELECTRA).

Authors

• Beomjune Kim, Eunsun Lee, Dongbin Na

Abstract

Many users reading online articles in various magazines may suffer considerable difficulty in distinguishing the implicit intents in texts. In this work, we focus on automatically recognizing the political intents of a given online newspaper by understanding the intent of the text. To solve this task, we present a novel Korean text classification dataset that contains various articles. We also provide deep-learning-based text classification baseline models trained on the proposed dataset. Our dataset contains 12,000 news articles that may contain political intentions, from the \textit{politics} section of six of the most representative newspaper organizations in South Korea. All the text samples are labeled simultaneously in two aspects (1) the level of political orientation and (2) the level of pro-government. To the best of our knowledge, our paper is the most large-scale Korean news dataset that contains long text and addresses multi-task classification problems. We also train recent state-of-the-art (SOTA) language models that are based on transformer architectures and demonstrate that the trained models show decent text classification performance. All the source codes, datasets, and trained text classification models will be publicly available after our paper is accepted.

Model

• The conceptual illustration of our model with input and output examples.

Source Codes

Model	SeqLen	Code
KoBigBird	1024	link1
KoBigBird	2048	link2
KoBigBird	3072	link3
KoPolitic	1024	link4
KoPolitic	2048	link5
KoPolitic (Ours)	3072	link6
Kobert Single	512	link7
Kobert Multi	512	link8
KoElectra Single	512	link9
KoElectra Multi	512	link10

Models Performance

• The classification performance of trained model.

Dataset

The dataset consists of 12,000 news articles. Articles were crawled and collected from six leading newspapers in South Korea. This includes two newspapers classified as conservative, two as neutral, and two as liberal. Out of the 12,000 labeled articles, 5,000 of the dataset are labeled based on two criteria: the level of political orientation and the level of pro-government. The level of political orientation is rated on a 5-point scale, where a score closer to 1 indicates a liberal stance and a score closer to 5 indicates a conservative stance. The level of pro-government is rated on a 6-point scale, where 0 stands for ‘None', scores closer to 1 indicate support for the government, and scores closer to 5 indicate criticism of the government.

Download Dataset

Citation

If this work can be useful for your research, please cite our paper:

@misc{kim2023new, 
      title={A New Korean Text Classification Benchmark for Recognizing the Political Intents in Online Newspapers}, 
      author={Beomjune Kim and Eunsun Lee and Dongbin Na},
      year={2023},
      eprint={2311.01712},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}