This project contains code for training and evaluating and inferring lightweight models for diacritics restoration. This repository accompanies our paper.
Diacritics restoration has become a ubiquitous task in the Latin-alphabet-based English-dominated Internet language environment. In this repository, we provide a model built with small footprint 1D convolution-based approach, which works on character-level. We find that solutions based on 1D dilated convolutional neural networks are competitive alternatives to models based on recursive neural networks or linguistic modeling for the task of diacritics restoration. Our solution surpasses the performance of similarly sized models and is also competitive with larger models. A special feature of our solution is that it even runs locally in a web browser. We also provide a working example of this browser-based implementation. Our model is evaluated on different corpora, with emphasis on the Hungarian language. We performed comparative measurements about the generalization power of the model in relation to three Hungarian corpora. We also analyzed the errors to understand the limitation of corpus-based self-supervised training.
If you want to try out the model, the demo is available here.
For training the model:
The project logs both locally and to neptune.ai,
to use a neptune.ai an account is neeeded. Logging to neptune can be disabled for individual experiments in the experiment's config, or globally by not providing an api token in the neptune_cfg.yaml.
Copy neptune_cfg_template.yaml to neptune_cfg.yaml, and fill out the appropriate details:
project_qualified_name:
api_token:
offline_logging_dir: Refer to the example below and install the missing packages manually,
or use the environment.yml file: conda env create -f environment.yml.
The following command should work with the small example data provided in this repository.
CUDA_VISIBLE_DEVICES=0,1 python run.py -c conf/example.yaml