ABA

Alignment-Based Approach for automatic modernization of french texts from the 17th to the 18th century

Online demo at https://igm.univ-mlv.fr/~gambette/text-processing/aba/

Install

Install Packages

• With make

make

• Without make

Add an extra line with ASR_metrics in the end of the file requirements.txt if you want to use the evaluation metrics

pip install -r requirements.txt

Generate Data

Download, Align and Analyze PARALLEL17

1. Download PARALLEL17 and put it into the download folder or run script

python -m aba.download_git 'https://github.com/PhilippeGambette/PARALLEL17.git'

2. Align PARALLEL17 by words

python -m aba.align_words

3. Extract dictionaries from PARALLEL17

python -m aba.analyze

Extract Morphalou Dictionary

1. Download Morphalou
2. Copy morphalou/4/Morphalou3.1_formatCSV_toutEnUn/Morphalou3.1_CSV.csv to download folder
3. Run script

python -m aba.extract_dic_morphalou

Extract Wikisource Dictionary

Extract old french → modern french dictionary from Wikisource.

python -m aba.extract_dic_wikisource

Extract Name Dictionary

Extract dictionary from multiple .dic files located in resources folder.

python -m aba.extract_dic_resources

Main Scripts

Modernize Corpus

python -m aba.modernize_corpus

Modernize Text

Modernize a text in old French. ¹

python -m aba.modernize [-h] text_old_path

Modernize Text and Evaluate It

Modernize a text in old French and evaluate it by comparing it with a reference version stored in a file TEXT_NEW_PATH

python -m aba.modernize_and_evaluate [-h] -n TEXT_NEW_PATH text_old_path

Tools

Rules Chart

Opens a labeled dictionary and displays an interactive plotly pie chart showing the frequence of modernization rules. A copy of the chart is saved in data/rules_chart.html.

python -m aba.rules_chart

Find Strings

Search 2-columns .tsv files in a given directory for two corresponding strings old and new. Prints files, rows and lines where both strings appear.

python -m aba.find_strings [-h] [-d DIRECTORY] old new

Run Tests

py.test

Footnotes

1. Path arborescence must be written with forward slashes /. ↩