Skip to content
Implementation of Shen et al.'s "The Role of Context in Neural Morphological Disambiguation"
Branch: master
Clone or download
onurgu Dockerfile small change
README updated
Latest commit b6340b2 Nov 11, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data Training and test data are uploaded. Jun 26, 2017
public_html Adding screenshot image Aug 3, 2017
scripts
.dockerignore Dockerfile Nov 11, 2017
.gitignore Adding the .idea directory to .gitignore Mar 20, 2017
Dockerfile
README.md Dockerfile small change Nov 11, 2017
corpus-statistics.md Corpus statistics: added a module to calculate some statistics about … Aug 3, 2017
loader.py
maskedreshape.py We have a functioning replica of the model. The results are in the RE… Jun 26, 2017
model.py
requirements.txt Dockerfile Nov 11, 2017
sample.data Adds the code for reading input data Mar 20, 2017
train.py Corpus statistics: added a module to calculate some statistics about … Aug 3, 2017
utils.py WIP: added a GUI and introduced in the README Aug 3, 2017
webapp.py Dockerfile Nov 11, 2017

README.md

neural-turkish-morphological-disambiguator

Implementation of Shen et al.'s "The Role of Context in Neural Morphological Disambiguation"

Training

python train.py --command train --train_filepath data/train.merge.utf8 --test_filepath data/test.merge.utf8 --run_name nmd-20170619-06

Prediction

python train.py --command predict --train_filepath data/train.merge.utf8 --test_filepath data/test.merge.utf8 --model_path ./models/ntd-nmd-20170619-06.epoch-32-val_acc-0.99507.hdf5 --run_name testing

Disambiguating a sentence

python train.py --command disambiguate --train_filepath data/train.merge.utf8 --test_filepath data/test.merge.utf8 --model_path ./models/ntd-nmd-20170619-06.epoch-32-val_acc-0.99507.hdf5 --label2ids_path ./models/ntd-nmd-20170619-06.epoch-32-val_acc-0.99507.hdf5.label2ids --run_name testing

Results

Evaluation finished, batch_id: 42
only the filled part of the sentence
868
889
0.976377952756
all the sentence
2817
2838
0.992600422833
===
ambigous
369
390
0.946153846154
===
disambiguations out of n_analyses: 2 ===> 1.000000 219 219
disambiguations out of n_analyses: 3 ===> 0.916667 55 60
disambiguations out of n_analyses: 4 ===> 0.853333 64 75
disambiguations out of n_analyses: 5 ===> 0.882353 15 17
disambiguations out of n_analyses: 6 ===> 0.888889 8 9
disambiguations out of n_analyses: 8 ===> 0.777778 7 9
disambiguations out of n_analyses: 12 ===> 1.000000 1 1

Web demo

python webapp.py --command disambiguate --train_filepath data/train.merge.utf8 --test_filepath data/test.merge.utf8 --model_path ./models/ntd-nmd-20170619-06.epoch-32-val

Example screenshot:

Desktop app

WIP: Implemented a very simple and ugly GUI to get morphological analyzes of a given sentence

python utils.py --command gui --output_dir stats_train_merge --gold_data 1 --verbose 1

Docker instance

For easier installation, I have created a Dockerfile which can be used to run all functions easily. To use

docker build -t my_neural_turkish_disambiguator:0.0.1 .

Run the following to have a shell where you can run the training, prediction, disambiguation and webapp features. You can just use volumes to persist created files. Please see the releases page for the model files.

docker run -ti -p 10001:10001 -v neural_data_volume:/opt/neural-turkish-disambiguator my_neural_turkish_disambiguator:0.0.1
You can’t perform that action at this time.