Detecting Diachronic Meaning Shifts in Dutch
This is the source code for the work presented in the CLIN talk ''Can we spot meaning shifts in diachronic representations?'' (abstract | slides). A paper based on the experiments presented in this talk, ''A Critical Assessment of a Method for Detecting Diachronic Meaning Shifts: Lessons Learnt from Experiments on Dutch'' (pdf), is published in the CLIN Journal (pdf).
You can use this code to create diachronic meaning representations based on any significantly large diachronic corpus of Dutch, and investigate and analyse those representations to detect known meaning shifts.
To run this code, you'll need:
- Python 2.7.6
- gensim 0.13.2
- scipy 0.18.1
- matplotlib 2.0.0
Different versions might work just as well, but cannot be guaranteed. Most scripts also expect there to be a folder called
working in the main code directory.
In addition, we made use of:
most_frequent_*_5000.txt: Lists of the 5000 most frequent words/nouns/verbs in LASSY
most_frequent_words_5000_average_similarity.json: Average similarity in the Trouw and Volkskrant corpora for the 5000 most frequent words in LASSY
word_list_combined.txt: Initial list of words to investigate (before frequency cut-off)
word_list_combined_results.json: Counts, frequencies and self-similarities in both corpora for the words in the list above
plots/*.png: Plots of the words discussed in the paper and the talk, as generated by the script
E-mail me at firstname.lastname@example.org