Detection of diachronic meaning shifts in Dutch
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
plots
LICENSE
README.md
analyze_and_plot_words.py
compare_diachronic_embeddings.py
corpus_stats.txt
get_average_similarity.py
most_frequent_nouns_5000.txt
most_frequent_verbs_5000.txt
most_frequent_words_5000.txt
most_frequent_words_5000_average_similarity.json
query_words_in_embeddings.py
scrape_news.py
train_diachronic_embeddings.py
word_list_combined.txt
word_list_combined_results.json

README.md

Detecting Diachronic Meaning Shifts in Dutch

This is the source code for the work presented in the CLIN talk ''Can we spot meaning shifts in diachronic representations?'' (abstract | slides). A paper based on the experiments presented in this talk, ''A Critical Assessment of a Method for Detecting Diachronic Meaning Shifts: Lessons Learnt from Experiments on Dutch'' (pdf), is published in the CLIN Journal (pdf).

You can use this code to create diachronic meaning representations based on any significantly large diachronic corpus of Dutch, and investigate and analyse those representations to detect known meaning shifts.

Requirements

To run this code, you'll need:

  • Python 2.7.6
  • gensim 0.13.2
  • scipy 0.18.1
  • matplotlib 2.0.0

Different versions might work just as well, but cannot be guaranteed. Most scripts also expect there to be a folder called working in the main code directory.

In addition, we made use of:

Additional Files

  • most_frequent_*_5000.txt: Lists of the 5000 most frequent words/nouns/verbs in LASSY
  • most_frequent_words_5000_average_similarity.json: Average similarity in the Trouw and Volkskrant corpora for the 5000 most frequent words in LASSY
  • word_list_combined.txt: Initial list of words to investigate (before frequency cut-off)
  • word_list_combined_results.json: Counts, frequencies and self-similarities in both corpora for the words in the list above
  • plots/*.png: Plots of the words discussed in the paper and the talk, as generated by the script analyze_and_plot_words.py.

Contact

E-mail me at hessel.haagsma@rug.nl