Learnability and Semantic Universals
This repository accompanies the following paper:
- Steinert-Threlkeld and Szymanik 2018, "Learnability and Semantic Universals", in press at Semantics & Pragmatics. https://semanticsarchive.net/Archive/mQ2Y2Y2Z/LearnabilitySemanticUniversals.pdf
In particular, we attempt to explain semantic universals surrounding quantifiers by training a recurrent neural network (a long-short term memory network specifically) to learn quantifiers, some satisfying proposed universals and some not. Generally, we find that those satisfying universals are easier to learn by this model than those that do not.
This repository contains all the code needed to replicate the experiments reported in that paper. It also contains the data and figures from the experiments that we ran and which are reported therein.
If you have any questions and/or want to extend the code-base and/or run your own experiments, feel free to get in touch!
Python 2.7+, TensorFlow 1.4+, Pandas
[NB: the code should be compatible with Python 3, but has not been tested with it.]
Any of the experiments can be re-run very simply from the command-line. For example, to run experiment 1(a):
python quant_verify.py --exp one_a --out_path /tmp/exp1a/
Note that the output written to
out_path will also contain checkpoints and other data from TensorFlow in addition to CSV files recording the relevant information from each trial.
To analyze the output from an experiment, use the
analysis.experiment_analysis method. It takes two arguments: a path to a directory containing the CSV files output by a run of an experiment, and an array containing the names of quantifiers in the experiment.
To re-create the analysis from the paper and the figures included therein, you can use the predefined methods. For example, in a Python shell:
import analysis analysis.experiment_one_a_analysis()
Designing and Running New Experiments
The code is designed to make it easy to design and run new experiments. You can look at any of the
experiment_ methods in
quant_verify.py for inspiration. I recommend copy/pasting one of these methods, renaming it, updating the parameters to what you would like, and then running it! The next section tells you how to make new quantifiers if you want to test some not included.
Defining New Quantifiers
All of the quantifiers are defined in
quantifiers.py. The core is a verification method, named
_ver(seq). This takes in a sequence of characters (named
Quantifier.AnotB, et cetera) and outputs one of the truth values
Quantifier.F. This method implements the core semantics of a quantifier.
The verification method is then wrapped inside a
Quantifier object, which provides a name for the quantifier and assigns it certain semantic properties. Note that the name used here must match the names passed in as strings to the
Again, I recommend using an example from
quantifiers.py as your starting point for defining a new quantifier.