Author verification project built to detect whether a NYT article is written by Gina Kolata.
baseline_results.txt
- Baseline results generated by synonym based single feature system. File is formatted as 0 or 1 per line for each corresponding test paragraph.
baseline_v2.txt
- Modified the synonym based single feature system to further decrease the number of false negatives. Since our validation showed that the baseline model generated few false negatives but low overall accuracy, we decided to use combine this feature with others and fully utilize its advantages of having low false negatives. We simply added a "delta" so that an article would have to score much higher from the non-Kolata training set than from the Kolata one.
census-dist-2500-last.txt
- 2500 common last names in the U.S. extracted from the census. This file will be used to build a large stop word list.
census-dist-female-first.txt
- Common female first names in the U.S. extracted from the census. Similar to the census last name file.
census-dist-male-first.txt
- Common female first names in the U.S. extracted from the census. Similar to the census last name file.
3_4_5/combined.txt -During our 3rd experiment, we attempted to perform a binary comparison between our n-gram word/character feature and the synonym feature. This was the resulting file of predictions.
evaluate.py -Use to provide statistics such as true positive, true negative, false negative, and false positive counts when passed in a list of predictions and a list of actual labels. Used to give meaningful cross-validation results
3_4_5/features.txt -As requested by Dr. Nenkova, we retrieved a list of our features from the character n-gram to see what some of the interesting results were. The first line is the list, the second line is the total number of features present.
modification_justification.txt
- This file contains statistics generated from our single feature prediction model. These statistics show how big of the differences are between the Kolata score and non Kolata score for each test paragraph. The average number helps us decide how much "delta" we should add to further lower false negatives.
single_feature.py
- This file builds our synonym based model. This model effects values words more if 1) it is repeated more, 2) it has more synonyms, and 3) the synonyms frequently occur in the larger corpus (common words that other authors use). So we build a score for every unique word in the testing set against every unique words in the training set. And add the score to get the paragraph score. At last, we compare the score generated against Kolata training set and non Kolata training set. If the difference exceeds the "delta," we have a classification!
- This file also has scripts that build a larger stop word list from nltk, census and other sources. In addition, we used nltk to build synonym finders and lemmatizers.
- At the end, we validated against a hold out set in training.
stopwords.txt
- Very similar to the file we used in class. But we decided to leave some rare symbols in case it represents the author's writing style.
3_4_5/test.txt -Purely results from running our trigrams.py file, this is the predictions formed using our 3-gram word and 5-gram character feature.
3_4_5/trigrams.py -Main executable that provides the final product we turned in (the test.txt generated is the one we submitted last). Parses in stopwords.txt and applies this to the 3-gram word feature; combines our 3-gram word and 5-gram character feature vectors into a union of the two. Currently commented out is the call to combine the results from test.txt and the baseline_v2.txt results. We found that combining the results lowered accuracy, in the end.
unique_words.py
- Helper file for us to visualize unique words and their distributions. We experimented with this approach early on and combined with Stanford POS taggers. But we had little luck.