-
Notifications
You must be signed in to change notification settings - Fork 4
Comparing two or more stemmers
A direct method to compare the similarity of two stemmers is to use a common word list for the input to both stemmers, then to input the two outputs of the stemmers into File A and File B of the List Analyser.

The input to File A is the output.txt file from the Porter website http://tartarus.org/~martin/PorterStemmer/ File B is the output from Stemming Tester 1.4, using voc.txt file from the Porter website as the input word list and the default English Stemming.dat file from dtSearch Desktop. (The output files from Stemming Tester 1.4 are saved in the same folder as the input file, they are named inputfilename_Porter.txt if the built-in Porter(1980) stemmer is used, or inputfilename_Stemming.txt if the default dtSearch Stemming.dat rule file is used, so in the case above using the voc.txt file as input, the output file from the dtSearch stemmer is named voc_Stemming.txt)
The results of interest are the Similarity group metrics and the Mean Word Length and Unique Words count from each stemmer. See Stemmer Similarity Metric (SSM)