No description or website provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Author: Marie Dubremetz Description This script has been written because currently stanford coreNLP do not support lemmatization for the french language. Thus it takes the output file of stanford coreNLP (in XML) and pass each words through tree-tagger individually.




You need to install and be able to use Tree-Tagger and stanford coreNLP


Given a xmlfile parsed by stanford, the script replace all the words under the tag "<lemma>" by the lemmatized version

First run your stanford parser on your french text with output xml (with the annotator lemma, which will not do anything except from adding the "lemma" tag with the word in it. For me the command was:

	java -mx3g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLP -props -annotators tokenize,ssplit,pos,lemma,parse -file myfile -outputFormat xml


	$bash myfile.xml

old output:

	 <token id="1">

Example of desired output:

	 <token id="1">


This is a totally unofficial patch that overcomes the lack of lemmatization of Stanford coreNLP. It works only on the XML version with the annotators in the java command that I gave above! Since it lemmatizes words one by one the performance might be lower than usual and the process is very slow!


Marie Dubremetz.


You may cite the papers related to the version of the parser you use (Stanford CoreNLP) and TreeTagger