added own project

Author: mjfang27

CONTRIBUTING.md

@@ -8,13 +8,9 @@ However, Stanford CoreNLP is copyright by Stanford. (Technically, by The Board o
 In order for us to continue to be able to dual-license Stanford CoreNLP, we need to make sure that contributions from others do not restrict Stanford from separately licensing the code.
 
 Therefore, we can accept contributions on any of the following terms:
-
  * If your contribution is a bug fix of 6 lines or less of new code, we will accept it on the basis that both you and us regard the contribution as de minimis, and not requiring further hassle.
  * You can declare that the contribution is in the public domain (in your commit message or pull request).
  * You can make your contribution available under a non-restrictive open source license, such as the Revised (or 3-clause) BSD license, with appropriate licensing information included with the submitted code.
- * You can sign and return to us a contributor license agreement (CLA), explicitly licensing us to be able to use the code.
-   There is a [Contributor License Agreement for Individuals](http://nlp.stanford.edu/software/CLA/individual.html) and
-   a [Contributor License Agreement for Corporations](http://nlp.stanford.edu/software/CLA/corporate.html).
-   You can send them to us or contact us at: java-nlp-support@lists.stanford.edu .
+ * You can sign and return to us a contributor license agreement (CLA), explicitly licensing us to be able to use the code. You can find these agreements at http://nlp.stanford.edu/software/CLA/ . You can send them to us or contact us at: java-nlp-support@mailman.stanford.edu .
 
 You should do development against our master branch. The project's source code is in utf-8 character encoding. You should make sure that all unit tests still pass. (In general, you will not be able to run our integration tests, since they rely on resources in our filesystem.)

LICENSE.txt

@@ -1,52 +1,17 @@
 Stanford CoreNLP
 ================
 
-[![Build Status](https://travis-ci.org/stanfordnlp/CoreNLP.svg?branch=master)](https://travis-ci.org/stanfordnlp/CoreNLP)
+Stanford CoreNLP provides a set of natural language analysis tools written in Java. It can take raw human language text input and give the base forms of words, their parts of speech, whether they are names of companies, people, etc., normalize dates, times, and numeric quantities, and mark up the structure of sentences in terms of phrases and word dependencies, and indicate which noun phrases refer to the same entities. It was originally developed for English, but now also provides varying levels of support for Arabic, (mainland) Chinese, French, German, and Spanish. Stanford CoreNLP is an integrated framework, which make it very easy to apply a bunch of language analysis tools to a piece of text. Starting from plain text, you can run all the tools on it with just two lines of code. Its analyses provide the foundational building blocks for higher-level and domain-specific text understanding applications. Stanford CoreNLP is a set of stable and well-tested natural language processing tools, widely used by various groups in academia, government, and industry.
 
-Stanford CoreNLP provides a set of natural language analysis tools written in Java. It can take raw human language text input and give the base forms of words, their parts of speech, whether they are names of companies, people, etc., normalize and interpret dates, times, and numeric quantities, mark up the structure of sentences in terms of phrases or word dependencies, and indicate which noun phrases refer to the same entities. It was originally developed for English, but now also provides varying levels of support for (Modern Standard) Arabic, (mainland) Chinese, French, German, and Spanish. Stanford CoreNLP is an integrated framework, which make it very easy to apply a bunch of language analysis tools to a piece of text. Starting from plain text, you can run all the tools with just two lines of code. Its analyses provide the foundational building blocks for higher-level and domain-specific text understanding applications. Stanford CoreNLP is a set of stable and well-tested natural language processing tools, widely used by various groups in academia, industry, and government. The tools variously use rule-based, probabilistic machine learning, and deep learning components.
+The Stanford CoreNLP code is written in Java and licensed under the GNU General Public License (v3 or later). Note that this is the full GPL, which allows many free uses, but not its use in proprietary software that you distribute.
 
-The Stanford CoreNLP code is written in Java and licensed under the GNU General Public License (v3 or later). Note that this is the full GPL, which allows many free uses, but not its use in proprietary software that you distribute to others.
-
-#### Build Instructions
-
-Several times a year we distribute a new version of the software, which corresponds to a stable commit.
-
-During the time between releases, one can always use the latest, under development version of our code.
-
-Here are some helpful instructions to use the latest code:
-
-#### build with Ant
-
-1. Make sure you have Ant installed, details here: [http://ant.apache.org/](http://ant.apache.org/)
-2. Compile the code with this command: `cd CoreNLP ; ant`
-3. Then run this command to build a jar with the latest version of the code: `cd CoreNLP/classes ; jar -cf ../stanford-corenlp.jar edu`
-4. This will create a new jar called stanford-corenlp.jar in the CoreNLP folder which contains the latest code
-5. The dependencies that work with the latest code are in CoreNLP/lib and CoreNLP/liblocal, so make sure to include those in your CLASSPATH.
-6. When using the latest version of the code make sure to download the latest versions of the [corenlp-models](http://nlp.stanford.edu/software/stanford-corenlp-models-current.jar), [english-models](http://nlp.stanford.edu/software/stanford-english-corenlp-models-current.jar), and [english-models-kbp](http://nlp.stanford.edu/software/stanford-english-kbp-corenlp-models-current.jar) and include them in your CLASSPATH.  If you are processing languages other than English, make sure to download the latest version of the models jar for the language you are interested in.
-
-#### build with Maven
-
-1. Make sure you have Maven installed, details here: [https://maven.apache.org/](https://maven.apache.org/)
-2. If you run this command in the CoreNLP directory: `mvn package` , it should run the tests and build this jar file: `CoreNLP/target/stanford-corenlp-3.7.0.jar`
-3. When using the latest version of the code make sure to download the latest versions of the [corenlp-models](http://nlp.stanford.edu/software/stanford-corenlp-models-current.jar), [english-models](http://nlp.stanford.edu/software/stanford-english-corenlp-models-current.jar), and [english-models-kbp](http://nlp.stanford.edu/software/stanford-english-kbp-corenlp-models-current.jar) and include them in your CLASSPATH.  If you are processing languages other than English, make sure to download the latest version of the models jar for the language you are interested in.  
-4. If you want to use Stanford CoreNLP as part of a Maven project you need to install the models jars into your Maven repository.  Below is a sample command for installing the Spanish models jar.  For other languages just change the language name in the command.  To install `stanford-corenlp-models-current.jar` you will need to set `-Dclassifier=models`.  Here is the sample command for Spanish: `mvn install:install-file -Dfile=/location/of/stanford-spanish-corenlp-models-current.jar -DgroupId=edu.stanford.nlp -DartifactId=stanford-corenlp -Dversion=3.7.0 -Dclassifier=models-spanish -Dpackaging=jar` 
-
-You can find releases of Stanford CoreNLP on [Maven Central](http://search.maven.org/#artifactdetails%7Cedu.stanford.nlp%7Cstanford-corenlp%7C3.6.0%7Cjar).
+You can find releases of Stanford CoreNLP on [Maven Central](http://search.maven.org/#browse%7C11864822).
 
 You can find more explanation and documentation on [the Stanford CoreNLP homepage](http://nlp.stanford.edu/software/corenlp.shtml#Demo).
 
 The most recent models associated with the code in the HEAD of this repository can be found [here](http://nlp.stanford.edu/software/stanford-corenlp-models-current.jar).
 
-Some of the larger (English) models -- like the shift-reduce parser and WikiDict -- are not distributed with our default models jar.
-The most recent version of these models can be found [here](http://nlp.stanford.edu/software/stanford-english-corenlp-models-current.jar).
-
-We distribute resources for other languages as well, including [Arabic models](http://nlp.stanford.edu/software/stanford-arabic-corenlp-models-current.jar),
-[Chinese models](http://nlp.stanford.edu/software/stanford-chinese-corenlp-models-current.jar),
-[French models](http://nlp.stanford.edu/software/stanford-french-corenlp-models-current.jar),
-[German models](http://nlp.stanford.edu/software/stanford-german-corenlp-models-current.jar),
-and [Spanish models](http://nlp.stanford.edu/software/stanford-spanish-corenlp-models-current.jar).
-
-For information about making contributions to Stanford CoreNLP, see the file [CONTRIBUTING.md](CONTRIBUTING.md).
+For information about making contributions to Stanford CoreNLP, see the file `CONTRIBUTING.md`.
 
-Questions about CoreNLP can either be posted on StackOverflow with the tag [stanford-nlp](http://stackoverflow.com/questions/tagged/stanford-nlp),
+Questions about CoreNLP can either be posted on StackOverflow with the tag [stanford-nlp](http://stackoverflow.com/questions/tagged/stanford-nlp), 
   or on the [mailing lists](http://nlp.stanford.edu/software/corenlp.shtml#Mail).

README.md

@@ -11,7 +11,7 @@ sourceCompatibility = 1.8
 targetCompatibility = 1.8
 compileJava.options.encoding = 'UTF-8'
 
-version = '3.7.0'
+version = '3.4.1'
 
 // Gradle application plugin
 mainClassName = "edu.stanford.nlp.pipeline.StanfordCoreNLP"
@@ -41,15 +41,12 @@ sourceSets {
   }
 }
 
-task listDeps {
-    doLast {
-        configurations.compile.each { File file -> println file.name }
-    }
+task listDeps << {
+    configurations.compile.each { File file -> println file.name }
 }
 
 dependencies {
   compile fileTree(dir: 'lib', include: '*.jar')
-  testCompile fileTree(dir: 'liblocal', include: '*.jar')
 }
 
 // Eclipse plugin setup
@@ -63,7 +60,3 @@ eclipse {
     }
   }
 }
-
-task wrapper(type: Wrapper) {
-    gradleVersion = '3.2'
-}

build.gradle

@@ -26,10 +26,6 @@
         <include name="*.jar"/>
         <exclude name="javanlp*"/>
       </fileset>
-      <fileset dir="${basedir}/liblocal">
-        <include name="*.jar"/>
-        <exclude name="javanlp*"/>
-      </fileset>
     </path>
   </target>
 
@@ -128,16 +124,6 @@
         <compilerarg value="-Xmaxwarns"/>
         <compilerarg value="10000"/>  -->
     </javac>
-    <copy todir="${build.path}/edu/stanford/nlp/pipeline/demo">
-      <fileset dir="${source.path}/edu/stanford/nlp/pipeline/demo">
-        <exclude name="**/*.java"/>
-      </fileset>
-    </copy>
-    <copy todir="${build.path}/edu/stanford/nlp/pipeline">
-      <fileset dir="${source.path}/edu/stanford/nlp/pipeline">
-        <exclude name="**/*.java"/>
-      </fileset>
-    </copy>
   </target>
 
   <target name="test" depends="classpath,compile"
@@ -160,7 +146,7 @@
   <target name="itest" depends="classpath,compile"
           description="Run core integration tests">
     <echo message="${ant.project.name}" />
-    <junit fork="yes" maxmemory="12g" printsummary="off" outputtoformatters="false" forkmode="perTest" haltonfailure="true">
+    <junit fork="yes" maxmemory="8g" printsummary="off" outputtoformatters="false" forkmode="perTest" haltonfailure="true">
       <classpath refid="classpath"/>
       <classpath path="${build.path}"/>
       <classpath path="${data.path}"/>
@@ -178,7 +164,7 @@
   <target name="slowitest" depends="classpath,compile"
           description="Run really slow integration tests">
     <echo message="${ant.project.name}" />
-    <junit fork="yes" maxmemory="12g" printsummary="off" outputtoformatters="false" forkmode="perTest" haltonfailure="true">
+    <junit fork="yes" maxmemory="8g" printsummary="off" outputtoformatters="false" forkmode="perTest" haltonfailure="true">
       <classpath refid="classpath"/>
       <classpath path="${build.path}"/>
       <classpath path="${data.path}"/>
@@ -313,7 +299,7 @@
         <include name="commons-lang3-3.1.jar"/>
         <include name="xom-1.2.10.jar"/>
         <include name="joda-time.jar"/>
-        <include name="jollyday-0.4.9.jar"/>
+        <include name="jollyday-0.4.7.jar"/>
       </lib>
       <zipfileset prefix="WEB-INF/data"
                   file="/u/nlp/data/pos-tagger/distrib/english-left3words-distsim.tagger"/>
@@ -389,9 +375,7 @@
         <zipfileset prefix="WEB-INF/data"
                     file="/u/nlp/data/lexparser/arabicFactored.ser.gz"/>
         <zipfileset prefix="WEB-INF/data"
-                    file="/u/nlp/data/lexparser/frenchFactored.ser.gz"/>
-        <zipfileset prefix="WEB-INF/data"
-                    file="/u/nlp/data/lexparser/chineseFactored.ser.gz"/>
+                    file="/u/nlp/data/lexparser/xinhuaFactored.ser.gz"/>
         <zipfileset prefix="WEB-INF/data/chinesesegmenter"
                     file="/u/nlp/data/gale/segtool/stanford-seg/classifiers-2010/05202008-ctb6.processed-chris6.lex.gz"/>
         <zipfileset prefix="WEB-INF/data/chinesesegmenter"
@@ -448,7 +432,7 @@
         <include name="xom-1.2.10.jar"/>
         <include name="xml-apis.jar"/>
         <include name="joda-time.jar"/>
-        <include name="jollyday-0.4.9.jar"/>
+        <include name="jollyday-0.4.7.jar"/>
       </lib>
       <!-- note for John: c:/Users/John Bauer/nlp/stanford-releases -->
       <lib dir="/u/nlp/data/StanfordCoreNLPModels">

build.xml

@@ -6,7 +6,7 @@ Small data files (test files, properties files, etc) can go here.
 Don't put large models here.  That would cause git to bloat too much.
 
 Open source: Don't put here any file that is copyright or that we
-don't have the rights to redistribute. The contents of this
+don't have the rights to rights to redistribute. The contents of this
 directory appear in our public github repository.
 
 Contact: John Bauer (horatio@gmail.com)

data/README

@@ -17,16 +17,16 @@ outDir=SPIEDPatternsout
 #Number of threads available on the machine
 numThreads=1
 #***Use these options if you are limited by memory
-batchProcessSents = false
+batchProcessSents = true
 #This name is a misnomer. Max number of *lines* per batch file. Works only for text file format; ser files cannot be broken down
 numMaxSentencesPerBatchFile=100
-saveInvertedIndex=false
+saveInvertedIndex=true
 invertedIndexDirectory=${outDir}/invertedIndex
 #Loading index from invertedIndexDirectory
 #loadInvertedIndex=true
 
 #Useful for memory heavy apps. 
-#invertedIndexClass=edu.stanford.nlp.patterns.LuceneSentenceIndex
+invertedIndexClass=edu.stanford.nlp.patterns.LuceneSentenceIndex
 
 
 ### Example for running it on presidents biographies. For more data examples, see the bottom of this file
@@ -43,7 +43,7 @@ saveSentencesSerDir=${outDir}/sents
 #fileFormat=ser
 #file=${outDir}/sents
 
-#We are learning names of presidential candidates, places, and other names. In each line, all text after tabs are ignored in these seed files
+#We are learning names of presidential candidates, places, and other names
 seedWordsFiles=NAME,${DIR}/names.txt;PLACE,${DIR}/places.txt;OTHER,${DIR}/otherpeople.txt
 #Useful for matching lemmas or spelling mistakes
 fuzzyMatch=false
@@ -103,7 +103,7 @@ targetAllowedTagsInitialsStr=NAME,N;OTHER,N
 computeAllPatterns = true
 
 #Options: MEMORY, DB, LUCENE. If using SQL for storing patterns for each token --- populate SQLConnection class, that is provide those properties!
-storePatsForEachToken=MEMORY
+storePatsForEachToken=LUCENE
 #***If your code is running too slow, try to reduce this number. Samples % of sentences for learning patterns
 sampleSentencesForSufficientStats=1.0
 

data/edu/stanford/nlp/patterns/surface/example.properties

@@ -885,7 +885,7 @@ origins
 ''
 Libyan
 ruler
-Mu`ammar
+Mu‘ammar
 al-Qaddafi
 referred
 to

data/edu/stanford/nlp/process/ptblexer.gold

@@ -6,25 +6,27 @@
 *	VBD	VerbForm=Fin|Mood=Ind|Tense=Past
 *	VBN	Tense=Past|VerbForm=Part
 *	VBP	VerbForm=Fin|Mood=Ind|Tense=Pres
-*	MD	VerbForm=Fin
 *	JJ	Degree=Pos
 *	JJR	Degree=Cmp
 *	JJS	Degree=Sup
+*	RB	Degree=Pos
+*	RBR	Degree=Cmp
+*	RBS	Degree=Sup
 *	CD	NumType=Card
 am	VBP	VerbForm=Fin|Mood=Ind|Tense=Pres|Person=1|Number=Sing
 was	VBD	VerbForm=Fin|Mood=Ind|Tense=Past|Number=Sing
-i	PRP	Number=Sing|Person=1|PronType=Prs|Case=Nom
+i	PRP	Number=Sing|Person=1|PronType=Prs
 you	PRP	Person=2|PronType=Prs
-he	PRP	Number=Sing|Person=3|Gender=Masc|PronType=Prs|Case=Nom
-she	PRP	Number=Sing|Person=3|Gender=Fem|PronType=Prs|Case=Nom
+he	PRP	Number=Sing|Person=3|Gender=Masc|PronType=Prs
+she	PRP	Number=Sing|Person=3|Gender=Fem|PronType=Prs
 it	PRP	Number=Sing|Person=3|Gender=Neut|PronType=Prs
-we	PRP	Number=Plur|Person=1|PronType=Prs|Case=Nom
-they	PRP	Number=Plur|Person=3|PronType=Prs|Case=Nom
-me	PRP	Number=Sing|Person=1|PronType=Prs|Case=Acc
-him	PRP	Number=Sing|Person=3|Gender=Masc|PronType=Prs|Case=Acc
-her	PRP	Number=Sing|Person=3|Gender=Fem|PronType=Prs|Case=Acc
-us	PRP	Number=Plur|Person=1|PronType=Prs|Case=Acc
-them	PRP	Number=Plur|Person=3|PronType=Prs|Case=Acc
+we	PRP	Number=Plur|Person=1|PronType=Prs
+they	PRP	Number=Plur|Person=3|PronType=Prs
+me	PRP	Number=Sing|Person=1|PronType=Prs
+him	PRP	Number=Sing|Person=3|Gender=Masc|PronType=Prs
+her	PRP	Number=Sing|Person=3|Gender=Fem|PronType=Prs
+us	PRP	Number=Plur|Person=1|PronType=Prs
+them	PRP	Number=Plur|Person=3|PronType=Prs
 my	PRP$	Number=Sing|Person=1|Poss=Yes|PronType=Prs
 mine	PRP$	Number=Sing|Person=1|Poss=Yes|PronType=Prs
 your	PRP$	Person=2|Poss=Yes|PronType=Prs
@@ -37,70 +39,24 @@ our	PRP$	Number=Plur|Person=1|Poss=Yes|PronType=Prs
 ours	PRP$	Number=Plur|Person=1|Poss=Yes|PronType=Prs
 their	PRP$	Number=Plur|Person=3|Poss=Yes|PronType=Prs
 theirs	PRP$	Number=Plur|Person=3|Poss=Yes|PronType=Prs
-myself	PRP	Number=Sing|Person=1|PronType=Prs
-yourself	PRP	Number=Sing|Person=2|PronType=Prs
-himself	PRP	Number=Sing|Person=3|Gender=Masc|PronType=Prs
-herself	PRP	Number=Sing|Person=3|Gender=Fem|PronType=Prs
-itself	PRP	Number=Sing|Person=3|Gender=Neut|PronType=Prs
-ourselves	PRP	Number=Plur|Person=1|PronType=Prs
-yourselves	PRP	Number=Plur|Person=2|PronType=Prs
-themselves	PRP	Number=Plur|Person=3|PronType=Prs
+myself	PRP	Number=Sing|Person=1|Reflex=Yes|PronType=Prs
+yourself	PRP	Person=2|Reflex=Yes|PronType=Prs
+himself	PRP	Number=Sing|Person=3|Reflex=Yes|Gender=Masc|PronType=Prs
+herself	PRP	Number=Sing|Person=3|Reflex=Yes|Gender=Fem|PronType=Prs
+itself	PRP	Number=Sing|Person=3|Reflex=Yes|Gender=Neut|PronType=Prs
+ourselves	PRP	Number=Plur|Person=1|Reflex=Yes|PronType=Prs
+themselves	PRP	Number=Plur|Person=3|Reflex=Yes|PronType=Prs
 the	DT	Definite=Def|PronType=Art
 a	DT	Definite=Ind|PronType=Art
 an	DT	Definite=Ind|PronType=Art
+some	DT	Definite=Ind|PronType=Art
+any	DT	Definite=Ind|PronType=Art
 this	DT	PronType=Dem|Number=Sing
 that	DT	PronType=Dem|Number=Sing
 these	DT	PronType=Dem|Number=Plur
 those	DT	PronType=Dem|Number=Plur
-here	RB	PronType=Dem
-there	RB	PronType=Dem
-then	RB	PronType=Dem
-whose	WP$	Poss=Yes
-hard	RB	Degree=Pos
-fast	RB	Degree=Pos
-late	RB	Degree=Pos
-long	RB	Degree=Pos
-high	RB	Degree=Pos
-easy	RB	Degree=Pos
-early	RB	Degree=Pos
-far	RB	Degree=Pos
-soon	RB	Degree=Pos
-low	RB	Degree=Pos
-close	RB	Degree=Pos
-well	RB	Degree=Pos
-badly	RB	Degree=Pos
-little	RB	Degree=Pos
-harder	RBR	Degree=Cmp
-faster	RBR	Degree=Cmp
-later	RBR	Degree=Cmp
-longer	RBR	Degree=Cmp
-higher	RBR	Degree=Cmp
-easier	RBR	Degree=Cmp
-quicker	RBR	Degree=Cmp
-earlier	RBR	Degree=Cmp
-further	RBR	Degree=Cmp
-farther	RBR	Degree=Cmp
-sooner	RBR	Degree=Cmp
-slower	RBR	Degree=Cmp
-lower	RBR	Degree=Cmp
-closer	RBR	Degree=Cmp
-better	RBR	Degree=Cmp
-worse	RBR	Degree=Cmp
-less	RBR	Degree=Cmp
-hardest	RBS	Degree=Sup
-fastest	RBS	Degree=Sup
-latest	RBS	Degree=Sup
-longest	RBS	Degree=Sup
-highest	RBS	Degree=Sup
-easiest	RBS	Degree=Sup
-quickest	RBS	Degree=Sup
-earliest	RBS	Degree=Sup
-furthest	RBS	Degree=Sup
-farthest	RBS	Degree=Sup
-soonest	RBS	Degree=Sup
-slowest	RBS	Degree=Sup
-lowest	RBS	Degree=Sup
-closest	RBS	Degree=Sup
-best	RBS	Degree=Sup
-worst	RBS	Degree=Sup
-least	RBS	Degree=Sup
+
+
+
+
+