Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

Change how we handle the PU tag in Chinese.

We now only use the word as a tag if the word is a known tag.  Otherwise,
we continue to use the PU tag.  Better error message for the TRAIN/
version.
  • Loading branch information...
commit 9c0b960f69f4ab032f5cec8fca5568fe6cc12635 1 parent c2736bf
David McClosky dmcc authored
4 first-stage/PARSE/InputTree.C
@@ -174,7 +174,9 @@ newParse(istream& is, int& strt, InputTree* par)
174 174 /* fixes bugs in Chinese Treebank */
175 175 if(Term::Language == "Ch")
176 176 {
177   - if(trm == "PU") trm = wrd;
  177 + if (trm == "PU" && Term::get(wrd)) {
  178 + trm = wrd;
  179 + }
178 180 const Term* ctrm = Term::get(trm);
179 181 if(!ctrm)
180 182 {
8 first-stage/TRAIN/InputTree.C
@@ -202,10 +202,14 @@ newParse(istream& is, int& strt, InputTree* par)
202 202 }
203 203 }
204 204 }
205   - if(Term::Language == "Ch" && trm == "PU") trm = wrd;
  205 + if(Term::Language == "Ch" && trm == "PU") {
  206 + if (Term::get(wrd)) {
  207 + trm = wrd;
  208 + }
  209 + }
206 210 if (!Term::get(trm))
207 211 {
208   - cerr<<trm<<endl;
  212 + cerr << "Couldn't find term: " << trm << endl;
209 213 assert(Term::get(trm));
210 214 }
211 215 if(wrd == "" && subTrs.size() == 0) return NULL;

0 comments on commit 9c0b960

Please sign in to comment.
Something went wrong with that request. Please try again.