Coreference crucial enhancement - account for predicate's arguments

[kleinay]

Following #31 major change at splitting entities and marking implicit propositions, it is crucial to enhance proposition coreference algorithm by accounting for the arguments of the predicate (and not only the head lemma).

[kleinay]

Changed the input for the cluster_mentions function for propositions.
until now, each mention in the mention_list was a (mention-id, head_lemma, ) tuple, where the score function only used the second element from the tuple (the head lemma).
After my change, each mention is a (mention-unique-id, mention-head-lemma, mention-full-info) tuple.
mention-head-lemma is a string, and is given for backward compatibility.
mention-full-info is a dict containing all the info about the proposition-mention as given by props_wrapper, with a modification of the "Arguments" field, which would be a dict mapping template-symbols (e.g. "A1" or "P2") to their mention records.

a mention for example:
('7_P1', 'suspect',
{'Arguments':
{'A1': {'indices': (2,), 'sentence_id': '7', 'terms': u'down'},
'A2': {'indices': (0,), 'sentence_id': '7', 'terms': u'Turkey'},
'A3': {'indices': (5,), 'sentence_id': '7', 'terms': u'plane'}},
'Bare predicate': ('suspect', (3,)),
'Head': {'Lemma': 'suspect', 'POS': 'VBP', 'Surface': ('suspect', [3])},
'Template': '{A2} {A1} suspect {A3}',
'sentence_id': '7'}

[OriShapira]

Just as another example, in the attached file, notice proposition P.16 has many different unrelated predicates coreferred ("have been targeting", "killing", "raping and killing", "am trying", "western", "able", "are attacking over", "floating in", "one of", ...).

Burma.in.json.txt

[kleinay]

this was addressed by @shanybar in PR #40.

[kleinay]

@OriShapira , the strange coreference cluster was caused by clustering all proposition that has empty head lemma (failure of the lammatizer return empty string). fixed in #41.