Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overly rigid treatment of top nodes in SMATCH #12

Closed
oepen opened this issue Jun 15, 2019 · 2 comments
Closed

overly rigid treatment of top nodes in SMATCH #12

oepen opened this issue Jun 15, 2019 · 2 comments
Assignees

Comments

@oepen
Copy link
Contributor

oepen commented Jun 15, 2019

when comparing our (forthcoming) MCES scorer with SMATCH, it emerged that there is a difference in the count of correct tuples for two test AMRs from the SMATCH distribution:

# ::snt The boy is a hard worker.
(p / person
      :domain (b / boy)
      :ARG0-of (w / work-01
            :manner (h / hard)))
# ::snt The boy is a hard worker.
(w / worker
      :mod (h / hard)
      :domain (b / boy))

SMATCH correctly establishes three node correspondences: the ‘b’ and ‘h’ nodes from both graphs, and ‘p’ from the first graph with ‘w’ from the second. on this view, the two graphs have the same top node, share two node labels (‘boy’ and ‘hard’), as well as one edge (:domain). however, SMATCH appears to represent the top properties as triples that include the node label, which results in an undesirable double penalty. i wonder whether this treatment in SMATCH is background to the critique by Anchieta et al. (2019)?

@danielhers
Copy link
Collaborator

At least according to the example in the smatch paper, the top triple is not part of the scoring... but it seems ‘attribute’ triples (including top) were probably just introduced later.

@oepen
Copy link
Contributor Author

oepen commented Aug 2, 2019

closing this issue in mtool (for now), since we decided we wanted to maintain compatibility with the ‘official’ version of SMATCH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants