-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Referencing implicit/empty/ghost words from span annotation #58
Comments
It is time to pick this up again now that a lot of progress on FoLiA v2 is made. Most of this discussion, with @luutuntin, took place in proycon/flat#138. We need an explicit mechanism to refer to tokens that do not really exist, mostly for syntactic movements. I have a proposal: introduce a <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0">
<desc>empty expletive subject</desc>
<pos class="EX" />
</hiddenw>
<w xml:id="s.1.w.1" space="no">
<t>Is</t>
<pos class="BEP" />
</w>
<w xml:id="s.1.w.2">
<t>n't</t>
<pos class="NEG" />
</w>
<w xml:id="s.1.w.3">
<t>a</t>
<pos class="D" />
</w>
<w xml:id="s.1.w.4">
<t>whole</t>
<pos class="ADJ" />
</w>
<w xml:id="s.1.w.5">
<t>lot</t>
<pos class="N" />
</w>
<w xml:id="s.1.w.6" space="no">
<t>left</t>
<pos class="VAN" />
</w>
<w xml:id="s.1.w.7">
<t>.</t>
<pos class="PUNC" />
</w>
<syntax>
<su xml:id="s.1.su.1" class="IP-MAT">
<su xml:id="s.1.su.2" class="NP-SBJ">
<wref id="s.1.w.0" />
</su>
<su xml:id="s.1.su.3" class="VP">
<su xml:id="s.1.su.4" class="BEP">
<wref id="s.1.w.1" />
</su>
<su xml:id="s.1.su.5" class="NEG">
<wref id="s.1.w.2" />
</su>
<su xml:id="s.1.su.6" class="VP">
<su xml:id="s.1.su.7" class="NP=LGS">
<wref id="s.1.w.3" />
<su xml:id="s.1.su.8" class="ADJP">
<wref id="s.1.w.4" />
</su>
<wref id="s.1.w.5" />
</su>
<wref id="s1.w.6" />
</su>
</su>
<su class="PUNC">
<wref id="s.1.w.7" />
</su>
</su>
</syntax>
</s> The hidden tokens would have their own annotation type and can be bound to a set, which allows for multiple hidden tokenisation layers, in case multiple are needed for different purposes. The Thoughts and comments welcome! |
I'm happy with your proposal. I also agree that we shouldn't forbid text content ( <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0">
<t>*exp*</t>
<desc>empty expletive subject</desc>
<pos class="EX" />
</hiddenw>
...
</s> When you mentioned that the hidden tokens would have their own annotation type, do you mean that hidden tokens can have the same token annotations as word tokens ( |
Yes
I rather meant that hidden tokens would be a new specific annotation type itself and needs to be declared. There's no annotation layer associated with this type, as it's a structural element rather than a span element. |
…enough to omit it from any text serialisation and consistency checks #58
Thank you. What I mean by "additional (hidden) token annotations such as annotation layer" is, for example: <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0">
<t>*exp*</t>
<desc>empty expletive subject</desc>
<pos class="EX" />`
<annotation_layer='syntax'>
</hiddenw>
...
</s> |
The syntax annotation layer would be embedded in <hiddenw class="syntactic"> |
I see. So the example above should be like this, right?: <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0">
<t>*exp*</t>
<desc>empty expletive subject</desc>
<pos class="EX" />`
<annotation_layer class='syntax'>
</hiddenw>
...
</s> |
proycon/flat#138 and serving as an example and test for hidden tokens (#58)
no, like this: <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0" class="syntax">
<t>*exp*</t>
<desc>empty expletive subject</desc>
<pos class="EX" />`
<annotation_layer class='syntax'>
</hiddenw>
</s> "syntax" is then just a custom defined class in a set which you use for hidden tokens (you could also opt for something more especific like <hiddentoken-annotation set="http://wherever/the/set/definition/is/if/it/exists/at/all" /> |
But then we don't need <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0" class='syntax'>
<t>*exp*</t>
<desc>empty expletive subject</desc>
<pos class="EX" />`
<!--annotation_layer='syntax'-->
</hiddenw>
...
</s> My next questions (which are not critical now) will be how do we deal with the case:
|
Ah yes, sorry, I accidentally left in
Multiple classes are not allowed indeed. But you can refer to the same token from multiple span annotation layers using The fact you refer to a hidden token from a syntax layer should already make clear that it has a role in syntax, so putting classes on the <s xml:id="s.1">
<hiddenw xml:id="s.1.w.0">
<t>*exp*</t>
<desc>empty expletive subject</desc>
<pos class="EX" />`
</hiddenw>
...
<syntax>
<su xml:id="s.1.su.1" class="IP-MAT">
<su xml:id="s.1.su.2" class="NP-SBJ">
<wref id="s.1.w.0" /> <!-- reference to the hiddenw -->
</su>
...
</syntax>
..
<semroles>
<predicate xml:id="s.1.pr.1">
<semrole xml:id="s.1.pr.1.r.1" class="AGENT">
<wref id="s.1.w.0" /> <!-- another reference to the hiddenw -->
</semrole>
...
</predicate>
</semroles> |
Thank you. |
Today, I just reviewed FoLiA documentation (PDF) - section 2.10.8 Corrections. I can see that, for example, an insertion can be a solution for introducing a hidden word/token. Is there any reason that makes this not a good solution? |
That would not be an appropriate solution for hidden words, because those are not explicitly hidden words. Corrections really express "the old situation was wrong, this is how it was and this is how it should be instead", which is semantically different from what you want with hidden tokens. |
Thank you. |
This is now released as proposed with FoliA v2.0.0 and documented here as part of the new FoLiA documentation: https://folia.readthedocs.io/en/latest/hiddentoken_annotation.html |
Great. |
How to refer to words/tokens that are not actually there?
Discussed as part of proycon/flat#138
The text was updated successfully, but these errors were encountered: