Separate metadata for different texts in a parallel corpus document

[thomaskrause]

Currently (V2.2.1), metadata is annotatable per document. Since parallel corpus documents are each one document containing many texts in the relANNIS data model, there is no way to give them separate meta-annotations. However this would make sense for e.g. language (text1 is English, text2 is German), name of the translator, etc. This is already possible in PAULA and SaltNPepper but relANNIS and the ANNIS interface need to be adapted to fit this possibility. Example query:

cat="NP" & meta::is_translated="true"

Outputs all cat="NP" annotations in texts of a parallel document that are translated (but not the original text, which is meta-annotated as meta::is_translated="false"

---

Imported from Launchpad using lp2gh.

• date created: 2012-05-08T09:34:47Z
• owner: amir-zeldes
• assignee: krause
• the launchpad url was https://bugs.launchpad.net/bugs/996442

[dlukes]

I'm wondering what the status is of this proposed enhancement? Is it a feature that is being worked on?

It would definitely be useful for corpora of spoken language as well, with layers featuring multiple interacting speakers annotated with sociolinguistic metadata (my case). E.g.

"erm" & meta::gender="male"

would retrieve all hesitations by male speakers.

I'm currently exploring roundabout, hacky ways to allow such queries using pointing relations, but a systemic solution would definitely be much better :)

[amir-zeldes]

This is indeed an important feature, although I don’t think it is being worked on at this moment. The thought was more that this would be useful for translation corpora (e.g. a document will have different metadata for an original version, including publication year, etc., and then for each translated version, including separate translator, year etc.).

Are you using parallel corpora to model dialog data? This is conceivable, but not really the way dialog corpora have been implemented in ANNIS so far. Generally we have use annotation spans to express the overlapping tokens of each speaker, which are flagged as ‘segmentation layers’ using Peppers add-order-relation manipulator. This allows searches like “yeah” to go over all speakers, as opposed to speaker1="yeah".

Adding metadata to speakers within such a model is an interesting idea (similar to Exmaralda’s data model), but a concept of speaker is not currently implemented in ANNIS. So I’m not sure if parallel text metadata will solve your issue, but speakers are also an interesting concept to think about modeling more directly.

From: David Lukes [mailto:notifications@github.com]
Sent: Thursday, April 30, 2015 09:18
To: korpling/ANNIS
Cc: Amir Zeldes
Subject: Re: [ANNIS] Separate metadata for different texts in a parallel corpus document (#10)

I'm wondering what is the status of this proposed enhancement? Is it a feature that is being worked on?

It would definitely be useful for corpora of spoken language as well, with layers featuring multiple interacting speakers annotated with sociolinguistic metadata (my case). E.g.

"erm" & meta::gender="male"

would retrieve all hesitations by male speakers.

I'm currently exploring roundabout, hacky ways to allow such queries using pointing relations, but a systemic solution would definitely be much better :)

—
Reply to this email directly or view it on GitHub #10 (comment) .Image removed by sender.

[dlukes]

Are you using parallel corpora to model dialog data?

No no, I followed the multiple segmentation corpora guide, so my approach is pretty standard in this respect. Plus I threw in speaker namespaces, so instead of e.g. norm1 or phon1 I have spk1:norm and spk1:phon, which allow me to search even non-segmentation layers comprehensively without specifying a particular speaker (by saying phon=...).

I'm sorry for the confusion, I don't know much about parallel corpora in ANNIS, I was just trying to avoid double-posting a feature request and this seemed like a similar requirement at first glance when initially searching the issue list, so I bumped it instead of starting a new one. Now that I've inspected the structure of parallel.sample, I see that parallel corpora are implemented quite differently from spoken ones (and with good reason).

I assumed based on this snippet...

Since parallel corpus documents are each one document containing many texts in the relANNIS data model, there is no way to give them separate meta-annotations. ... This is already possible in PAULA and SaltNPepper

... that the solution would be to allow per-namespace metadata in each document, which would be the ideal solution for my use case (each spk1, spk2... namespace would have its own set of metadata in each document). But perhaps this is not possible.

[thomaskrause]

This can now be implemented by creating the corpus e.g. with the graphANNIS Python API and adding sub-corpus nodes for each text and connect these to a parent sub-corpus with an PartOf edge. It would be possible to search for this metadata by using the @ syntax.

Unfortunately, the relANNIS export format does not support this yet, so there is no nice way to use this enhanced data model using Pepper and ANNIS.