Repository navigation
Anthology ingestion format
This page describes the information needed by the ACL Anthology for ingestion.
Ingestion into the Anthology produces XML files, for example, 2020.wmt.xml.
An XML file is assigned to each (year, venue) pairing.
Inside this XML file are one or more volumes.
Each of these volumes has an alphanumeric name.
Together, the year, venue ID, and volume name produce the Volume ID, e.g., 2020.acl-main, 2021.wmt-1, etc.
Each volume creates an implicit event.
We can link together multiple volumes into a single event by listing them in a joint file.
For example, the ACL event contains a list of all the volumes of all the workshops that were colocated with it.
This information we can infer when importing many volumes together, though see the note below.
The basic unit of ingestion is a volume. A volume groups together one or more papers. The volume has a booktitle, a list of editors, a publisher, and the month and year it was published. If it was associated with a conference, the location can be recorded. For conferences, the list of editors is typically the program chair(s) and the general chairs. For workshops, the editors may include all the top-level organizers.
In your proposed layout, it seems that each volume gets its own directory structure. The demo example is a good example. The files used by the Anthology are as follows:
-
conference_details.yml. This contains most of the information above. Ideally, it would explicitly list the year and the month, so that they don’t need to be inferred. It should also include a list of editors, where each editor’s given and family names are separated out. Other editor information, such as affiliation, can also be included, but is currently ignored. It may help with author disambiguation, though, since these are tied to author pages in the Anthology (e.g., Matt Post). Alternately, the list of editors can be added to the frontmatter, conventionally listed as paper ID 0.
Also of importance is the acronym field. This will become the venue name (e.g., WMT, ACL, etc) after slugification. It must be composed only of ASCII letters and digits (i.e.,
[0-9A-Za-z]+): no hyphens or other special characters. New venue names must be approved by the Anthology directory. -
papers.yaml. This contains the paper names. Author names should be split into two fields, the given name and the family name, and should also include other information (Semantic Scholar ID, ORCID, affiliation, etc), where available, that will assist in author disambiguation.
Missing or incomplete
- Where is the complete book? In ACLPUB, this is
cdrom/book.pdf. - Where is the frontmatter for the book? In ACLPUB, this is paper ID 0.
- Where is the volume name? The file
conference_details.ymlmixes up information relating to the conference and the volume. For example, is “title” the title of the conference or the volume? Whereas “isbn” clearly belongs to the volume. - I suggest this be split into two files (adding, for example, a
volume.yamlfile), or field names be used to distinguish them. - For completeness and sanity checking, it would be nice if paper entries had CRC32 checksums for the PDFs
- Papers should be watermarked or footstamped with conference identifying information
-
papers.yamlhas paths that are relative topdf/(e.g.,1.pdf). I think the paths should be relative to the root dir, i.e., the file should havepdf/1.pdfas the path. This is more general and allows for future paths pointing to files not inpdf/. - People often submit attachments: appendices, datasets, code, etc. How do you plan to handle this?
We can ingest multiple volumes at once, and explicitly associate them as part of a single conference or event. I suggest a directory layout like the above, where each directory is a volume. For example, you can have
acl-main/
conference_details.yml
papers.yml
pdf/
...
acl-demos/
conference_details.yml
papers.yml
pdf/
...
bea/
conference_details.yml
papers.yml
pdf/
...
Missing or incomplete
However, we need meta-information about the event that answers the following questions:
- Which of the events above is the main or central event?
- Should the event be cross-listed, e.g.,
aclwithijcnlp?
For cross-listing, we have a problem that sometimes there are true joint events, such as acl and ijcnlp.
However, our naming convention forces one of these to be selected for the Anthology ID.
We would like the event to be listed under both https://aclanthology.org/venues/acl/ and https://aclanthology.org/venues/ijcnlp.
So we need a way to establish this.