Skip to content

Anthology ingestion format

Matt Post edited this page Feb 9, 2022 · 5 revisions

This page describes the information needed by the ACL Anthology for ingestion.

General comments

Ingestion into the Anthology produces XML files, for example, 2020.wmt.xml. An XML file is assigned to each (year, venue) pairing. Inside this XML file are one or more volumes. Each of these volumes has an alphanumeric name. Together, the year, venue ID, and volume name produce the Volume ID, e.g., 2020.acl-main, 2021.wmt-1, etc.

Each volume creates an implicit event. We can link together multiple volumes into a single event by listing them in a joint file. For example, the ACL event contains a list of all the volumes of all the workshops that were colocated with it. This information we can infer when importing many volumes together, though see the note below.

Single volume

The basic unit of ingestion is a volume. A volume groups together one or more papers. The volume has a booktitle, a list of editors, a publisher, and the month and year it was published. If it was associated with a conference, the location can be recorded. For conferences, the list of editors is typically the program chair(s) and the general chairs. For workshops, the editors may include all the top-level organizers.

In your proposed layout, it seems that each volume gets its own directory structure. The demo example is a good example. The files used by the Anthology are as follows:

  • conference_details.yml. This contains most of the information above. Ideally, it would explicitly list the year and the month, so that they don’t need to be inferred. It should also include a list of editors, where each editor’s given and family names are separated out. Other editor information, such as affiliation, can also be included, but is currently ignored. It may help with author disambiguation, though, since these are tied to author pages in the Anthology (e.g., Matt Post). Alternately, the list of editors can be added to the frontmatter, conventionally listed as paper ID 0.

    Also of importance is the acronym field. This will become the venue name (e.g., WMT, ACL, etc) after slugification. It must be composed only of ASCII letters and digits (i.e., [0-9A-Za-z]+): no hyphens or other special characters. New venue names must be approved by the Anthology directory.

  • papers.yaml. This contains the paper names. Author names should be split into two fields, the given name and the family name, and should also include other information (Semantic Scholar ID, ORCID, affiliation, etc), where available, that will assist in author disambiguation.

Missing or incomplete

  • Where is the complete book? In ACLPUB, this is cdrom/book.pdf.
  • Where is the frontmatter for the book? In ACLPUB, this is paper ID 0.
  • Where is the volume name? The file conference_details.yml mixes up information relating to the conference and the volume. For example, is “title” the title of the conference or the volume? Whereas “isbn” clearly belongs to the volume.
  • I suggest this be split into two files (adding, for example, a volume.yaml file), or field names be used to distinguish them.
  • For completeness and sanity checking, it would be nice if paper entries had CRC32 checksums for the PDFs
  • Papers should be watermarked or footstamped with conference identifying information
  • papers.yaml has paths that are relative to pdf/ (e.g., 1.pdf). I think the paths should be relative to the root dir, i.e., the file should have pdf/1.pdf as the path. This is more general and allows for future paths pointing to files not in pdf/.
  • People often submit attachments: appendices, datasets, code, etc. How do you plan to handle this?

Multiple volumes

We can ingest multiple volumes at once, and explicitly associate them as part of a single conference or event. I suggest a directory layout like the above, where each directory is a volume. For example, you can have

acl-main/
  conference_details.yml
  papers.yml
  pdf/
    ...
acl-demos/
  conference_details.yml
  papers.yml
  pdf/
    ...
bea/
  conference_details.yml
  papers.yml
  pdf/
    ...

Missing or incomplete

However, we need meta-information about the event that answers the following questions:

  • Which of the events above is the main or central event?
  • Should the event be cross-listed, e.g., acl with ijcnlp?

For cross-listing, we have a problem that sometimes there are true joint events, such as acl and ijcnlp. However, our naming convention forces one of these to be selected for the Anthology ID. We would like the event to be listed under both https://aclanthology.org/venues/acl/ and https://aclanthology.org/venues/ijcnlp. So we need a way to establish this.

Clone this wiki locally