Skip to content

wtsnjp/MioGatto

main
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
lib
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

MioGatto: Math Identifier-oriented Grounding Annotation Tool

System requirements

  • Python3 (3.9 or later)
  • A Web Browser with MathML support (for the GUI annotation system)

Installation

The dependencies will be all installed with one shot:

python -m pip install -r requirements.txt

In case you don't want to install the dependencies into your system, please consider using venv.

Project structure

Files in this repository

All the components of MioGatto is included in this repository:

  • lib/ contains the project library.
  • server/ contains the implementation of the server.
  • client/ contains the implementation of the client.
  • tools/ contains our utility Python scripts.

Files not in this repository

On the other hand, the annotation data is not included in this repository due to the NDA constrain for the arXMLiv dataset. The data is licensed to SIGMathLing members as Dataset for Grounding of Formulae. Please consider joining SIGMathLing to acquire the dataset.

  • arxmliv/ contains the original documents from the arXMLiv dataset
  • data/ contains the annotation data
  • sources/ contains the preprocessed documents

Annotator's guide

For the guide with GIF animation, please refer to our Wiki:

Prepare the input and analyze the annotated data (Advanced)

The Python scripts under the tools directory are mostly for the developers for the grounding dataset. The --help (-h) option is available for all scripts and should provide guides to their basic usages.

Preparing data

As mentioned above, the HTML5 files in the arXMLiv dataset are suitable as the input document for MioGatto. Alternatively, you can provide the equivalent HTML5 files from LaTeX sources by using LaTeXML:

latexmlc --preload=[nobibtex,ids,mathlexemes,localrawstyles]latexml.sty --format=html5 --pmml --cmml --mathtex --nodefaultresources --dest=<output HTML file> <input TeX file>

Then you can give the HTML5 files to our preprocess script:

python -m tools.preprocess <HTML file>

This will output the preprocessed HTML file to the sources/ and generate the initialized JSON files for the annotation to the data/ by default. Please refer to the help message for the options.

python -m tools.preprocess -h

Analysing the annotation results

For the basic analyses for annotation data, execute:

python -m tools.analyzer <paper id>

Some supplemental files including graph images will be saved in the results directory as default.

Similarly, analyses for the sources of grounding annotation can be performed with the tools.sog script.

python -m tools.sog <paper id>

To calculate agreements between data by two annotators, execute:

python -m tools.agreement --target=<path to annotator's data dir> <paper id>

Developing client

The client is developed with TypeScript. All development tools will be installed with:

cd client
npm install

To compile the client source client/index.ts, execute the following in the client directory:

npm run build

Publications

Acknowledgements

This project has been supported by JST, ACT-X Grant Number JPMJAX2002, Japan.

License

Copyright 2021 Takuto Asakura (wtsnjp)

This software is licensed under the MIT license.

Third-party software


Takuto Asakura (wtsnjp)