Be notified of new releases
Create your free GitHub account today to subscribe to this repository for new releases and build software alongside 28 million developers.Sign up
Changes since UDPipe 1.1.0:
- On-demand loading of models in REST server, with a pool of least recently used models.
- Make GRU tokenizer dimension configurable (16, 24, 64 supported).
- Track paragraph boundaries even under
- Support experimental sentence segmentation using jointly both the tokenizer and the parser.
- Add EPE output format.
- Make default model in REST server explicit.
- Support pre-filling according to URL params in the webapp.
Changes since UDPipe 1.0.0:
- Morphodita_parsito models (now version 3) require at least UDPipe version 1.1.0.
- CoNLL-U v2 format is supported. Notably spaces in forms and lemmas are now allowed, as are empty nodes.
- Support options for
- Preserve all spacing when tokenizing.
- Optionally generate document-level token ranges in the original text.
- Optionally respect given segmentation during tokenization.
- Tokenizer can be trained to allow spaces in tokens (default if there are forms with spaces in the training data).
- Parser can be trained to return always one root per sentence (default).
input_formatAPI to allow inter-block state (for correct tracking of inter-sentence spaces and document-level offsets).
output_formatAPI to support begin/end document marks and to allow state in the output_format instance (to allow numbering output sentences, for example).