Skip to content

v0.24.0

Latest

Choose a tag to compare

@percevalw percevalw released this 13 Nov 17:36

Changelog

Added

  • New attention pooling mode in eds.span_pooler, a softmax over the words of each span of scores given by a learned linear layer, computed on the flat word indices without padding
  • New eds.doc_classifier trainable component to predict attributes of a whole document, such as its type or its diagnoses. It is configured with one head per predicted attribute, given as a dict of options: a single-label head when a document carries exactly one label (cross-entropy loss, argmax decoding), a multi-label head (multi_label: true) when it carries a set of them (binary cross-entropy loss, decoded by thresholding the probabilities). The labels can be given as a list or as the path to a JSON or text file, and are saved with the model in a vocab.json. A document that lacks a gold value for a head does not supervise it, so partially labelled corpora can be mixed in the same training run. The per-label probabilities can also be stored in doc._.prob, optionally restricted to the top-k labels or to those above a threshold
  • New eds.doc_pooler embedding component, which aggregates the word embeddings of a document, or the [CLS] embeddings of its transformer windows with use_cls, into a single vector (mean, max, sum or a learned attention pooling, through the shared Pooling layer). eds.doc_classifier accepts any component returning one embedding per document
  • New eds.doc_classification metric to score document-level classification, for one or several Doc._ attributes at a time. Attributes holding a single value are scored with per-label, micro and macro P-R-F1 (micro then amounting to the accuracy); attributes holding a list of values are scored as a multi-label task. Documents with no gold value for an attribute are left out of its score
  • eds.transformer now also returns the [CLS] embedding of each window under the cls key, and the index of the context of each window under cls_contexts
  • New Training a document classifier tutorial, training a model that predicts both the principal diagnosis (single-label) and the associated diagnoses (multi-label) of a hospital stay

Fixed

  • eds.split no longer cuts annotated spans when splitting on regex or max_length, the new span_getter parameter (default ents) selects the spans to keep whole
  • Custom metrics in GenericScorer now predict on copies of the docs without their gold entities and span groups
  • edsnlp.train now collates training batches on CPU and moves them to the training device before the forward pass, so data workers (num_workers > 0) no longer open a CUDA context nor allocate batches on the GPU
  • GenericScorer now predicts on the device where training placed the model, so validation no longer moves a model trained with cpu=True to an available GPU, which made the next training step fail
  • Model packages built with code="dependency" or code="none" no longer copy the dependencies, optional dependencies, scripts and entry points of the project pyproject.toml, which belong to the code package
  • eds.transformer new_tokens now works with tokenizers that have no normalizer
  • eds.transformer now reloads the normalizer saved in tokenizer.json, since transformers 5 drops nested normalizers when it rebuilds tokenizers like CamemBERT, which changed the tokenization of saved models
  • eds.transformer no longer wraps the normalizer in one more layer of new_tokens replacements at each reload of a saved model
  • Numeric note_datetime values in milliseconds, microseconds or nanoseconds are now converted to seconds instead of raising an out of range error
  • eds.span_pooler now accepts embeddings that return flat words instead of padded contexts

Pull Requests

Full Changelog: v0.23.0...v0.24.0