Repository navigation
Changelog
Added
- New
attentionpooling mode ineds.span_pooler, a softmax over the words of each span of scores given by a learned linear layer, computed on the flat word indices without padding - New
eds.doc_classifiertrainable component to predict attributes of a whole document, such as its type or its diagnoses. It is configured with one head per predicted attribute, given as a dict of options: a single-label head when a document carries exactly one label (cross-entropy loss,argmaxdecoding), a multi-label head (multi_label: true) when it carries a set of them (binary cross-entropy loss, decoded by thresholding the probabilities). The labels can be given as a list or as the path to a JSON or text file, and are saved with the model in avocab.json. A document that lacks a gold value for a head does not supervise it, so partially labelled corpora can be mixed in the same training run. The per-label probabilities can also be stored indoc._.prob, optionally restricted to the top-k labels or to those above a threshold - New
eds.doc_poolerembedding component, which aggregates the word embeddings of a document, or the[CLS]embeddings of its transformer windows withuse_cls, into a single vector (mean,max,sumor a learnedattentionpooling, through the sharedPoolinglayer).eds.doc_classifieraccepts any component returning one embedding per document - New
eds.doc_classificationmetric to score document-level classification, for one or severalDoc._attributes at a time. Attributes holding a single value are scored with per-label, micro and macro P-R-F1 (micro then amounting to the accuracy); attributes holding a list of values are scored as a multi-label task. Documents with no gold value for an attribute are left out of its score eds.transformernow also returns the[CLS]embedding of each window under theclskey, and the index of the context of each window undercls_contexts- New Training a document classifier tutorial, training a model that predicts both the principal diagnosis (single-label) and the associated diagnoses (multi-label) of a hospital stay
Fixed
eds.splitno longer cuts annotated spans when splitting onregexormax_length, the newspan_getterparameter (defaultents) selects the spans to keep whole- Custom metrics in
GenericScorernow predict on copies of the docs without their gold entities and span groups edsnlp.trainnow collates training batches on CPU and moves them to the training device before the forward pass, so data workers (num_workers > 0) no longer open a CUDA context nor allocate batches on the GPUGenericScorernow predicts on the device where training placed the model, so validation no longer moves a model trained withcpu=Trueto an available GPU, which made the next training step fail- Model packages built with
code="dependency"orcode="none"no longer copy the dependencies, optional dependencies, scripts and entry points of the projectpyproject.toml, which belong to the code package eds.transformernew_tokensnow works with tokenizers that have no normalizereds.transformernow reloads the normalizer saved intokenizer.json, since transformers 5 drops nested normalizers when it rebuilds tokenizers like CamemBERT, which changed the tokenization of saved modelseds.transformerno longer wraps the normalizer in one more layer ofnew_tokensreplacements at each reload of a saved model- Numeric
note_datetimevalues in milliseconds, microseconds or nanoseconds are now converted to seconds instead of raising an out of range error eds.span_poolernow accepts embeddings that return flat words instead of padded contexts
Pull Requests
- Attention pooling by @percevalw in #445
- Release fixes by @percevalw in #535
- Training cpu collate by @percevalw in #537
- fix: accept embeddings that return flat words in eds.span_pooler by @percevalw in #538
- Fixes 0.24 by @percevalw in #539
- fix: keep the model on its training device during validation by @percevalw in #540
- feat: add trainable document classifier by @LucasDedieu in #423
Full Changelog: v0.23.0...v0.24.0