Merge v9 into thinc.ai
#931
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge `master` into `v9`
* `NumpyOps`: Remove unused/vestigial free functions, reuse functions in `Ops` * Remove superfluous `typedef`
* disable mypy run for Python 3.10 * dot
* `CBlas`: Add `sscalv` * `NumpyOps`: Replace usage of `.linalg` with `numpy` and `BLAS` calls * Remove vestigial/mostly unused `backends.linalg` module * Use BLAS notation for `sscal`, add `dscal`
* `NumpyOps`: Move `blis` detection to `compat` module, replace `blis.cy.gemm` calls with `CBlas` calls * `NumpOps`: Call `self.cblas()` instead of directly instantiating `CBlas` * `CBlas`: Add `dgemm` * `NumpyOps`: Use `CBlas.?gemm` in `gemm`
Sync v9 with the latest from master
* return logloss instead of squared differrence * check whether to comput binary or categorical loss value * function to apply label smoothing to 2d array * force exclusive classes * formatting * mypy debug * bugfix * compare cross entropy to torch * fix type and error message * updating cross-entropy tests * all categorical crossentropy tests updated * sequence crossentropy test * rearrange if statements * sequence ce negprefix test start * all tests for (sequence) cross entropy * use CategoricalCrossentropy as loss * don't run conversion and validation twice in __call__ * add type for truths in convert_truths (thnx @ richardpaulhudson) * fix one-hot check and no unexpected error branch * cupy support for torch comparison * import floats2d * hopefully right type to pass old torch cross-entropy * nonstrict sum to 1 * typo * remove redundant work for sequential cross entropy * type typo * fix imports * remove misleading comments * assertion for clarity * add back mistakenly removed imports * throw error rather than assert * legacy versions and tests for crossentropy + sequential * type fix * Update thinc/legacy/loss.py Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> * legacy cross-entropy import through registry * no legacy test module * type fix * hacking types for mypy * return type * Update thinc/legacy/loss.py Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> * Update thinc/legacy/__init__.py Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> * initial functional sparse ce los * separate functionality for SparseCE and CategoricalCrossentropy * fix missing value type * correcting label smoothing param contraint * test new label smooth validation error * less than 0 input validation * string concat * small update to error msg * fix max smoothing coefficient * double check error message * Categorical and Sparse factories and tests * Update thinc/util.py Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> * update test with less strict match * Fix types, pair-hacked with @kadarakos * (Sparse)CategoricalEntropy: support Ragged guesses Since we can encoder sequences as Ragged, this could replace (Sparse)SequenceCategoricalEntropy. * follow updated api * Update thinc/util.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * indent fix * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * remove unnecessary list copy * add type to truths * fix missing assignment * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * rever suggestion * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/legacy/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/tests/test_loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/util.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Update thinc/loss.py Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * move check1d out of loss and more general signature * mypy fix * SparseCE rename Co-authored-by: Kádár Ákos <akos@onyx.uvt.nl> Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> Co-authored-by: Daniël de Kok <me@danieldk.eu> Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Sync `master` into `v9`
…ion#809) * Bring back support for missing labels to legacy cross entropy * Use `missing_value` to detect missing values * Typing fixes
…221220 Merge master into v9
…n#804) * Give schedulers access to the key, step, and last eval score Before this change schedules were generators that generate a value for each training step. This, however has the limitation that scheduler cannot use other information that is available in the optimizer such as the parameter key. This information is useful for e.g. discriminative learning rates, where certain parameters are on a different schedule than others. To accommodate passing additional information, this change converts schedules to callables. These callables are passed the training step, the parameter key, and the last evaluation score (when available). Traditional scalar and generated schedules are converted to callables by the optimizer for compatibility. * Fix use of the `t` parameter where used in the schedules Also add tests, so that doesn't break again. * Fixes from @shadeMe * Call _schedule_args once * Make Optimizer.step private * Fix two missed step uses in tests * Float fix Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * Fix schedule call * Move `ScheduleCallable` to `thinc.types` * Move from callables to a `Schedule` class The new learning rate functionality used `Callable`s. However, the issue with callables it that they cannot be pickled. This is problematic, because schedules can end up in spaCy pipelines (e.g. through the optimizer associated with the `Language` object). This change solves this issue by refactoring the schedules into regular objects. This now works similar to Thinc `Model`s -- there is a new `Scheduler` class which can be constructed with composition. I tested the changes with spaCy and pickling as well as usin existing configurations works. * Remove stray `runtime_checkable` import * Apply suggestions from code review Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
* Add `Schedule.to_generator` This method turns a `Schedule` into a generator by feeding the `Schedule` steps with a given starting step and increment. * Doc fix Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> * docs: add default values for Schedule.to_generator * fix anchor Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com> Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
…20230112 Merge `master` into `v9`
Set version to v9.0.0.dev2
* Add plateau.v1 schedule This schedule yields values from the wrapped schedule, exponentially scaled by the number of times optimization has plateaued. * Fix anchor * Remove stagnant wording in favor of plateaus * Type annotation: last_score is Optional Also set a default value, to that the schedule does not fail when the last_score argument is not provided. * Update docs to clarify that passing last_score is not mandatory * Document plateau arguments
* fix valid label smoothing parameter * remove print * fix typo * ensure number of classes larger than one
…-20230322 Sync v9 with master
…ter-1 Update v9 from master
* Revert "Cross entropy fix (explosion#647)" This reverts commit c8ac07f. * Cherry pick MPS Torch bug to get CI unstuck
…rge-master-20240109
PR explosion#897 fixed the dtypes in strings2arrays, however also broke strings2arrays for batches with sequences if inequal lengths.
The way we used local thread storage before did not typecheck, since we assigned to `Thread`. Thread local storage can be a global variable, the state of this object will be different per thread.
* remove slow marker from basic tagger test * fix strings2array * isort
…aster-20240109 Merge master into v9
* Fix `cupy.cublas` import Reported in explosion#920. * Update mypy to work with recent Torch versions * CI: Do not run MyPy on Python 3.6/3.7.
…er-20240212 Merge `master` into `v9`
This change adds `AppleOps` to Thinc, to ensure that the AMX unit is always used on Apple Silicon Macs. Before this change, a user would get much worse performance if they forgot to install `thinc-apple-ops`. The `apple_ops` and `_accelerate` modules are built conditionally. When detecting the best CPU implementation, we rely on a `try...except` import to determine whether Apple ops are available. Even though x86_64 Macs do not have an AMX unit, Accelerate is competitive with BLIS, so it does not hurt to enable Apple ops on all Macs.
👷 Deploy request for thinc-ai pending review.Visit the deploys page to approve it
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Warning: do not squash
Types of change
Docs
Checklist