Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major bugfix + new efficiency-focused functionality. #84

Merged
merged 34 commits into from
Jan 3, 2021
Merged

Conversation

patrick-kidger
Copy link
Collaborator

@patrick-kidger patrick-kidger commented Dec 23, 2020

Heyhey. I've been tweaking some of the SDE-GAN stuff (for the paper, not the example), and realised that it could be made much more efficient with library support... and what's the point of being a developer of the library if not for that?

  • Major bugfix: fixed calling solver.integrate twice in sdeint.py::integrate (!) (I can make this a separate PR if you want to put this in immediately.)
  • Added f_and_g, g_prod, and f_and_g_prod as functions that the SDE can define, which will be used in the solvers if possible. f and g no longer actually have to be defined if the solver doesn't need them. [1]
  • Improved efficiency:
    • logqp=True now takes advantage of this to define an f_and_g function that is more efficient than calling them separately. (Unrelated to what I'm doing, but why not.) [2]
    • Milstein and derivative-free Milstein now no longer have extra g computations.
  • Improved the error checking in check_contract to explicitly require a single batch dimension.

(N.B. This PR depends upon #85.)

[1] My use case for this: First of all, I'd like drift and diffusion to just be different final layers at the end of the same MLP, hence f_and_g. Second, for the discriminator, the diffusion ends up being a product of matrices. Doing (matrix-matrix)-vector would be inefficient, hence g_prod/f_and_g_prod.

[2] There's still some improvements that could be made here: for simplicity logqp still requires f/g/h to all be specified, without exploiting f_and_g or f_and_g_prod if they're available. The logical extension of this would be to also add f_and_g_and_h, f_and_g_prod_and_h as possible methods.

@patrick-kidger patrick-kidger mentioned this pull request Dec 24, 2020
15 tasks
@patrick-kidger patrick-kidger changed the base branch from master to dev December 24, 2020 03:50
@patrick-kidger patrick-kidger marked this pull request as ready for review December 24, 2020 22:34
@lxuechen
Copy link
Collaborator

lxuechen commented Jan 2, 2021

Thanks for this update! solver.integrate twice is a serious bug that might have been introduced by merging commits. Could you split this into a separate PR? We could merge this to master immediately.

Could you also split bullets 2-4 into 2 separate PRs? One for 2-3, and another for 4. That'll help a lot in terms of going through the code.

It might be too pesky to split this into 4 PRs, but I think it's reasonable to factor out the f_and_g functionality, as I think it would affect a few things.

patrick-kidger and others added 9 commits January 2, 2021 21:34
* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice. (#86)

* Bugfix for calling integrate twice.

* Increment version.

* Fix scipy version imcompatibility with Py3.6.

* Fix numpy incompatibility with Py3.6.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Simplify.

* Fix docstring.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>
@lxuechen
Copy link
Collaborator

lxuechen commented Jan 3, 2021

For efficiency purposes, I'm committing minor tweaks so we can merge. For the record, I'm rebasing from dev directly after merging #85, and I added the license for diagnostics/run_all.py

@lxuechen lxuechen merged commit 0b654e2 into dev Jan 3, 2021
@lxuechen lxuechen deleted the dev-call-optim branch January 3, 2021 07:18
lxuechen added a commit that referenced this pull request Jan 5, 2021
* Major bugfix. New efficiency improvements.

* bugfix

* bugfix for diagonal noise. Improved Milstein efficiency.

* Documentation update

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* bugfix for Milstein

* recursion fix

* Added tests for specialised functions and fixed bugs as a result

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice.

* Dev diagnostics fixes (#85)

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice. (#86)

* Bugfix for calling integrate twice.

* Increment version.

* Fix scipy version imcompatibility with Py3.6.

* Fix numpy incompatibility with Py3.6.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Simplify.

* Fix docstring.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Major bugfix. New efficiency improvements.

* bugfix

* bugfix for diagonal noise. Improved Milstein efficiency.

* Documentation update

* bugfix for Milstein

* Added tests for specialised functions and fixed bugs as a result

* Patch license.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>
patrick-kidger added a commit that referenced this pull request Jan 5, 2021
* Improve usability. (#81)

* Improve usability.

* Minor fix.

* Fixes from comments.

* Fixes.

* Dev diagnostics fixes (#85)

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice. (#86)

* Bugfix for calling integrate twice.

* Increment version.

* Fix scipy version imcompatibility with Py3.6.

* Fix numpy incompatibility with Py3.6.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Simplify.

* Fix docstring.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Major bugfix + new efficiency-focused functionality. (#84)

* Major bugfix. New efficiency improvements.

* bugfix

* bugfix for diagonal noise. Improved Milstein efficiency.

* Documentation update

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* bugfix for Milstein

* recursion fix

* Added tests for specialised functions and fixed bugs as a result

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice.

* Dev diagnostics fixes (#85)

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice. (#86)

* Bugfix for calling integrate twice.

* Increment version.

* Fix scipy version imcompatibility with Py3.6.

* Fix numpy incompatibility with Py3.6.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Simplify.

* Fix docstring.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Major bugfix. New efficiency improvements.

* bugfix

* bugfix for diagonal noise. Improved Milstein efficiency.

* Documentation update

* bugfix for Milstein

* Added tests for specialised functions and fixed bugs as a result

* Patch license.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Minor formatting.

* Simplify detach logic in core adjoint functions.

* Improve usability. (#81)

* Improve usability.

* Minor fix.

* Fixes from comments.

* Fixes.

* Dev diagnostics fixes (#85)

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice. (#86)

* Bugfix for calling integrate twice.

* Increment version.

* Fix scipy version imcompatibility with Py3.6.

* Fix numpy incompatibility with Py3.6.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Simplify.

* Fix docstring.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Major bugfix + new efficiency-focused functionality. (#84)

* Major bugfix. New efficiency improvements.

* bugfix

* bugfix for diagonal noise. Improved Milstein efficiency.

* Documentation update

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* bugfix for Milstein

* recursion fix

* Added tests for specialised functions and fixed bugs as a result

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice.

* Dev diagnostics fixes (#85)

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Bugfix for calling integrate twice. (#86)

* Bugfix for calling integrate twice.

* Increment version.

* Fix scipy version imcompatibility with Py3.6.

* Fix numpy incompatibility with Py3.6.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* fixes for diagnostics

* Added every applicable method to every diagnostic. Added ito_general. Added run_all.

* fixed levy area appproximation choices

* added no_grad wrappers

* switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders

* bugfix for Ex2

* increased Ex complexity slightly

* introduced the neural SDEs as separate without replacing Ex1-3

* recursion fix

* Renamed Ex1->ExDiagonal etc.

* Simplify.

* Fix docstring.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Major bugfix. New efficiency improvements.

* bugfix

* bugfix for diagonal noise. Improved Milstein efficiency.

* Documentation update

* bugfix for Milstein

* Added tests for specialised functions and fixed bugs as a result

* Patch license.

Co-authored-by: Xuechen Li <12689993+lxuechen@users.noreply.github.com>

* Minor formatting.

* Simplify detach logic in core adjoint functions.

* Increment version count.

Co-authored-by: Patrick Kidger <33688385+patrick-kidger@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants