Skip to content
Branch: master
Commits on Jun 18, 2019
  1. Simplify working with keyword arguments (#176)

    ararslan committed Jun 18, 2019
    Currently, `at-explicit_intercepts` generates a new function when given
    keyword arguments. It uses a function called `kwfname` to determine what
    the name of this new function should be. This is kind of weird and
    convoluted. Instead, we can just store the keyword arguments used for
    the call as part of the `Branch` type, and make sure they're passed
    around as appropriate. This allows us dump `kwfname` and all of that
Commits on Jun 6, 2019
  1. Fill out the documentation page on custom sensitivities (#170)

    ararslan committed Jun 6, 2019
    * Fix a couple of typos in the details page
    * Fill out the documentation on custom sensitivities
    * Clarify explicit_intercepts in the complex example
  2. Checkpointing (#171)

    willtebbutt committed Jun 6, 2019
    * An initial hack
    * Add some tests
    * Some extra documentation
    * kwargs + fieldcount check + remove Arg{1} method
    * Commit Lyndon's suggestion
    Co-Authored-By: Lyndon White <>
    * Fix cholesky test
Commits on May 30, 2019
  1. Merge pull request #172 from invenia/aa/fdm

    ararslan committed May 30, 2019
    Updates for FDM v0.6.0
  2. Set version to v0.12.1

    ararslan committed May 29, 2019
  3. Updates for FDM v0.6.0

    ararslan committed May 29, 2019
Commits on May 23, 2019
  1. Add some documentation on the design of Nabla (#169)

    ararslan committed May 23, 2019
Commits on May 21, 2019
  1. Set Project.toml version to 0.12.0 (#168)

    ararslan committed May 21, 2019
  2. Refactor SVD and getproperty sensitivities to use U, S, and V (#164)

    ararslan and wesselb committed May 21, 2019
    Consider a computation which uses multiple fields of `svd(X)`. The
    forward graph looks like:
      F = svd(X)
       / | \
    F.U F.S F.V
    When we perform the reverse pass, traversing this graph back upward,
    `F.U`, `F.S`, and `F.V` will try to update the sensitivity for the
    initial `svd` call. That means we'll be updating with multiple,
    potentially nonzero values passed to `∇(svd, ...)` at once in order to
    properly update. The current machinery does not allow for that; instead,
    we try to update the `svd` sensitivity one component at a time, which
    can cause dimensionality issues when `X` and its SVD components have
    different sizes.
    To fix this, we'll have `∇(getproperty, ...)` return a `NamedTuple` with
    `U`, `S`, and `V` fields, one of which will be populated with the
    incoming sensitivity, which corresponds to one of these fields, and the
    rest of them are set to zero. Then, when we go to update this
    `NamedTuple`-based sensitivity, we simply update the relevant component.
    Then, when `∇(svd, ...)` receives a `NamedTuple`, it will have all of
    the information it needs to update.
    Co-authored-by: Wessel <>
Commits on May 14, 2019
  1. Remove incorrect binary linear algebra sensitivities (#166)

    ararslan committed May 14, 2019
    These definitions are not correct. Furthermore, the tests which were
    intended to exercise these methods in actuality do not; they too end up
    calling the `op(::AbstractMatrix, ::AbstractMatrix)` sensitivity which
    *is* correct, which is why we never noticed this.
  2. Refactor generic linear algebra tests (#167)

    ararslan committed May 14, 2019
    This refactors the generic linear algebra tests to use test sets, each
    with a separate RNG state. This avoids issues where running a particular
    group of tests multiple times fails, apparently due to the RNG state.
    A particular RNG seed was chosen for the unary tests that allows them
    not to fail, and seeds for other tests were chosen arbitrarily.
    This commit also has a (somewhat) unrelated change that simplifies the
    sensitivity definition for `-` from `map`ping `-` to just applying it.
Commits on May 13, 2019
  1. Add matrix exponential sensitivities (#161)

    ararslan committed May 13, 2019
    Scalar exponential is covered by the DiffRules integration.
    Currently only implemented for symmetric inputs.
Commits on May 9, 2019
  1. Use named tuples for more reliable dispatch in SVD (#163)

    ararslan committed May 9, 2019
    Currently we're assuming that a `getproperty` call for `SVD` objects
    will return a vector for `S`, an `Adjoint` for `V`, and any other kind
    of matrix for `U`. This is a bit fragile and requires making a copy for
    `V` to ensure it's an `Adjoint`.
    We can do better! If we have `getproperty`'s sensitivity return a
    singleton `NamedTuple` containing only the requested property, we can be
    sure of exactly which property `svd`'s sensitivity is getting.
Commits on May 7, 2019
  1. Merge pull request #159 from invenia/aa/ref

    iamed2 committed May 7, 2019
    Allow working with Refs
  2. Test transpose wrappers for generic linalg sensitivities (#162)

    ararslan committed May 7, 2019
    Currently, when passed `Transpose` or `Adjoint` as a type, the test-only
    `trand*` functions return a regular matrix. Because of this, we were not
    testing the sensitivity definitions specific to those types, but rather
    we were just testing the `AbstractMatrix` definitions a bunch of times.
    This change ensures that we're actually hitting those code paths.
Commits on Apr 23, 2019
  1. Allow working with Refs

    ararslan committed Apr 23, 2019
    The existing sensitivity definition for `getindex` is generic enough to
    work for `Ref`s out of the box. We just need to define the zero- and
    one-dimensional container functions for `Ref`s.
Commits on Apr 22, 2019
  1. Fix FDM version bounds, bump Nabla version to 0.11.1 (#157)

    ararslan committed Apr 22, 2019
    Currently FDM's version is restricted to 0.1.0 or 0.2.0, which doesn't
    really make sense; any version of FDM since 0.1.0 should be fully
    compatible with Nabla. Imposing the version restriction makes later
    versions of FDM uninstallable in the global environment if Nabla was
    installed first.
    This change sets FDM to have a lower bound only, it frees DualNumbers to
    have a lower bound only rather than a fixed version, it adds an explicit
    caret to the Julia bound (non-functional change as carets are implicit),
    and it sets Nabla's own version to 0.11.1 in preparation for a new tag.
Commits on Apr 16, 2019
  1. Remove REQUIRE, add version to Project.toml (#154)

    ararslan committed Apr 16, 2019
    Package versions are now registered via Registrator rather than METADATA
    and Attobot, which means REQUIRE files are no longer needed and that
    versioning information is based on that declared in Project.toml files.
Commits on Apr 15, 2019
  1. Point to Invenia bot's Appveyor (#152)

    iamed2 authored and ararslan committed Apr 15, 2019
  2. Add sensitivities for fill (#150)

    ararslan committed Apr 15, 2019
    Fixes #141.
  3. Fix map extension for Nodes (#151)

    ararslan committed Apr 15, 2019
    Consider the function call `map(f, xs...)`. We want to use Nabla if any
    of the `xs` are `Node`s. Unfortunately that isn't expressible in a
    coherent way in Julia after 0.6. What we currently do introduces method
    ambiguities, so we overwrite the Base method for `map` on numeric arrays
    to call `invoke`. That workaround itself causes other ambiguities for
    packages which also extend `map`, e.g. StaticArrays, as observed in
    issue #136.
    The solution here is to limit what we seek to detect among the
    aforementioned `xs`. If we limit ourselves to, say, the first 10 iterable
    arguments to `map` for extension with `Node`s, everything is much
    simpler and we don't need to do any method overwriting. As currently
    implemented, the cutoff is 10, though it can be easily changed to any
    number; 10 just seemed a reasonable enough cutoff.
    Fixes #136.
Commits on Apr 13, 2019
  1. Add an internal helper function to do more in-place updating (#145)

    ararslan committed Apr 13, 2019
    Currently the `∇(x̄, f, Arg{N}, args...)` method updates `x̄` with the
    result of `∇(f, Arg{N}, args...)`. This is done in-place for some
    functions `f` but not all. In the case of the fallback method, we can
    use dispatch to determine whether it's safe to do this in-place, thereby
    hopefully saving some allocations.
  2. Fix a very unlikely corner case for get_output (#153)

    ararslan committed Apr 13, 2019
    Currently, when passing the `get_output` keyword argument to `∇`, you
    only actually get the output if evaluating the function on the arguments
    converted to nodes returns a node. With this change, `get_output` always
    gives you the output of the function call, whether it's a node or not.
Commits on Apr 11, 2019
  1. Implement sensitivities for mean (#149)

    ararslan committed Apr 11, 2019
Commits on Apr 6, 2019
  1. Merge pull request #148 from invenia/ep/isapprox

    eperim committed Apr 6, 2019
    Break `isapprox` method ambiguity
  2. Test new isapprox method

    Eric Perim
    Eric Perim committed Apr 6, 2019
  3. Use explicit_intercepts for sum (#146)

    ararslan committed Apr 6, 2019
    This allows us to consume and record `sum` calls as such, rather than
    simply recording that `mapreduce` was called. This should make reading
    the tape a bit more informative, plus it permits specializations on the
    particular mapping function `f` passed in `sum(f, x)`.
    Because of the aforementioned specialization feature, we can now add a
    specialization for `sum(abs2, x)`, which is a common pattern and is
    equivalent to `dot(x, x)` and `sum(x.^2)`.
  4. Break method ambiguity

    Eric Perim
    Eric Perim committed Apr 6, 2019
  5. Make printing of Tapes more verbose (and informative) (#147)

    ararslan committed Apr 6, 2019
    Currently the printing of `Tape`s leaves a bit to be desired in terms of
    examining the actual contents, due in part to the lack of separation
    between position and value, and due to how values are printed. With this
    change, values are printed using `show`, which is the intended way to
    display things nicely, and positions are wrapped in brackets with a
    colon. Additionally, a header is printed that lists the number of tape
    elements, in the same manner as e.g. `Dict`s.
Commits on Mar 25, 2019
  1. Add line information to more generated code (#144)

    ararslan committed Mar 25, 2019
    In some places we are willfully omitting line information by passing
    `nothing` to certain `Expr`s, and in others we're generating expressions
    that are syntactically valid but are not what Julia itself produces.
    Part of that is the omission of line information.
    This changes the function definition expressions from things like
    f(x::Node{Int}) = Branch(f, (x,), getfield(x, :tape))
    function f(x::Node{Int})
        #= line info here =#
        Branch(f, (x,), getfield(x, :tape))
    "What's wrong with the former?" you might ask. Nothing in principle,
    though if you were to write that in your own code, Julia parses it as
    f(x::Node{Int}) = begin
        #= line info here =#
        Branch(f, (x,), getfield(x, :tape))
    For ease of visual parsing, we'll change it to use `:function`
    expression heads, i.e. long-form function definitions, as it's
    functionally equivalent.
    The benefit of having line information is for backtraces. If something
    goes wrong, the user should now be able to get a better idea of where a
    problem is, rather than Julia providing a backtrace that just says
    something along the lines of "in Nabla."
Commits on Mar 22, 2019
  1. Add sensitivities for float (#143)

    ararslan committed Mar 22, 2019
    `float` acts on arrays and scalars, turning integers into floats of the
    corresponding size (e.g. `float(::Int32)` -> `Float32`) and is a no-op
    for floats. This same behavior is applied when differentiating.
Commits on Mar 20, 2019
  1. Fix indexing + add regression test (#140)

    willtebbutt committed Mar 20, 2019
  2. Merge pull request #142 from invenia/aa/no-mixed-plus

    ararslan committed Mar 20, 2019
    Remove sensitivities for + and - on mixed array/scalar
  3. Drop support for Julia 0.7

    ararslan committed Mar 20, 2019
    Also add 1.1 to CI.
Commits on Mar 19, 2019
  1. Remove Distributions rand workaround

    ararslan committed Mar 19, 2019
    This definition is no longer necessary and causes method ambiguities.
You can’t perform that action at this time.