New Algorithm

[ds300]

Up until now, DerivableJS used a mark-and-sweep style algorithm to both propagate change and free unused memory.

This had a couple of minor performance implications, and one obscure-but-serious memory leak vulnerability (see #36).

This reimplementation of DerivableJS' core data structures removes the mark-and-sweep stuff, replacing it with a significantly simpler system inspired by Clojure's epochal STM.

The practical implications are:

• atoms never ever store references to derivations. Only to reactors. This is ideal, and solves the memory management issue once and for all.
• the time complexity of updating atoms is now linear in the number of dependent reactors rather than in the number of dependent (derivations + reactors), and should be much more performant due to cache locality goodness.
• the cost of dereferencing derivables should be either similar or faster than before, but I'll have to do benchmarking to be sure of this.
• smaller file size (.min.gz has gone from 4,330 bytes down to 3,718 bytes. Makes all the difference.)

The progress of this reimplementation is as follows:

• typescript prototype of epochal system to verify viability
• retrofit DJS with epochal system
• get all the unit tests passing
• trial epochal system in real-world projects already using DJS
• modernize code base (npm build scripts, es6 modules + rollup, CI)
• do white box testing. get 100% test coverage.
• check performance and rework if necessary
• modify read-me to talk about push-pull as opposed to mark-and-sweep, like Agera
• release

[willmruzek]

Does the flux challenge you've written use CQRS/redux?

[ds300]

Neither, although the approach is very similar indeed. The main difference from Redux is that the actions are implicit so they're not serializable. The reducer pattern is more-or-less intact. Porting the code to redux should be super easy.

[willmruzek]

I'll just give that a look and maybe convert it to Redux as an exercise.

[oskbor]

So close now! When will this land?

[ds300]

Not sure. I was planning to make nice new docs and a respectable-looking landing page before pushing out a 1.0.0 release. This was partly done, then summer happened and I got put on a very fulfilling project at work so my motivation to code on evenings and weekends has been waning.

However I've lately been learning the value of MVPs and thinking that it would be better to put out something more lean (docs-wise) to begin with at least. But still need to wait until I have some spare time at a weekend, which is looking like early October right now.

[zerkalica]

Can you shortly explain about breaking changes in feature release? What should be avoided in the current version?

[ds300]

The main breaking change is that the Reactor class and its associated API (Derivation#reactor) have gone away.

The top-level withEquality function was removed too.

Behaviour-wise, laziness is now only provided for 'active' derivations (which are upstream of an 'active' reactor). I might still change this back to the old behaviour (always lazy) before 1.0.0 is released if I can find a way to do it without impacting performance for active derivations.

I think everything else is the same.

[zerkalica]

I try to test 1.0.0-beta11, but catch strange bug: derive does not cache result, each get recalculate value again. It's some optimizations or bug?

        const a = atom({p: 1})

        const b = a.derive((p) => {
            console.log('test')
        })

        b.get() // "test"
        b.get() // error: "test" again

[ds300]

This is expected behaviour. Derivations are now only cached when the
derivation is being actively used by a downstream reactor. This enabled
some good performance enhancements, but I still would like to bring caching
back for all derivations somehow before it leaves beta.

On Tue, 4 Oct 2016, 01:22 Stefan, notifications@github.com wrote:

I try to test 1.0.0-beta11, but catch strange bug: derive does not cache
result, each get recalculate value again. It's some optimizations or bug?

    const a = atom({p: 1})

    const b = a.derive((p) => {
        console.log('test')
    })

    b.get() // "test"
    b.get() // error: "test" again

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#37 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABL1qWVHBIlca888kMTZWW2kfZTi9xFDks5qwY5NgaJpZM4IOY1p
.

[andreypopp]

but I still would like to bring caching
back for all derivations somehow before it leaves beta.

Any idea how to achieve that? I'd be interested to work on a pull request.

[ds300]

@andreypopp The epochal system I started this PR with would work if kind-of superimposed over the current setup. That could end up being quite complex to implement and test, but it should add only a small amount of constant overhead. A PR would be super cool if you're feeling intrepid :) Let me know if you want any help figuring stuff out!

[oskbor]

How is the rewrite coming along? ☺️

[ds300]

Oh gosh. Well the code is pretty tight (currently released as beta), but I haven't had the extra-curricular moxie to finish off the docs and landing page in recent months. I've got a big 5-week chunk of holiday coming up in February; maybe that with replenish my OSS mana.

…

On 14 Dec 2016 8:14 am, "Oskar Börjesson" ***@***.***> wrote: How is the rewrite coming along? ☺️ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABL1qTkIsixvfwaiZ7S97VvPVB2q0_Mvks5rH5dygaJpZM4IOY1p> .

[oskbor]

Ill send you a mana potion for Christmas 🎅 On Wed, Dec 14, 2016, 10:10 David Sheldrick <notifications@github.com> wrote:

…

Oh gosh. Well the code is pretty tight (currently released as beta), but I haven't had the extra-curricular moxie to finish off the docs and landing page in recent months. I've got a big 5-week chunk of holiday coming up in February; maybe that with replenish my OSS mana. On 14 Dec 2016 8:14 am, "Oskar Börjesson" ***@***.***> wrote: > How is the rewrite coming along? ☺️ > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#37 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/ABL1qTkIsixvfwaiZ7S97VvPVB2q0_Mvks5rH5dygaJpZM4IOY1p > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPNfWDu5rMCD8MiAFj587g2wiRxpZx6ks5rH7KWgaJpZM4IOY1p> .

[ds300]

This PR is ridiculous. I'm just gonna merge it into master and start doing feature and release branches.