Numba for speed up

[amoodie]

Highlights

I was messing about and tried to see what kind of speed-up we could get with just using Numba.

I profiled the following loop on develop (i.e., the tests/time_water.py script):

delta = pyDeltaRCM()
start = time.time()
for _t in range(0, 10):
    print("timestep:", _t)
    delta.update()
end = time.time()

for an elapsed time = 1.2779 min.

With some really simple changes to the water routing routines (only changed one of three list(map(lambda(... calls) and change of random picking in a few places, I get an elapsed time 0.5052 min.

This is really promising for #1, I think.

Question

How much do we care about exact consistency, down to the random number sequences?

These changes use the Numba random number generator (rather than the numpy rng), and so they produce different results. For example, the tests/test_output_consistency.py test fails with the Numba implementation. The expected bed elevation at a small slice after one update():

what is returned with Numba:

array([[ 0.        , -1.        , -1.        , -1.        ,  0.        ],
       [-0.92157906, -0.9095978 , -0.8295175...99838   , -0.99919   ],
       [-1.        , -1.        , -1.        , -1.        , -1.        ]],

what is expected:

array([[ 0.        , -1.        , -1.        , -1.        ,  0.        ],
       [-0.8469125 , -0.82371545, -0.8295175...   , -0.9981775 , -0.9989875 , -0.9997975 ],
       [-1.        , -1.        , -1.        , -1.        , -1.        ]]))

This ultimately would be an issue for any C implementations too. My feeling is that it's fine to break exact consistency between versions, as long as we note it in a changelog. I think breaking behavior changes of the model are much more important, and require a higher-level need to allow.

Anyway, I don't want to merge this now, just looking for some disucssion. @elbeejay @ericbarefoot @ammilten

Also, sorry the commit history will shorten when #26 is merged.

[amoodie]

On second thought, reproducible runs are important, so I'll look at ways to make sure the runs are deterministic from a seed.

[amoodie]

okay, well turns out that was pretty easy, just have to make sure the rng seed is set in a jitted function, and then only and always call np.random from jitted functions.

[elbeejay]

Seems very similar in intent to the shared_tools.py file we have. Any reason they can't be combined? (I have no attachment to either name)

[amoodie]

yeah this would eliminate the shared_tools file. I have this done in another commit, but haven't pushed it. I'll do some of the jitting again, on top of Eric's changes.

[ericbarefoot]

It turns out that numba doesn't support jitting classes in the way that we would like to use them (for example, they don't import inheritance) and so the way you've done it seems like the simplest way forward for us.

[elbeejay]

Idea is quite nice, that is a pretty significant speed up indeed.

[ericbarefoot]

Since we merged #36, I'm going to go ahead and close this. Thanks for the great idea and all the good investigation and development that this PR motivated.