-
Notifications
You must be signed in to change notification settings - Fork 176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache probe data on access. #1175
Conversation
LGTM. |
Awesome, that's a huge speedup! 🌈 Looking through the diff, will this give incorrect results in the following case? with nengo.Simulator(net) as sim:
sim.run(0.1)
d = sim.data[probe]
sim.run(0.1)
assert d.shape != sim.data[probe].shape If so, then I can see two possible solutions... either invalidate the cache in |
Good catch! |
Nice, that's a better fix for sure! I'll take a closer look now, likely merge shortly. |
p = nengo.Probe(ens) | ||
|
||
dt = 0.001 | ||
with nengo.Simulator(model, dt=dt) as sim: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that this should be RefSimulator
. I've fixed it locally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I though of that when writing the function signature, but then I forgot when writing the actual code. 😨
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haha, force of habit 😉 🌈
This makes repeated access faster. Fixes #1076.
fec481d
to
ceaf387
Compare
Motivation and context:
Repeated access to probes is slow (see #1076). This caches the read-only NumPy array to make it faster.
How has this been tested?
Ran this script adopted from @arvoelke in #1076 on master and this branch:
The repeated access block took 17s on master vs. 0.017s on this branch. The second single access block was reduced to 0.00017s from 0.017s. (I don't know what's up with all the 17s ... 👻 )
How long should this take to review?
Where should a reviewer start?
The usual place: “Files changed”
Note that the cache does not need to use weak references because the
raw
dict in theProbeDict
class doesn't use weak references either. Thus, as long as theProbeDict
instance is around, there will always be a strong reference to each key.Types of changes:
Checklist: