Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only cache things in a registry #1068

Merged
merged 2 commits into from
Sep 2, 2016
Merged

Only cache things in a registry #1068

merged 2 commits into from
Sep 2, 2016

Conversation

tbekolay
Copy link
Member

This is an alternative implementation of #1066 that uses a class registry in the Fingerprint rather than modifying frontend objects.

It occurred to me that the cache system (at least for now) is specific to the reference backend, so it would be best to keep any cache-specific code local to the cache, rather than modifying frontend objects. So that's what this branch does.

I think the amount of code is, in the end, about the same. Fingerprint.__init__ is more clear this way (IMO) but we don't get the benefit of subclassing inheriting things, so each class that should be fingerprinted needs a separate call to Fingerprint.whitelist. But perhaps that's desirable anyhow?

@jgosmann
Copy link
Collaborator

@tcstewar mentioned he is using the cache to cache something else. Not sure if that use is affected by this?

The downside I see with this approach is that there check_* functions required that are not tied to the actual class and might come out of sync with the actual implementation.

it would be best to keep any cache-specific code local to the cache, rather than modifying frontend objects.

Except that there is still the whitelist call in those files.

I think both approaches have their advantages and disadvantages and overall I don't have a strong preference either way.

@tbekolay
Copy link
Member Author

Except that there is still the whitelist call in those files.

Yes, I considered having these in nengo/cache.py, which we could still do. Does that make more sense?

I think both approaches have their advantages and disadvantages and overall I don't have a strong preference either way.

I agree, pros and cons to both! I have an aversion to modifying user-facing objects whenever possible, and would like to reduce the surface area of the API (i.e., not introduce new classes like Cacheable*) which is why I wanted to try out this implementation. But when I realized I needed the check_* functions and that those functions would have to access Fingerprint internals I was less than 100% on this approach. So I'm open to arguments both ways.

@jgosmann
Copy link
Collaborator

Yes, I considered having these in nengo/cache.py, which we could still do. Does that make more sense?

I don't know ...

btw: Not sure if I mentioned this, but my approach was inspired by how the for example pickle allows objects to define the own pickling function.

@mundya
Copy link
Contributor

mundya commented May 18, 2016

It occurred to me that the cache system (at least for now) is specific to the reference backend

nengo_spinnaker uses the cache!

@tcstewar
Copy link
Contributor

nengo_spinnaker uses the cache!

Yup! I think this is another situation like the Parameter system where we need to be careful about changes because other backends are using this too... nengo_brainstorm also can use the cache, and in a weird way....

@tbekolay tbekolay added this to the 2.1.1 release milestone May 26, 2016
@tbekolay tbekolay modified the milestones: 2.2.0 release, 2.1.1 release Jun 7, 2016
@tbekolay tbekolay removed this from the 2.2.0 release milestone Jun 28, 2016
@drasmuss
Copy link
Member

drasmuss commented Aug 4, 2016

I pushed a commit that moves all the fingerprinting related code to cache.py, so we can see what that looks like. Personally I like it, it keeps the caching apparatus hidden from the rest of the code, as I think it should be.

@jgosmann
Copy link
Collaborator

I looked at this again and noticed that 9074d8c removed the dependence on solver internals. I added another commit to take this a step further and not just check attributes of the solver that are instances of Solver and LeastSquaresSolver, but all public attributes and recurse on those. It also add support for tuples and lists.

@jgosmann
Copy link
Collaborator

I think I'm ok with and without that additional commit. I think the additional commit makes this more conservative because it checks more.

@tbekolay
Copy link
Member Author

Interesting! I'm 👍 on 64cf8f3. Pushed a fixup to remove some unused imports and space things out a bit more (is more readable for me, at least, but maybe I've been doing too much JavaScript). There are some flake8 errors in this branch, but they will be resolved once it's rebased to master.

@jgosmann
Copy link
Collaborator

space things out a bit more

Definitely nice when adding/removing things because you don't need to reformat everything to fit the line length. Only downside I see is somewhat more scrolling.

(np.ndarray, check_dtype),
(tuple, check_seq),
(list, check_seq)
] + [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no :barf: emoji but if there was I would use it on this line just gonna settle for 😱

@drasmuss
Copy link
Member

I'm shrugs on the reformatting; not how I would write it myself, but I think it's fine either way. And the rest all looks 👍 to me.

tbekolay and others added 2 commits September 2, 2016 13:18
Addresses #1054.

Code can change and we can't detect it, so we allow only things that
are marked as supporting fingerprints to be cached and only mark
objects that should be stable as supporting this.

This also removes the solver_fn from the fingerprint because it is not
needed in there. It should always be the same function which is part of
the builder (the solver itself is still part of the fingerprint of
course). This in turn allows to simplify the SolverMock in the test
code a little bit. (It actually is slightly misnamed as it mocks the
solver_fn and not a Solver.)

Co-authored by: Jan Gosmann <jgosmann@uwaterloo.ca>
Co-authored by: Daniel Rasmussen <daniel.rasmussen@appliedbrainresearch.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants