Skip to content

Contribute implementation to Python #2

Open
thedrow opened this Issue May 18, 2014 · 11 comments

6 participants

@thedrow
thedrow commented May 18, 2014

See http://bugs.python.org/issue21145.
All we need to add is the TTL feature that other implementations like pip and werkzeug and Django have.

This package already has some basic testing and it would be nice to start from there.

@pydanny
Owner
pydanny commented May 18, 2014
@thedrow
thedrow commented May 18, 2014

High five. I hope to see this in 3.5 :)

@eevee
eevee commented Jun 16, 2014

Posting here because it's easier and more polite than arguing on a CPython bug thread. This probably smells like bikeshedding, but hear me out.

If it gets into the standard lib, please don't call it @cached_property, and certainly don't merge it with a TTL version.

"Cache" suggests "I can't afford to run this very often". The semantics of a cache are generally understood to include that if it abruptly vanishes out from under you, your code still works as intended, just more slowly.

@cached_property is different: it says "I only want to run this at most one time, period." Much of the code I've written using similar implementations would break if the underlying value were recomputed, because it's expected to act like a regular attribute and always be the same object.

The semantics here are really about lazy computation, not about caching. There's a value you only want or need to compute once, but it's expensive, so you want to put it off as long as possible and avoid it entirely if you never need it.

I really like Pyramid's name, @reify. It's short (no underscore!), it's unlikely to collide with existing names, and it literally means "to make real".


Other observations that might be worth addressing before this goes to rot in the stdlib :)...

  • You can use setattr instead of mucking with __dict__ directly, which would then fire __setattr__, which seems correct under the interpretation of a deferred attribute assignment.
  • Like every other implementation, this doesn't work with __slots__. You'd have to do something really weird with types.GetSetDescriptorType to make it work, and I'm not even sure whether it's possible, but I'd expect something in the stdlib to work with builtin language features.
@thedrow
thedrow commented Jun 17, 2014

@eevee I'm not sold. TTL is widely used with this feature. If we provide a version without it, you'll still find implementations of it everywhere. That sort of thing indicates that this feature should be in stdlib.
Not sure about the name, cached_property has been in use for a very long time. I'm -0 on it.

I agree that we should use setattr.
If cached_property doesn't work with __slots__ it is indeed a problem. Can you describe why it doesn't work?

@pydanny
Owner
pydanny commented Jun 17, 2014

@eevee The problem with reify is that it's obscure. It's not explicitly named, and discovery within Python and it's documentation will be a challenge. People will continue to use custom implementations because they won't know that core contains it.

I'm not entirely convinced that this is a 'lazy' technique, but if other minds prevail on the subject, perhaps calling it lazy_property makes more sense.

@eevee
eevee commented Jun 17, 2014

@thedrow I'm not saying there shouldn't be a decorator with a TTL, just that in my experience the plain decorator has a vastly different meaning from a configurable cache with automatic eviction.

I'd actually never heard of it called "cached_property" before; my first encounter with the idea at all was Pyramid :)

__slots__ works by adding C-level slot descriptors to the class, but they get overwritten by the property descriptor. Without access to the slot descriptor, it's impossible to actually store the computed value anywhere, because there's no __dict__ in a class with __slots__.

@pydanny Will people realize the core contains it anyway? Regardless of the name, the best way to make it discoverable is probably to mention it in the docs for @property. functools is definitely the right place for this to live, yet I highly suspect most people still think of the module as only "the place you import partial from".

@lazy_property would certainly be an improvement! I just have a particular soft spot for short memorable quippy names :) (Imagine if enumerate had been called iter_with_index...)

@Tinche
Tinche commented Jun 17, 2014

I like pithy names too, but speaking from personal experience (with the caveat that anecdotes are not data, etc.) it would not occur to me that the decorator in question here does what it does if it was called reify. In fact about the only place I've heard of reification in the context of programming is in the different approaches to generics between Java and C# (type erasure vs type reification). cached_property gave me an exact and pretty correct idea of what the decorator does immediately. Again I realize this is subjective and not a strong argument. (Also I'm not a native speaker, but I'd wager most Python programmer aren't either...)

Also, when something is 'lazy' I expect it will be calculated/retrieved on demand, not that it'll be calculated only once (although that's certainly part of it). Asking Google to define reification (making something real, bringing something into being, or making something concrete) somehow makes me think this would be the opposite of lazy.

How about using 'memoization' in the name? (memoized_property, memoize...)

@eevee
eevee commented Jun 17, 2014

"Reify" is admittedly not a particularly common English word. I always interpreted it as "this function becomes a real attribute once called".

Calculating on demand is certainly part of the behavior here; if I expect to always need the value I might as well just compute it in __init__.

I don't think "memoize" is right, since that brings to mind a mapping of arguments to return values, and part of the point of a property is that there are no arguments.

@Julian
Julian commented Nov 28, 2014

To throw my 2 cents in because I've always hated the name cached_property too -- my preference on naming is:

cached_property < reify < lazy_property

but I don't like "property" at all to be honest, since I think that's part of the confusion on multiple calculation. Personally I don't like short names if they're not descriptive enough :P, so I might like something like @calculated_once or @calculate_and_replace most.

@RonnyPfannschmidt

just my 2 cents, once this goes into the std-lib its dead :)

@thedrow
thedrow commented Feb 15, 2016

But there aren't many code changes to makes. It works and that's it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.