weaklist

[atuells]

BPO	487738
Nosy	@tim-one, @loewis, @freddrake, @birkenfeld, @rhettinger, @facundobatista
Files	weaklist.py: weaklist.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/freddrake'
closed_at = <Date 2008-01-22.14:08:32.609>
created_at = <Date 2001-12-01.01:36:06.000>
labels = ['type-feature']
title = 'weaklist'
updated_at = <Date 2008-01-22.14:08:32.608>
user = 'https://bugs.python.org/atuells'

bugs.python.org fields:

activity = <Date 2008-01-22.14:08:32.608>
actor = 'fdrake'
assignee = 'fdrake'
closed = True
closed_date = <Date 2008-01-22.14:08:32.609>
closer = 'fdrake'
components = ['None']
creation = <Date 2001-12-01.01:36:06.000>
creator = 'atuells'
dependencies = []
files = ['8191']
hgrepos = []
issue_num = 487738
keywords = ['patch']
message_count = 12.0
messages = ['53355', '53356', '53357', '53358', '53359', '53360', '53361', '53362', '53363', '60138', '60245', '61505']
nosy_count = 8.0
nosy_names = ['tim.peters', 'loewis', 'fdrake', 'georg.brandl', 'rhettinger', 'facundobatista', 'atuells', 'g-lite']
pr_nums = []
priority = 'low'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue487738'
versions = ['Python 2.6']

[atuells]

WeakList are list whose entries are referenced weakly.
When the object is gc it is deleted from the weaklist
(and from its iterators). To be added to weakref.py

[loewis]

Logged In: YES
user_id=21627

Thanks for the patch. I would recommend to publish it as a
separate package first, to get user feedback. I can't see
this as a universally-useful data type, so I'm not sure it
should be added to the standard library.

*If* it is added, a number of corrections must be made to
the code:

• remove removes elements by equality, not identity. I
  believe in removeAll, you are looking for identical objects,
  not merely equal ones.

• Why is it the right thing to remove elements from the list
  if the underlying object dies? doesn't this have undesirable
  side effects on indexing, e.g. when the list is being
  iterated over?

• In the standard library, I think inheriting from UserList
  is deprecated, in favour of inheriting from list.

• It seems that the class creates many unnecessary copies of
  lists, e.g. in extend, or setslice.

• The references create cycles involving WeakList. Since the
  WeakList refers to ref objects through data, and the ref
  objects refer to the list throught the callback, the list
  itself will become garbage as long as list elements remain
  alive (although GC will detect those cycles). That should be
  avoided.

• What is the point of the infinite loop in __getitem__?

[freddrake]

Logged In: YES
user_id=3066

Needs motivation. Without an need for the data structure,
this will be rejected. Lowering priority and marking this
for consideration for Python 2.3; it's too late to add this
for Python 2.2.

Set to "pending" while awaiting an explanation of the
motivation.

[freddrake]

Logged In: YES
user_id=3066

Oops, I meant to adjust the priority on this.

[g-lite]

Logged In: YES
user_id=890349

Mind if I bring this back up? This doesn't seem to be in
yet. I didn't look at the implementation, but as for
motivation...

Andres mentioned his original motivation on a list:
http://aspn.activestate.com/ASPN/Mail/Message/python-list/929285

I've also seen it duplicated in this recipe:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/87056

I'd like to see it in for the same reason.

[freddrake]

Logged In: YES
user_id=3066

This might be interesting to have. Would this be more
useful than, say, a weak set? Ordering may be important for
the publish/subscribe use case, at least to ensure
predictability.

I've not looked at the contributed code, so can't make any
comment on that.

[g-lite]

Logged In: YES
user_id=890349

I'm not sure if either is more useful than the other, they
both seem to have their advantages.

It looks like a set would work for the links I mentioned,
but I'd personally like to have the ability to connect to a
signal/event with priority, thus needing a list.

A weak set could possibly be implemented on top of
WeakKeyDictionary? Have weak sets already been implemented
somewhere?

[rhettinger]

Logged In: YES
user_id=80475

Weaksets would not be warranted unless the use cases
demonstrated needs for methods unique to sets (intersection,
union, etc). Otherwise, weakdictionaries would suffice and
we could avoid bloating the module.

Also, I'm not sure weaklists are a good idea. First, it is
not clear that the subscription use case can be reliably
implemented with gc as the primary means of unsubscribing --
traditionally, that is done with an explicit unsubscribe()
call issued by the subscriber.

Second, weaklists only have a differential advantage when it
comes to maintaining insertion order. It is not clear that
that feature is really useful. What is clear is that the
approach incurs extra costs for maintaing order and O(n)
removal time.

When order tracking is necessary, there are reasonable
implementations using weakkeydictionaries with the entry
values being a sequence number indicating creation order.
With that data structure, group notification is still simple:

for subscriber in sorted(wkd, key=wkd.__getitem__):
    self.notify(subscriber, message)

This approach incurs an O(n log n) sort cost for each group
notify but has only an O(1) deletion cost which is an
improvement over weaklists. The only way I see around the
deletion time issue is to have a weakdoublylinked list which
would allow O(1) deletions, appends, and O(n) iteration.

None of this came up on the referenced newsgroup posting
because there was NO active discussion. IOW, the idea has
not shown any demand and has not been through the basic due
diligence needed to tease out the best approach. I recommend
leaving this as a recipe until we see a battlehardened
implementation, a convincing use case, and some user demand.

[tim-one]

Logged In: YES
user_id=31435

FYI, ZODB has a WeakSet implementation, in utils.py here:

http://svn.zope.org/ZODB/trunk/src/ZODB/

I wouldn't add this to the core -- it's very specific to ZODB's
needs. For example, it actually uses a weak value
dictionary, because ZODB can't assume the set elements
are usably comparable or hashable.

One thing that "should be" addressed in the core is explained
in a long comment there: the core weak containers don't
supply a sane way to iterate over their weakly referenced
containees. You either risk unpredictable "dict changed size
during iteration" exceptions, or unboundedly worse gc
behavior (read the comment for more detail). The ZODB
WeakSet implementation has to break into the weak-value
dict implementation to supply a partially sane way to iterate
over its elements ("partially" means it's not incremental, and
exposes weakrefs to the client; OTOH, it doesn't suffer
mystery "dict changed size" exceptions, and plays nicely
with gc).

[facundobatista]

No news after two years and a half. Considering the arguments of Raymond
after S. Kochen brought the issue back up, I'd close the patch as rejected.

Feel free to bring this issue to python-dev, and if there's real need,
we can always open the patch back.

Fred, it's assigned to you... what do you think?

Thank you!

[birkenfeld]

Also we already have a WeakSet now since the abc module needs it.

[freddrake]

Facundo: Agreed as well; since the use case isn't strong, let's avoid
adding this.