UserList-subclass Tree slicing changes the original list unexpectedly

[ctarn]

BPO	39110
Nosy	@terryjreedy, @mdickinson, @stevendaprano, @ericsnowcurrently, @ctarn
Files	bug.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-12-21.11:55:33.192>
created_at = <Date 2019-12-20.15:00:48.103>
labels = ['3.8', 'type-bug', '3.7', 'invalid']
title = 'UserList-subclass Tree slicing changes the original list unexpectedly'
updated_at = <Date 2019-12-21.12:43:31.439>
user = 'https://github.com/ctarn'

bugs.python.org fields:

activity = <Date 2019-12-21.12:43:31.439>
actor = 'ctarn'
assignee = 'none'
closed = True
closed_date = <Date 2019-12-21.11:55:33.192>
closer = 'mark.dickinson'
components = []
creation = <Date 2019-12-20.15:00:48.103>
creator = 'ctarn'
dependencies = []
files = ['48799']
hgrepos = []
issue_num = 39110
keywords = []
message_count = 14.0
messages = ['358709', '358710', '358711', '358712', '358713', '358714', '358725', '358741', '358756', '358763', '358764', '358765', '358768', '358769']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'mark.dickinson', 'steven.daprano', 'eric.snow', 'ctarn']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39110'
versions = ['Python 3.7', 'Python 3.8']

[ctarn]

Sorry, it is caused by list(). I will update the issue very soon.

[ctarn]

In the code, each item of ls = [[0], [1], [2],...] has an owner pointing to d, which is a Tree inheriting from collections.UserList.
When d[0] = a, and a.owner = d, and _ = list(d[0:1]) is called, a.owner will be changed to d[0:1].

[stevendaprano]

Can you give an example? Something simple, showing what you tried, what you expected, and what happened instead.

This may help: http://sscce.org/

Your bug.py file doesn't ever use the result of calling list(), so how do you know it changes the value of the parameter?

[ctarn]

Hi, thanks. It did call list() through _ = list(d[0:2])

[ctarn]

I printed the value of *.owner before and after _ = list(d...), and marked the results in the comments.

[ctarn]

OK, I mean... when I call a = list(b), list() changes b unexpectedly, not a != b in this case. That is why I replace the left operand with _.

[ericsnowcurrently]

Your problem is with UserList. This is from the implementation:

    def __getitem__(self, i):
        if isinstance(i, slice):
            return self.__class__(self.data[i])
        else:
            return self.data[i]

So each slice is creating a new Tree. Then the __setitem__() of that new Tree gets triggered for each item, causing the owner to be set to the new Tree. So that's the cause.

Is there a particular reason you are subclassing UserList? Consider subclassing list or collections.abc.MutableSequence instead. Or deal with the slice semantics manually.

Also, perhaps you expected that a slice would produce a view into the sequence? I know that's how it works in some programming languages (but not Python, though you could make it do that if you wanted to).

Regardless, as far as I can tell everything is working as designed. I recommend closing this.

[terryjreedy]

(ctarn), if you want to discuss what you are doing further, please try a discussion list, such as python-list.

[ctarn]

Sorry but it is not. See the first time I print ls[4].owner. We get d as expected, not a slice of d, that is, d[0:2].

However the next time we print it after _ = list(d[0:1]), noticed that ls[4] == d[0:1], we get d[0:1], it’s extremely surprising!!!

I have to highlight it: we just print *.owner before and after _ = list(d[0:1]), and the results are different!!!

The latest 3 lines show more strange results.

By the way, it’s my first time to report bug, and I don’t know what is discussion list, and reopened the issue. Thank you.

[ctarn]

I tried to remove list(), and just use cmd like _ = d[0:1], and it surprisingly changed d[0].owner from d to d[0:1]!
The file attached is updated.
I pretty sure that it is a (serious) bug. Please check it more carefully. Thanks.

[ctarn]

Moreover, it works as expected with Python 3.6 (the owner of each of them is d), and Python 3.8 and Python 3.7 work differently.
I didn't try it using Python 3.9 yet.

[ctarn]

and more... It doesn't happen when Tree directly subclasses list.

[mdickinson]

[ctarn]

it works as expected with Python 3.6 (the owner of each of them is d), and Python 3.8 and Python 3.7 work differently

The change in behaviour is the result of a bug fix that was applied in 3.7 and upwards: see #57378 and bpo-27639.

As Eric says, UserList is behaving as intended here. Your problem stems from a design flaw in your code, namely that Nodes have owners. If a node exists both in a Tree and in a slice of that Tree (which slice, since the bpo-27639 fix, is again a Tree), that node can't have both the original Tree and the slice as owner. In this case, what's happening is that accessing d[0:1] creates a new Tree object, and the init method for that Tree object then reassigns the "owner" of the node in that slice.

I don’t know what is discussion list

See https://www.python.org/community/lists/, and particularly https://mail.python.org/mailman/listinfo/python-list

Closing again here, but feel free to start a discussion or ask questions on the list above.

[ctarn]

GOD... I see. Thank you very much!