Manager documentation unclear about lists and thread safeness

[IrvinProbst]

BPO	20626
Nosy	@erlend-aasland
Files	example.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-02-14.14:56:06.403>
labels = ['type-feature', 'docs']
title = 'Manager documentation unclear about lists and thread safeness'
updated_at = <Date 2021-04-04.14:33:26.059>
user = 'https://bugs.python.org/IrvinProbst'

bugs.python.org fields:

activity = <Date 2021-04-04.14:33:26.059>
actor = 'erlendaasland'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2014-02-14.14:56:06.403>
creator = 'Irvin.Probst'
dependencies = []
files = ['34080']
hgrepos = []
issue_num = 20626
keywords = []
message_count = 2.0
messages = ['211220', '390190']
nosy_count = 4.0
nosy_names = ['docs@python', 'sbt', 'Irvin.Probst', 'erlendaasland']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20626'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

[IrvinProbst]

In the the Manager's lists documentation one can read:

"""
list()
list(sequence)
Create a shared list object and return a proxy for it.
"""

IMHO it is really unclear whether these lists have something more than traditionnal lists or not.

When you have a look at managers.py it is quite obvious, unless I'm completely wrong, that the underlying shared object is a standard list with a basic proxy to expose all the "underscore underscore stuff".

"""
BaseListProxy = MakeProxyType('BaseListProxy', (
'__add__', '__contains__', '__delitem__', '__getitem__', '__len__',
'__mul__', '__reversed__', '__rmul__', '__setitem__',
'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove',
'reverse', 'sort', '__imul__'
))

class ListProxy(BaseListProxy):
    def __iadd__(self, value):
        self._callmethod('extend', (value,))
        return self
    def __imul__(self, value):
        self._callmethod('__imul__', (value,))
        return self

[snip a couple of lines]

SyncManager.register('list', list, ListProxy)
"""

That's really confusing because, unless reading managers.py, you have the feeling that the manager's lists() are somehow different than standard lists.

The other problem is that, if you don't know what level of thread-safeness the GIL guarantees on the lists in the manager's server thread, you have the feeling that the safeness comes from some obscure Manager's black magic managing concurrent access for you.

May I suggest to add in the documentation:

1/

"""
list()
list(sequence)
Create a shared list (add here a link to http://docs.python.org/3.3/library/stdtypes.html#list) object and return a proxy for it.
"""

2/

Clearly state somewhere in the manager's documentation that it's the developper's job to ensure that the proxied methods are thread safe. Write it in bold and red please :-)

3/
Perhaps add an example with a custom object like the code I attached to this report.

Thanks for your time.

[erlend-aasland]

1. A link was added in commit 86a7668
2. The Programming Guidelines mentions thread safety of proxies: https://docs.python.org/3/library/multiprocessing.html#programming-guidelines

Can this be closed?