Expose dict_proxy internal type as types.MappingProxyType

[vstinner]

BPO	14386
Nosy	@gvanrossum, @birkenfeld, @rhettinger, @pitrou, @vstinner, @giampaolo, @merwok, @bitdancer, @voidspace, @skrah, @ericsnowcurrently, @JimJJewett
Files	dictproxy.patch dictproxy-2.patch mappingview-3.patch mappingview-4.patch mappingproxy-5.patch mappingproxy_abc.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-04-19.23:42:55.098>
created_at = <Date 2012-03-22.01:50:03.533>
labels = ['interpreter-core', 'type-feature']
title = 'Expose dict_proxy internal type as types.MappingProxyType'
updated_at = <Date 2012-04-19.23:42:55.097>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2012-04-19.23:42:55.097>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2012-04-19.23:42:55.098>
closer = 'python-dev'
components = ['Interpreter Core']
creation = <Date 2012-03-22.01:50:03.533>
creator = 'vstinner'
dependencies = []
files = ['24990', '24991', '25028', '25063', '25070', '25229']
hgrepos = []
issue_num = 14386
keywords = ['patch']
message_count = 40.0
messages = ['156536', '156538', '156550', '156551', '156634', '156780', '156781', '156783', '156785', '156786', '156788', '156789', '156792', '156796', '156815', '156860', '156864', '156867', '156868', '156871', '156934', '156936', '156971', '156973', '156980', '157008', '157012', '157030', '157036', '157083', '157111', '157112', '157113', '157517', '157618', '157619', '157630', '158374', '158375', '158776']
nosy_count = 15.0
nosy_names = ['gvanrossum', 'georg.brandl', 'rhettinger', 'pitrou', 'vstinner', 'giampaolo.rodola', 'eric.araujo', 'Arfrever', 'r.david.murray', 'michael.foord', 'skrah', 'python-dev', 'eric.snow', 'poq', 'Jim.Jewett']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue14386'
versions = ['Python 3.3']

[vstinner]

Attached patch makes the dictproxy type public. Example:

$ ./python 
Python 3.3.0a1+ (default:059489cec7b9+, Mar 22 2012, 02:45:36) 
>>> d=dictproxy({1:2})
>>> d
dict_proxy({1: 2})
>>> d[1]
2
>>> d[1]=3
TypeError: 'dictproxy' object does not support item assignment
>>> del d[1]
TypeError: 'dictproxy' object doesn't support item deletion
>>> d.copy()
{1: 2}
>>> dir(d)
[..., '__getitem__', 'copy', 'get', 'items', 'keys', 'values']

The patch doesn't have any test or documentation yet.

See also the (now rejected) PEP-416 (frozendict).

[vstinner]

dictproxy.patch: proxy_new() should check that dict is a dict (PyDict_Check).

[vstinner]

Patch version 2:

• add tests and documentation
• dictproxy code moved to dictproxyobject.c
• dict_proxy replaced with dictproxy in repr(dictproxy)
• "key in dictproxy" now handles correctly dict subclasses
• dictproxy constructor raises a TypeError if the argument is not a dict

I added a section "Immutable Mapping" in the doc, this title is maybe wrong because a dictproxy is not truly immutable. If the dict referenced by the dictproxy is modified, the dictproxy is also modified.

dictproxy implementation supports any mapping, it is not specific to dict. It would be interesting any mapping, collections.ChainMap for example. The problem is to reject sequence in dictproxy constructor.

The "PyMapping_Check(dict) && !PyMapping_Check(dict)" fails on ChainMap. type.__new__(ChainMap) fills tp_as_sequence->sq_item slot is defined because ChainMap has a __getitem__ method.

Accepting sequences would be a bad idea because dictproxy(sequence)[missing_key] raises IndexError instead of KeyError.

Note: issubclass(collections.ChainMap, collections.abc.Sequence) is False.

[vstinner]

The dictproxy is useful to implement a Python sandbox: see the related issue bpo-14385.

[jimjjewett]

The problem is to reject sequence in dictproxy constructor.

Why? Just because you can't delegate in quite the same way? A sequence *does* meet the (immutable) Mapping interface; it just won't happen to have any non-integer keys.

Or are you worried about IndexError vs KeyError? Personally, I would just document it as returning LookupError, but if you're really worried, you could always catch the IndexError and raise a KeyError from it.

The "PyMapping_Check(dict) && !PyMapping_Check(dict)"

I'm not seeing how anything could sanely pass that ... was there a typo when pasting?

[vstinner]

A sequence *does* meet the (immutable) Mapping interface

Ah? Let's try:

>>> x=[1, 2, 3]
>>> x.get(2)
AttributeError: 'list' object has no attribute 'get'
>>> x.keys()
AttributeError: 'list' object has no attribute 'keys'

list doesn't implement the collections.abc.Mapping ABC.

[vstinner]

The "PyMapping_Check(dict) && !PyMapping_Check(dict)"

Oh, I mean "PyMapping_Check(dict) && !PySequence_Check(dict)"

[rhettinger]

I'm -1 on exposing this API.

Victor's sandboxing project doesn't warrant cluttering the public toolset.

As a consultant who gets to see many large codebases in different companies and I see zero need for this type to be exposed. It is not hard to build proxies around any existing type. Victor's use case is unique in that he needs the type to be implemented in C. To that end, it would be okay to expose _dictproxy but not to make it a documented type.

[bitdancer]

I believe it was actually Guido who suggested exposing dict_proxy, in response to Victor (but independent of Victor's needs, if I understood the message correctly).

It has always seemed odd to me ("always" referring my time working with Python3, of course) that a class that implements the mapping protocol has no way to return the same type of objects that real dict methods do.

[vstinner]

Victor's use case is unique in that he needs the type
to be implemented in C.

mxProxy, zope.proxy and zope.security implement object proxies in C (for security purpose). These proxies are not specific to dict, but are generic.

dictproxy is already used in Python by the type.__dict__ descriptor getter of new-style classes since Python 2.2, since this changeset:

changeset: 18933:09df3254b49d
branch: legacy-trunk
user: Tim Peters <tim.peters@gmail.com>
date: Thu Aug 02 04:15:00 2001 +0000
files: Include/Python.h Include/abstract.h Include/ceval.h Include/classobject.h Include/descrobject.h Include/dictobject.h Include/eval.h Include/func
description:
Merge of descr-branch back into trunk.

Guido van Rossum wrote in its rejection notice of the PEP-416: "On the other hand, exposing the existing read-only dict proxy as a built-in type sounds good to me. (It would need to be changed to allow calling the constructor.) GvR."

--

To that end, it would be okay to expose _dictproxy
but not to make it a documented type.

My sandbox only needs the issue bpo-14385, pysandbox can implement its own dictproxy class.

If the issue bpo-14385 is accepted, pysandbox can use any mapping, even a mapping implemented in Python! It's just simpler to implement a secure proxy in C.

So this issue is not directly related to sandboxing.

[rhettinger]

It's clutter and we should show some restraint before adding new types.

Enthought's Python Distribution isn't shy about adding tools when they have a demonstrated need. Likewise, Django adds any reusable tools they need. Neither have a dict proxy. I've never run into a use case for it and haven't seen any feature requests for it. PyPi doesn't have package for it (at least not one with any uptake). The languages doesn't *need* this, nor do the other implementations which are trying to keep up with CPython.

Guido may let you put in it, but that doesn't mean you should. Again, I ask you to show some restraint. The language has already exploded beyond what fits into most developer's heads (even core devs don't even know about many the newer features).

[vstinner]

It's clutter and we should show some restraint before adding new types.

dictproxy() doesn't need to be a public builtin type. We can just implement its constructor without documenting the type. Or it may be exposed in a module like collections.

[gvanrossum]

Given that dictproxy is needed to implement the proper interface for classes I don't think it is an implementation detail, and so I think it ought to be constructable. We have more esoteric types constructable (e.g. function).

I also happen to think that its use to implement classes is a pretty darn good use case, and I am sure there are other use cases.

I don't really care if we make it a builtin but there should be a standard name for it somewhere in the stdlib.

[rhettinger]

[haypo]

dictproxy() doesn't need to be a public builtin type.

+1

We can just implement its constructor without documenting the type.

+1

Or it may be exposed in a module like collections.

Please don't use collections as a dumping ground for this sort of thing.