Implement PEP 560: Core support for typing module and generic types

[ilevkivskyi]

BPO	32226
Nosy	@gvanrossum, @ned-deily, @serhiy-storchaka, @ilevkivskyi, @xgid
PRs	bpo-32226: Implementation of PEP 560 (core components) #4732 bpo-32226: PEP 560: improve typing module #4906 bpo-32226: Make __class_getitem__ an automatic class method. #5098 bpo-32226: Fix memory leak in generic_alias_dealloc() #5212

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ilevkivskyi'
closed_at = <Date 2018-01-29.23:35:04.753>
created_at = <Date 2017-12-05.20:36:48.434>
labels = ['interpreter-core', 'type-feature', 'library', '3.7']
title = 'Implement PEP 560: Core support for typing module and generic types'
updated_at = <Date 2018-01-29.23:35:04.752>
user = 'https://github.com/ilevkivskyi'

bugs.python.org fields:

activity = <Date 2018-01-29.23:35:04.752>
actor = 'levkivskyi'
assignee = 'levkivskyi'
closed = True
closed_date = <Date 2018-01-29.23:35:04.753>
closer = 'levkivskyi'
components = ['Interpreter Core', 'Library (Lib)']
creation = <Date 2017-12-05.20:36:48.434>
creator = 'levkivskyi'
dependencies = []
files = []
hgrepos = []
issue_num = 32226
keywords = ['patch']
message_count = 28.0
messages = ['307682', '308092', '308101', '308121', '308151', '308171', '308291', '308312', '308323', '308341', '308346', '308347', '308351', '309484', '309485', '309829', '309830', '309927', '310149', '310150', '310208', '310336', '311027', '311035', '311036', '311038', '311148', '311199']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'ned.deily', 'serhiy.storchaka', 'levkivskyi', 'xgdomingo']
pr_nums = ['4732', '4906', '5098', '5212']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue32226'
versions = ['Python 3.7']

[ilevkivskyi]

As discussed before, there will be two PRs. One for the core components, and the second one (large) for typing updates. I will open the first PR shortly.

[serhiy-storchaka]

As for __class_getitem__, why not implement type.__getitem__ instead of hacking PyObject_GetItem()?

[serhiy-storchaka]

For reference: PEP-560.

[ilevkivskyi]

As for __class_getitem__, why not implement type.__getitem__ instead of hacking PyObject_GetItem()?

This question was raised by Mark Shannon on python-dev. Actually, my initial implementation did exactly this, but I didn't like it for two reasons:

1. dir(type) is already very long, I don't want to make it even longer.
2. Some users may be surprised that although type.__getitem__ is defined, int[something] will rise "TypeError: type object is not subscriptable"

Mark however disappeared since then, so I don't know what is his current opinion. I am however not as sure about this now. If there are some arguments for this, then I can revert to the older approach (implementing type.__getitem__).

[ilevkivskyi]

Guido, what is your preference for the implementation of __class_getitem__: PyObject_GetItem (current one) or type.__getitem__ (old one)? Can we just go ahead with the version you like and then re-consider if some objections will appear? I am asking because you are going on vacation soon and there is a second part of implementation -- changes in typing (and someone needs to review them :-)

[gvanrossum]

I like the current approach (PyObject_GetItem). I agree with both of your reasons to prefer it.

[serhiy-storchaka]

This adds an overhead to every indexing.

$ ./python -m perf timeit --compare-to=./python0 -s 'a = [1]' --duplicate=10000 'a[0]'
python0: ..................... 15.3 ns +- 0.5 ns
python: ..................... 16.1 ns +- 0.3 ns

Mean +- std dev: [python0] 15.3 ns +- 0.5 ns -> [python] 16.1 ns +- 0.3 ns: 1.05x slower (+5%)

$ ./python -m perf timeit --compare-to=./python0 -s 'd = {1: 2}' --duplicate=10000 'd[1]'
python0: ..................... 17.6 ns +- 0.6 ns
python: ..................... 18.4 ns +- 0.5 ns

Mean +- std dev: [python0] 17.6 ns +- 0.6 ns -> [python] 18.4 ns +- 0.5 ns: 1.05x slower (+5%)

[gvanrossum]

This adds an overhead to every indexing.

I'm not doubting your results, but I'm curious how that happened. The extra
code in PyObject_GetItem is right before the point where it would otherwise
raise TypeError -- I presume you benchmark never reached that point. Could
it be that the compiler ends up rearranging the code in a way that is less
optimal for the instruction cache?

[serhiy-storchaka]

I think this is a cause. The difference may be dependent on the compiler and platform.

[ilevkivskyi]

New changeset 2b5fd1e by Ivan Levkivskyi in branch 'master':
bpo-32226: Implementation of PEP-560 (core components) (bpo-4732)
2b5fd1e