Remove import copyreg from os module

[tiran]

BPO	19209
Nosy	@warsaw, @birkenfeld, @vstinner, @tiran, @serhiy-storchaka
Files	os_no_copyreg.patch os_stat_statvfs_pickle.patch os_stat_statvfs_pickle2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-10-11.23:29:43.034>
created_at = <Date 2013-10-09.14:21:20.541>
labels = ['performance']
title = 'Remove import copyreg from os module'
updated_at = <Date 2013-10-11.23:42:01.723>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2013-10-11.23:42:01.723>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2013-10-11.23:29:43.034>
closer = 'christian.heimes'
components = []
creation = <Date 2013-10-09.14:21:20.541>
creator = 'christian.heimes'
dependencies = []
files = ['32020', '32026', '32051']
hgrepos = []
issue_num = 19209
keywords = ['patch']
message_count = 19.0
messages = ['199303', '199304', '199305', '199306', '199307', '199309', '199322', '199326', '199327', '199328', '199329', '199333', '199334', '199336', '199369', '199517', '199522', '199523', '199524']
nosy_count = 6.0
nosy_names = ['barry', 'georg.brandl', 'vstinner', 'christian.heimes', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue19209'
versions = ['Python 3.4']

[tiran]

The patch removes "import copyreg" from the os module and moves the registration of the hooks to copyreg. This speeds up the startup of the interpreter a tiny bit.

[birkenfeld]

+1

[vstinner]

I don't know the copyreg module! Does it have a unit test for the registered os objects? If not, how can it be tested manually?

[tiran]

>>> import os
>>> import pickle
>>> pickle.dumps(os.stat("."))
b"\x80\x03cos\n_make_stat_result\nq\x00(M\xfdAJ\x84\xa0k\x00M\x00\xfcK\x13M\xe8\x03M\xe8\x03M\x00`J\x9chURJ\xbdgURJ\xbdgURtq\x01}q\x02(X\x08\x00\x00\x00st_ctimeq\x03GA\xd4\x95Y\xefW\x04\xc1X\x07\x00\x00\x00st_rdevq\x04K\x00X\x0b\x00\x00\x00st_mtime_nsq\x05\x8a\x08\xe8/\xfdo@w+\x13X\x08\x00\x00\x00st_atimeq\x06GA\xd4\x95Z'$T\xb6X\x0b\x00\x00\x00st_ctime_nsq\x07\x8a\x08\xe8/\xfdo@w+\x13X\n\x00\x00\x00st_blksizeq\x08M\x00\x10X\x0b\x00\x00\x00st_atime_nsq\t\x8a\x08\xa0\x0e9htw+\x13X\x08\x00\x00\x00st_mtimeq\nGA\xd4\x95Y\xefW\x04\xc1X\t\x00\x00\x00st_blocksq\x0bK8u\x86q\x0cRq\r."
>>> pickle.loads(pickle.dumps(os.stat(".")))
posix.stat_result(st_mode=16893, st_ino=7053444, st_dev=64512, st_nlink=19, st_uid=1000, st_gid=1000, st_size=24576, st_atime=1381329052, st_mtime=1381328829, st_ctime=1381328829)

[serhiy-storchaka]

What will happen when do not register stat_result and statvfs_result at all?

[tiran]

You can still pickle and unpickle the objects but the result is no longer platform-independent as it refers to "posix" or "nt" instead of "os".

>>> import os, pickle, pickletools
>>> pickletools.dis(pickle.dumps(os.stat(".")))
    0: \x80 PROTO      3
    2: c    GLOBAL     'os _make_stat_result'
   24: q    BINPUT     0
   26: (    MARK
...

>>> pickletools.dis(pickle.dumps(os.stat(".")))
    0: \x80 PROTO      3
    2: c    GLOBAL     'posix stat_result'
   21: q    BINPUT     0
   23: (    MARK
...

[vstinner]

Can't we modify the qualified name instead?

[birkenfeld]

But for pickling something, you have to import pickle, which always imports copyreg anyway, doesn't it?

[tiran]

Exactly, the pickle module depends on the copyreg module. It's a submodule that acts as a registry for pickle-related lookups and hooks. My patch just moves the registration of these hooks out of the os module into the copyreg module.

[birkenfeld]

I don't see a problem with that.

[serhiy-storchaka]

How much this speed up the startup of the interpreter?

Proposed patch looks contrary to purpose of the copyreg module.

[tiran]

The speedup is minimal but it's a start.

[vstinner]

os_no_copyreg.patch looks good to me.

[serhiy-storchaka]

I don't think we should tangent code for such tiny benefit. copyreg is lightweight module specially designed to break coupling of the code.