compiler: merge co_code and co_lnotab

[methane]

BPO	42217
Nosy	@methane, @markshannon, @serhiy-storchaka
PRs	bpo-42217: compiler: merge same co_code and co_linetable objects #23056

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-02-10.00:20:59.942>
created_at = <Date 2020-10-31.07:50:02.020>
labels = ['interpreter-core', '3.10', 'performance']
title = 'compiler: merge co_code and co_lnotab'
updated_at = <Date 2021-02-10.00:21:17.632>
user = 'https://github.com/methane'

bugs.python.org fields:

activity = <Date 2021-02-10.00:21:17.632>
actor = 'methane'
assignee = 'none'
closed = True
closed_date = <Date 2021-02-10.00:20:59.942>
closer = 'methane'
components = ['Interpreter Core']
creation = <Date 2020-10-31.07:50:02.020>
creator = 'methane'
dependencies = []
files = []
hgrepos = []
issue_num = 42217
keywords = ['patch']
message_count = 5.0
messages = ['380045', '380049', '380050', '380061', '386749']
nosy_count = 3.0
nosy_names = ['methane', 'Mark.Shannon', 'serhiy.storchaka']
pr_nums = ['23056']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue42217'
versions = ['Python 3.10']

[methane]

co_consts are merged already, but I forget about code attributes other than co_consts.
Duplication ratio is vary by code style, but in case of logging, we can reduce 25% of them.

>>> data = open('Lib/logging/__pycache__/__init__.python-310.pyc', 'rb').read()
>>> import marshal
>>> x = []
>>> def walk(c):
...     x.append(c.co_code)
...     x.append(c.co_lnotab)
...     for cc in c.co_consts:
...         if hasattr(cc, 'co_code'):
...             walk(cc)
...
>>> code = marshal.loads(data[16:])
>>> walk(code)
>>> len(x)
336
>>> len(set(x))
249
>>> 249/336
0.7410714285714286

[serhiy-storchaka]

Do you mean sharing values of co_code and co_lnotab between code objects of the same module?

How much memory does this save (in absolute and relative value)? Which functions have the same co_code?

[markshannon]

PEP-626 has just been accepted.
Could you please leave this until I have merged in the implementation of PEP-626 which uses a different line number table format

[methane]

@serhiy

Do you mean sharing values of co_code and co_lnotab between code objects of the same module?

Yes.

How much memory does this save (in absolute and relative value)?

Maybe 1~3%, but I am not sure. I am more interested in reducing number of objects, because it will reduce import time.
Additionally, co_code is used while executing code. Unlike other cold data (e.g. docstring, annotations), sharing co_code will improve CPU cache utilization.

Which functions have the same co_code?

For example,

# logging/__init__.py

    @property
    def manager(self):
        return self.logger.manager
...
    @property
    def name(self):
        return self.logger.name

Such simple functions are very common in OO-style code.

@mark Shannon

Sure.

[methane]

New changeset bdb941b by Inada Naoki in branch 'master':
bpo-42217: compiler: merge same co_code and co_linetable objects (GH-23056)
bdb941b