Skip to content

Commit

Permalink
tools/mpy-tool: Set sane initial dynamic qstr pool size with frozen mods
Browse files Browse the repository at this point in the history
The first dynamic qstr pool is double the size of the 'alloc' field of
the last const qstr pool. The built in const qstr pool
(mp_qstr_const_pool) has a hardcoded alloc size of 10, meaning that the
first dynamic pool is allocated space for 20 entries. The alloc size
must be less than or equal to the actual number of qstrs in the pool
(the 'len' field) to ensure that the first dynamically created qstr
triggers the creation of a new pool.

When modules are frozen a second const pool is created (generally
mp_qstr_frozen_const_pool) and linked to the built in pool. However,
this second const pool had its 'alloc' field set to the number of qstrs
in the pool. When freezing a large quantity of modules this can result
in thousands of qstrs being in the pool. This means that the first
dynamically created qstr results in a massive allocation. This commit
sets the alloc size of the frozen qstr pool to 10 or less (if the number
of qstrs in the pool is less than 10). The result of this is that the
allocation behaviour when a dynamic qstr is created is identical with an
without frozen code.

Note that there is the potential for a slight memory inefficiency if the
frozen modules have less than 10 qstrs, as the first few dynamic
allocations will have quite a large overhead, but the geometric growth
soon deals with this.

(cherry picked from commit 6e5a40c)
  • Loading branch information
Rich Barlow authored and prusnak committed Oct 7, 2019
1 parent c5d8107 commit 64236f5
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion tools/mpy-tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -515,12 +515,15 @@ def freeze_mpy(base_qstrs, raw_codes):
print(' MP_QSTR_%s,' % new[i][1])
print('};')

# As in qstr.c, set so that the first dynamically allocated pool is twice this size; must be <= the len
qstr_pool_alloc = min(len(new), 10)

print()
print('extern const qstr_pool_t mp_qstr_const_pool;');
print('const qstr_pool_t mp_qstr_frozen_const_pool = {')
print(' (qstr_pool_t*)&mp_qstr_const_pool, // previous pool')
print(' MP_QSTRnumber_of, // previous pool size')
print(' %u, // allocated entries' % len(new))
print(' %u, // allocated entries' % qstr_pool_alloc)
print(' %u, // used entries' % len(new))
print(' {')
for _, _, qstr in new:
Expand Down

0 comments on commit 64236f5

Please sign in to comment.