obmalloc radix tree typo in code

[nascheme]

BPO	45521
Nosy	@nascheme, @methane, @miss-islington
PRs	bpo-45521: Fix a bug in the obmalloc radix tree code. #29051 [3.10] bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) #29122

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/methane'
closed_at = <Date 2021-10-21.15:41:32.700>
created_at = <Date 2021-10-19.05:37:58.958>
labels = ['interpreter-core', '3.10', '3.11']
title = 'obmalloc radix tree typo in code'
updated_at = <Date 2021-10-21.15:41:32.700>
user = 'https://github.com/nascheme'

bugs.python.org fields:

activity = <Date 2021-10-21.15:41:32.700>
actor = 'nascheme'
assignee = 'methane'
closed = True
closed_date = <Date 2021-10-21.15:41:32.700>
closer = 'nascheme'
components = ['Interpreter Core']
creation = <Date 2021-10-19.05:37:58.958>
creator = 'nascheme'
dependencies = []
files = []
hgrepos = []
issue_num = 45521
keywords = ['patch']
message_count = 5.0
messages = ['404268', '404269', '404351', '404602', '404610']
nosy_count = 3.0
nosy_names = ['nascheme', 'methane', 'miss-islington']
pr_nums = ['29051', '29122']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45521'
versions = ['Python 3.10', 'Python 3.11']

[nascheme]

There is a typo in the radix tree obmalloc code, spotted by Inada Naoki.

-#define MAP_TOP_MASK (MAP_BOT_LENGTH - 1)
+#define MAP_TOP_MASK (MAP_TOP_LENGTH - 1)

This should be fixed both in the main branch and in 3.10.x.

[methane]

When I am trying to understand this issue, I see this segfault.

https://gist.github.com/methane/1b83e2abc6739017e0490c5f70a27b52

I am not sure this segfault is caused by this issue or not. If this is unrelated, I will create another issue.

[nascheme]

I have not yet been able to reproduce methane's crash. My guess it it's not related.

An explanation of what I think the impact of this bug is:

The radix tree is used to determine if memory is from obmalloc or from the system malloc (i.e return value from address_in_range()). WIth ADDRESS_BITS set to 48, we ignore the top 16 bits of addresses. The next 10 bits are supposed to be the index into the top level node array for the radix tree. Due to the bug, we mask those and only use the bottom 8 of those 10. So, if you have virtual addresses that span more than that 8 bit range, we will index into the wrong node. That means address_in_range() could give the wrong answer. Which means you might try to free memory with the wrong malloc.

I think this is likely to be triggered only if you allocate a massive amount of memory, like 70 TB. However, triggering it would depend on how the kernel maps virtual memory to the Python process. I.e. there might be a wierd OS that gives pages at 0x7f0000000000 and then right after pages at 0x3f0000000000.

[nascheme]

New changeset 311910b by Neil Schemenauer in branch 'main':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)
311910b

[nascheme]

New changeset 1cdac61 by Miss Islington (bot) in branch '3.10':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) (GH-29122)
1cdac61