[easy C issue] ctypes: segfault with large number of callback arguments

[meadori]

BPO	13097
Nosy	@amauryfa, @abalkin, @vstinner, @meadori, @miss-islington, @furkanonder
PRs	bpo-13097: fix segfault in ctypes callback invocation #19914 [3.9] bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914) #20453 [3.8] bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914) #20454 [3.7] bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914) #20455

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/meadori'
closed_at = <Date 2020-05-27.15:54:46.297>
created_at = <Date 2011-10-04.03:57:17.101>
labels = ['easy', '3.8', '3.9', '3.10', 'ctypes', '3.7', 'type-crash']
title = '[easy C issue] ctypes: segfault with large number of callback arguments'
updated_at = <Date 2020-05-27.15:54:46.291>
user = 'https://github.com/meadori'

bugs.python.org fields:

activity = <Date 2020-05-27.15:54:46.291>
actor = 'vstinner'
assignee = 'meador.inge'
closed = True
closed_date = <Date 2020-05-27.15:54:46.297>
closer = 'vstinner'
components = ['ctypes']
creation = <Date 2011-10-04.03:57:17.101>
creator = 'meador.inge'
dependencies = []
files = []
hgrepos = []
issue_num = 13097
keywords = ['patch', 'easy (C)', 'newcomer friendly']
message_count = 13.0
messages = ['144852', '145259', '148648', '148701', '148712', '149508', '368001', '368064', '370093', '370099', '370102', '370103', '370105']
nosy_count = 7.0
nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'vstinner', 'meador.inge', 'python-dev', 'miss-islington', 'furkanonder']
pr_nums = ['19914', '20453', '20454', '20455']
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue13097'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[meadori]

Reproducible in 2.7 and tip:

[meadori@motherbrain cpython]$ ./python 
Python 3.3.0a0 (default:61de28fa5537+d05350c14e77+, Oct  3 2011, 21:47:04) 
[GCC 4.6.0 20110603 (Red Hat 4.6.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> NARGS = 2 ** 20
>>> proto = CFUNCTYPE(None, *(c_int,) * NARGS)
>>> def func(*args):
...    return (1, "abc", None)
... 
>>> cb = proto(func)
>>> cb(*(1,) * NARGS)
Segmentation fault (core dumped)

[meadori]

As mentioned in bpo-12881, this issue is a result of an unbounded 'alloca' call that trashes the stack.

[amauryfa]

Right, alloca() could be replaced by some malloc(), but is it really useful? After all, when a C function calls back to Python, all arguments needs to be pushed to the stack anyway.

[meadori]

On Wed, Nov 30, 2011 at 6:20 AM, Amaury Forgeot d'Arc
<report@bugs.python.org> wrote:

Right, alloca() could be replaced by some malloc(), but is it really useful?  After all, when a C function calls back to Python, all arguments needs to be > pushed to the stack anyway.

The case is somewhat pathological. However, there are *four* 'alloca'
calls in '_ctypes_callproc', three of which are proportional to
'argcount'. And as you point out there is an additional stack
allocation inside of 'libffi' when the callback is actually called
(also using 'alloca').

I see two reasons switching to 'malloc' might be beneficial: (1) by
shifting some of the allocation to dynamic allocation we may reduce
the chance that we blow the stack at all. Keep in mind that this gets
more likely if the C ffi function is called from a deep in a call tree
and *not* necessarily with just a huge number of parameters. (2) if
resources are exhausted, then we can exit more gracefully. Out of
dynamic memory errors can actually be handled as opposed to an
'alloca' call that ends up allocating more than is left.

That being said, if this does get changed it is low priority.

[vstinner]

Is there really an use case where you need 2 ** 20 (1,048,576) arguments? If yes, I'm not against the torture in this case :-) If no, why not raising an error if there are too many arguments? E.g. limit to 1,024 arguments or maybe just 10?

[meadori]

On Thu, Dec 1, 2011 at 2:11 AM, STINNER Victor <report@bugs.python.org> wrote:

Is there really an use case where you need 2 ** 20 (1,048,576) arguments? If yes, I'm not against the torture in this case :-)

Not very likely :-) However, the segfault can occur with less
arguments in low stack space situations (e.g. deep callstack).

If no, why not raising an error if there are too many arguments? E.g. limit to 1,024 arguments or maybe just 10?

That is certainly an option.

[furkanonder]

The issue continues in python 3.8.2.

[vstinner]

I suggest to raise an exception if it's called with more than 1024 arguments.

[vstinner]

New changeset 29a1384 by Sean Gillespie in branch 'master':
bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914)
29a1384

[miss-islington]

New changeset 788d7bf by Miss Islington (bot) in branch '3.9':
bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914)
788d7bf

[miss-islington]

New changeset 1c4dcaf by Miss Islington (bot) in branch '3.7':
bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914)
1c4dcaf

[miss-islington]

New changeset a285af7 by Miss Islington (bot) in branch '3.8':
bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914)
a285af7

[vstinner]

Thanks Meador Inge for the bug report and thanks Sean Gillespie for the fix! It just took 9 years to fix this corner case ;-)

Copy of the comment on the PR:
#19914 (review)

I tried to rewrite _ctypes_callproc() to use PyMem_Malloc() instead of alloca(), but it's quite complicated. There are 3 arrays with a length of argcount items: args, avalues, atypes. Moreover, resbuf is also allocated with alloca(). When using PyMem_Malloc(), error handling is much more complicated.

I also tried to restrict the overall usage of stack memory to 4096 bytes (size of one page on x86), but users would be surprised by CTYPES_MAX_ARGCOUNT value.

I would say that raising an exception is better than crashing for a lot of arguments. If someone is blocked by this new limitation, in that case we can revisit the PyMem_Malloc() idea.