Crash when RawIOBase.write(b) evaluates b.format

[vadmium]

BPO	27195
Nosy	@pitrou, @benjaminp, @skrah, @vadmium, @serhiy-storchaka, @ZackerySpytz, @tirkarthi
PRs	[2.7] bpo-27195: Fix crashes in io.BufferedReader() and io.BufferedWriter() #8415 bpo-27195: Handle NULL format in memoryview objects. #9615

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-12.01:01:37.688>
created_at = <Date 2016-06-03.04:59:41.390>
labels = ['expert-IO', 'type-crash']
title = 'Crash when RawIOBase.write(b) evaluates b.format'
updated_at = <Date 2020-01-12.01:01:37.687>
user = 'https://github.com/vadmium'

bugs.python.org fields:

activity = <Date 2020-01-12.01:01:37.687>
actor = 'cheryl.sabella'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-12.01:01:37.688>
closer = 'cheryl.sabella'
components = ['IO']
creation = <Date 2016-06-03.04:59:41.390>
creator = 'martin.panter'
dependencies = []
files = []
hgrepos = []
issue_num = 27195
keywords = ['patch']
message_count = 3.0
messages = ['267038', '323974', '323975']
nosy_count = 8.0
nosy_names = ['pitrou', 'benjamin.peterson', 'stutzbach', 'skrah', 'martin.panter', 'serhiy.storchaka', 'ZackerySpytz', 'xtreak']
pr_nums = ['8415', '9615']
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue27195'
versions = ['Python 2.7']

[vadmium]

$ gdb --args ./python -c '
import io
class R(io.RawIOBase):
    def writable(self): return True
    def write(self, b):
        print("About to evaluate {!r}.format".format(b))
        b.format
        print("Never reached")
                           
b = io.BufferedWriter(R())
b.write(b"x")
b.flush()
'
[. . .]
(gdb) run
[. . .]
About to evaluate <memory at 0x7ffff6ee4380>.format
python: Objects/stringobject.c:120: PyString_FromString: Assertion `str != NULL' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff71422a8 in raise () from /usr/lib/libc.so.6
(gdb) bt full
#0 0x00007ffff71422a8 in raise () from /usr/lib/libc.so.6
No symbol table info available.
#1 0x00007ffff714372a in abort () from /usr/lib/libc.so.6
No symbol table info available.
#2 0x00007ffff713b1b7 in __assert_fail_base () from /usr/lib/libc.so.6
No symbol table info available.
#3 0x00007ffff713b262 in __assert_fail () from /usr/lib/libc.so.6
No symbol table info available.
#4 0x000000000046f7b8 in PyString_FromString (str=0x0)
at Objects/stringobject.c:120
size = 0
op = 0x7ffff7f9d160
__PRETTY_FUNCTION__ = "PyString_FromString"
#5 0x0000000000562fec in memory_format_get (self=0x7ffff6ee4380)
at Objects/memoryobject.c:330
No locals.
#6 0x00000000005587d5 in getset_get (descr=0x7ffff7fb93a8,
obj=0x7ffff6ee4380, type=0x81da40 <PyMemoryView_Type>)
at Objects/descrobject.c:146
res = 0x7ffff7fb5580
#7 0x00000000004611e6 in _PyObject_GenericGetAttrWithDict (
obj=0x7ffff6ee4380, name=0x7ffff7f9d160, dict=0x0) at Objects/object.c:1399
tp = 0x81da40 <PyMemoryView_Type>
descr = 0x7ffff7fb93a8
res = 0x0
f = 0x55876a <getset_get>
dictoffset = 8675760
dictptr = 0xf74ab600
__PRETTY_FUNCTION__ = "_PyObject_GenericGetAttrWithDict"
#8 0x000000000046164a in PyObject_GenericGetAttr (obj=0x7ffff6ee4380,
name=0x7ffff7f9d160) at Objects/object.c:1461
No locals.
#9 0x0000000000460ab0 in PyObject_GetAttr (v=0x7ffff6ee4380,
name=0x7ffff7f9d160) at Objects/object.c:1196
tp = 0x81da40 <PyMemoryView_Type>
#10 0x00000000004d0dae in PyEval_EvalFrameEx (f=0x7ffff7e31c00, throwflag=0)
at Python/ceval.c:2541
[. . .]
#16 0x0000000000421b0e in PyObject_CallMethodObjArgs (callable=0x7ffff7e601e0,
name=0x7ffff7fc3820) at Objects/abstract.c:2751
args = 0x7ffff6cb94c0
tmp = 0x7ffff6ee4350
vargs = {{gp_offset = 24, fp_offset = 48,
overflow_arg_area = 0x7fffffffddb0,
reg_save_area = 0x7fffffffdcf0}}
#17 0x00007ffff6a83283 in _bufferedwriter_raw_write (self=0x7ffff6ee1318,
start=0x8ff710 "x", '\313' <repeats 198 times>, <incomplete sequence \313>..., len=1) at /media/disk/home/proj/python/cpython/Modules/_io/bufferedio.c:1726
buf = {buf = 0x8ff710, obj = 0x0, len = 1, itemsize = 1, readonly = 1,
ndim = 1, format = 0x0, shape = 0x7fffffffdde0, strides = 0x0,
suboffsets = 0x0, smalltable = {9432288, 140737488346688},
internal = 0x0}
memobj = 0x7ffff6ee4380
res = 0x7fffffffde60
n = 5605226
errnum = 0
#18 0x00007ffff6a8360d in _bufferedwriter_flush_unlocked (self=0x7ffff6ee1318)
at /media/disk/home/proj/python/cpython/Modules/_io/bufferedio.c:1773
written = 0
n = 1
rewind = 0
#19 0x00007ffff6a7faf6 in buffered_flush_and_rewind_unlocked (
self=0x7ffff6ee1318)
at /media/disk/home/proj/python/cpython/Modules/_io/bufferedio.c:756
res = 0x7ffff7fc8700
#20 0x00007ffff6a7fd52 in buffered_flush (self=0x7ffff6ee1318, args=0x0)
at /media/disk/home/proj/python/cpython/Modules/_io/bufferedio.c:783
res = 0x7ffff6ee1318
#21 0x00000000004d824c in call_function (pp_stack=0x7fffffffe078, oparg=0)
at Python/ceval.c:4334
meth = 0x7ffff6a7fc4f <buffered_flush>
self = 0x7ffff6ee1318
flags = 4
tstate = 0x8461b0
na = 0
nk = 0
n = 0
pfunc = 0x7ffff7f151e8
func = 0x7ffff6cb86f0
x = 0x7ffff6ee1318
w = 0x7ffff7f151e8
[. . .]

[ZackerySpytz]

Ping.

[skrah]

PyBUF_RECORDS pulls in PyBUF_STRIDES, which should IMO not be done in a stable release.

Maybe (PyBUF_CONTIG_RO|PyBUF_FORMAT), but I'm uneasy even about that. The potential for breakage is huge.