PyNode_Compile() crashes in Python 3.8.

[grahamd]

BPO	37072
Nosy	@gvanrossum, @vstinner, @encukou, @ambv
PRs	bpo-37072: Fix crash in PyAST_FromNodeObject() when flags is NULL #13634

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gvanrossum'
closed_at = <Date 2019-05-28.23:45:50.753>
created_at = <Date 2019-05-28.01:52:54.760>
labels = ['3.8', 'type-crash', 'release-blocker']
title = 'PyNode_Compile() crashes in Python 3.8.'
updated_at = <Date 2019-05-28.23:45:50.753>
user = 'https://bugs.python.org/grahamd'

bugs.python.org fields:

activity = <Date 2019-05-28.23:45:50.753>
actor = 'gvanrossum'
assignee = 'gvanrossum'
closed = True
closed_date = <Date 2019-05-28.23:45:50.753>
closer = 'gvanrossum'
components = []
creation = <Date 2019-05-28.01:52:54.760>
creator = 'grahamd'
dependencies = []
files = []
hgrepos = []
issue_num = 37072
keywords = ['patch']
message_count = 6.0
messages = ['343727', '343728', '343750', '343761', '343811', '343834']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'vstinner', 'grahamd', 'petr.viktorin', 'lukasz.langa']
pr_nums = ['13634']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue37072'
versions = ['Python 3.8']

[grahamd]

The code:

#include <Python.h>

int
main(int argc, char *argv[])
{
    FILE *fp = NULL;
    PyObject *co = NULL;
    struct _node *n = NULL;
    const char * filename = "/dev/null";

    Py_Initialize();

fprintf(stderr, "START\n");

    fp = fopen(filename, "r");

fprintf(stderr, "CALL PyParser_SimpleParseFile()\n");

    n = PyParser_SimpleParseFile(fp, filename, Py_file_input);

fprintf(stderr, "CALL PyNode_Compile()\n");

    co = (PyObject *)PyNode_Compile(n, filename);

fprintf(stderr, "DONE\n");

    Py_Finalize();

    return 0;
}

has worked fine since Python 2.3 (and maybe earlier) through Python 3.7, but now crashes in Python 3.8.

It crashes in PyNode_Compile().

START
CALL PyParser_SimpleParseFile()
CALL PyNode_Compile()
Segmentation fault: 11

Although it is part of the public interface of compile.h, the PyNode_Compile() seems never to actually be called anywhere in Python itself, and perhaps isn't even covered by tests. So if Python 3.8 internal changes mean this function implementation needs to be changed, that fact may have been missed.

[grahamd]

FWIW, this was occurring on macOS. Not been able to test on other platforms.

[encukou]

Looks like this is caused by: https://github.com/python/cpython/pull/12086/files#diff-4d35cf8992b795c5e97e9c8b6167cb34R787

PyAST_FromNodeObject doesn't ignore flags any more, so when PyNode_Compile passes NULL flags, it crashes.

(This is unfamiliar code for me; I won't have time to fix & test properly this week.)

[vstinner]

I mark this issue as a release blocker.

[gvanrossum]

OK, I'll look into this.

[gvanrossum]

New changeset 77f0ed7 by Guido van Rossum in branch 'master':
bpo-37072: Fix crash in PyAST_FromNodeObject() when flags is NULL (bpo-13634)
77f0ed7