Compiler crash while analysing generator expression #1270

robertwb · 2011-08-16T06:57:41Z

Compiling test/test_itertools.py in the latest Py3.3, I get this:

Error compiling Cython file:
------------------------------------------------------------
...
            ]:
            target = [if i < len(arg) else None for arg in args](tuple([arg[i]))
                      for i in range(max(map(len, args)))]
            self.assertEqual(list(zip_longest(*args)), target)
            self.assertEqual(list(zip_longest(*args, **{})), target)
            target = [is None and 'X' or e) for e in t) for t in target](tuple((e)   # Replace None fills with 'X'
                                              ^
------------------------------------------------------------

test/test_itertools.py:597:47: Compiler crash in AnalyseDeclarationsTransform

ModuleNode.body = StatListNode(test_itertools.py:1:0)
StatListNode.stats[= StatListNode(test_itertools.py:58:0)
StatListNode.stats[0](24]) = PyClassDefNode(test_itertools.py:58:0,
    name = u'TestBasicOps')
PyClassDefNode.body = StatListNode(test_itertools.py:60:4)
StatListNode.stats[= DefNode(test_itertools.py:585:4,
    modifiers = [...](19])/0,
    name = u'test_ziplongest',
    needs_closure = True,
    num_required_args = 1,
    reqd_kw_flags_cname = '0')
DefNode.body = StatListNode(test_itertools.py:586:8)
StatListNode.stats[= ForInStatNode(test_itertools.py:586:8)
ForInStatNode.body = StatListNode(test_itertools.py:593:12)
StatListNode.stats[3](0]) = SingleAssignmentNode(test_itertools.py:597:21)
SingleAssignmentNode.rhs = ComprehensionNode(test_itertools.py:597:21,
    has_local_scope = True,
    result_is_used = True,
    use_managed_ref = True)
ComprehensionNode.loop = ForInStatNode(test_itertools.py:597:65)
ForInStatNode.body = ComprehensionAppendNode(test_itertools.py:597:21)
ComprehensionAppendNode.expr = SimpleCallNode(test_itertools.py:597:27,
    result_is_used = True,
    use_managed_ref = True)
SimpleCallNode.args[= GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 5272, in analyse_declarations: GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'Nodes.py', line 3011, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...](0])/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2091, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [   name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2236, in declare_pyfunction: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...](...]/0,
)/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')

Compiler crash traceback from this point on:
  File "/.../Cython/Compiler/Nodes.py", line 2236, in declare_pyfunction
    entry = env.declare_pyfunction(name, self.pos, allow_redefine=not self.is_wrapper)
  File "/.../Cython/Compiler/Symtab.py", line 583, in declare_pyfunction
    cname=name, visibility='private')
  File "/.../Cython/Compiler/Symtab.py", line 1508, in declare_var
    cname = '%s%s' % (self.genexp_prefix, self.parent_scope.mangle(Naming.var_prefix, name))
  File "/.../Cython/Compiler/Symtab.py", line 1399, in mangle
    return prefix + name
TypeError: cannot concatenate 'str' and 'NoneType' objects

Migrated from http://trac.cython.org/ticket/715

The text was updated successfully, but these errors were encountered:

robertwb · 2011-08-16T06:58:43Z

scoder changed description from

Compiling test/test_itertools.py in the latest Py3.3, I get this:

Error compiling Cython file:
------------------------------------------------------------
...
            ]:
            target = [if i < len(arg) else None for arg in args](tuple([arg[i]))
                      for i in range(max(map(len, args)))]
            self.assertEqual(list(zip_longest(*args)), target)
            self.assertEqual(list(zip_longest(*args, **{})), target)
            target = [is None and 'X' or e) for e in t) for t in target](tuple((e)   # Replace None fills with 'X'
                                              ^
------------------------------------------------------------

test/test_itertools.py:597:47: Compiler crash in AnalyseDeclarationsTransform

ModuleNode.body = StatListNode(test_itertools.py:1:0)
StatListNode.stats[= StatListNode(test_itertools.py:58:0)
StatListNode.stats[0](24]) = PyClassDefNode(test_itertools.py:58:0,
    name = u'TestBasicOps')
PyClassDefNode.body = StatListNode(test_itertools.py:60:4)
StatListNode.stats[= DefNode(test_itertools.py:585:4,
    modifiers = [...](19])/0,
    name = u'test_ziplongest',
    needs_closure = True,
    num_required_args = 1,
    reqd_kw_flags_cname = '0')
DefNode.body = StatListNode(test_itertools.py:586:8)
StatListNode.stats[= ForInStatNode(test_itertools.py:586:8)
ForInStatNode.body = StatListNode(test_itertools.py:593:12)
StatListNode.stats[3](0]) = SingleAssignmentNode(test_itertools.py:597:21)
SingleAssignmentNode.rhs = ComprehensionNode(test_itertools.py:597:21,
    has_local_scope = True,
    result_is_used = True,
    use_managed_ref = True)
ComprehensionNode.loop = ForInStatNode(test_itertools.py:597:65)
ForInStatNode.body = ComprehensionAppendNode(test_itertools.py:597:21)
ComprehensionAppendNode.expr = SimpleCallNode(test_itertools.py:597:27,
    result_is_used = True,
    use_managed_ref = True)
SimpleCallNode.args[= GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 5272, in analyse_declarations: GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'Nodes.py', line 3011, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...](0])/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2091, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [   name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2236, in declare_pyfunction: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...](...]/0,
)/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')

Compiler crash traceback from this point on:
  File "/home/sbehnel/source/Python/cython/cython-git/Cython/Compiler/Nodes.py", line 2236, in declare_pyfunction
    entry = env.declare_pyfunction(name, self.pos, allow_redefine=not self.is_wrapper)
  File "/home/sbehnel/source/Python/cython/cython-git/Cython/Compiler/Symtab.py", line 583, in declare_pyfunction
    cname=name, visibility='private')
  File "/home/sbehnel/source/Python/cython/cython-git/Cython/Compiler/Symtab.py", line 1508, in declare_var
    cname = '%s%s' % (self.genexp_prefix, self.parent_scope.mangle(Naming.var_prefix, name))
  File "/home/sbehnel/source/Python/cython/cython-git/Cython/Compiler/Symtab.py", line 1399, in mangle
    return prefix + name
TypeError: cannot concatenate 'str' and 'NoneType' objects

to

Compiling test/test_itertools.py in the latest Py3.3, I get this:

Error compiling Cython file:
------------------------------------------------------------
...
            ]:
            target = [if i < len(arg) else None for arg in args](tuple([arg[i]))
                      for i in range(max(map(len, args)))]
            self.assertEqual(list(zip_longest(*args)), target)
            self.assertEqual(list(zip_longest(*args, **{})), target)
            target = [is None and 'X' or e) for e in t) for t in target](tuple((e)   # Replace None fills with 'X'
                                              ^
------------------------------------------------------------

test/test_itertools.py:597:47: Compiler crash in AnalyseDeclarationsTransform

ModuleNode.body = StatListNode(test_itertools.py:1:0)
StatListNode.stats[= StatListNode(test_itertools.py:58:0)
StatListNode.stats[0](24]) = PyClassDefNode(test_itertools.py:58:0,
    name = u'TestBasicOps')
PyClassDefNode.body = StatListNode(test_itertools.py:60:4)
StatListNode.stats[= DefNode(test_itertools.py:585:4,
    modifiers = [...](19])/0,
    name = u'test_ziplongest',
    needs_closure = True,
    num_required_args = 1,
    reqd_kw_flags_cname = '0')
DefNode.body = StatListNode(test_itertools.py:586:8)
StatListNode.stats[= ForInStatNode(test_itertools.py:586:8)
ForInStatNode.body = StatListNode(test_itertools.py:593:12)
StatListNode.stats[3](0]) = SingleAssignmentNode(test_itertools.py:597:21)
SingleAssignmentNode.rhs = ComprehensionNode(test_itertools.py:597:21,
    has_local_scope = True,
    result_is_used = True,
    use_managed_ref = True)
ComprehensionNode.loop = ForInStatNode(test_itertools.py:597:65)
ForInStatNode.body = ComprehensionAppendNode(test_itertools.py:597:21)
ComprehensionAppendNode.expr = SimpleCallNode(test_itertools.py:597:27,
    result_is_used = True,
    use_managed_ref = True)
SimpleCallNode.args[= GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 5272, in analyse_declarations: GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'Nodes.py', line 3011, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...](0])/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2091, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [   name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2236, in declare_pyfunction: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...](...]/0,
)/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')

Compiler crash traceback from this point on:
  File "/.../Cython/Compiler/Nodes.py", line 2236, in declare_pyfunction
    entry = env.declare_pyfunction(name, self.pos, allow_redefine=not self.is_wrapper)
  File "/.../Cython/Compiler/Symtab.py", line 583, in declare_pyfunction
    cname=name, visibility='private')
  File "/.../Cython/Compiler/Symtab.py", line 1508, in declare_var
    cname = '%s%s' % (self.genexp_prefix, self.parent_scope.mangle(Naming.var_prefix, name))
  File "/.../Cython/Compiler/Symtab.py", line 1399, in mangle
    return prefix + name
TypeError: cannot concatenate 'str' and 'NoneType' objects

commented

robertwb · 2011-08-30T06:03:21Z

@vitek changed owner from somebody to vitja
commented

robertwb · 2011-08-30T06:03:40Z

@vitek changed resolution to fixed
status from new to closed
commented

Fixed here 155664e

robertwb · 2011-09-01T07:45:42Z

scoder changed resolution from fixed to empty
status from closed to reopened
commented

robertwb · 2011-09-01T07:47:53Z

scoder changed owner from vitja to vitja, scoder
status from reopened to new
commented

Additional fix is here:

f3f5cd7

Generator expressions must create anonymous functions, otherwise, they'd show up in locals().

robertwb · 2011-09-01T07:48:07Z

scoder changed resolution to fixed
status from new to closed
commented

robertwb · 2011-09-05T17:35:57Z

@vitek changed milestone from wishlist to 0.15.1
commented

robertwb closed this as completed Sep 1, 2011

robertwb added Type Analysis defect labels Aug 16, 2016

robertwb added this to the 0.15.1 milestone Aug 16, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compiler crash while analysing generator expression #1270

Compiler crash while analysing generator expression #1270

robertwb commented Aug 16, 2011

robertwb commented Aug 16, 2011

robertwb commented Aug 30, 2011

robertwb commented Aug 30, 2011

robertwb commented Sep 1, 2011

robertwb commented Sep 1, 2011

robertwb commented Sep 1, 2011

robertwb commented Sep 5, 2011

Compiler crash while analysing generator expression #1270

Compiler crash while analysing generator expression #1270

Comments

robertwb commented Aug 16, 2011

robertwb commented Aug 16, 2011

robertwb commented Aug 30, 2011

robertwb commented Aug 30, 2011

robertwb commented Sep 1, 2011

robertwb commented Sep 1, 2011

robertwb commented Sep 1, 2011

robertwb commented Sep 5, 2011