Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] reedsolo Compiler crash in AnalyseExpressionsTransform (regression in 3.0.4) #5781

Closed
hroncok opened this issue Oct 23, 2023 · 1 comment

Comments

@hroncok
Copy link
Contributor

hroncok commented Oct 23, 2023

Describe the bug

When testing the impact of Cython upgrade from 3.0.2 to 3.0.4 in Fedora, I discovered reedsolo 1.7.0 failed to build.

I was able to reproduce this outside of our RPM environment, on x86_64.

The problem does not happen with Cython 3.0.3, only 3.0.4.

Code to reproduce the behaviour:

$ git clone git@github.com:tomerfiliba-org/reedsolomon.git
...
$ cd reedsolomon/
[reedsolomon (master)]$ git switch -d v1.7.0
HEAD is now at 9d27c91 wording
[reedsolomon ((v1.7.0))]$ python3.12 -m venv __venv__
[reedsolomon ((v1.7.0))]$ . __venv__/bin/activate
(__venv__) [reedsolomon ((v1.7.0))]$ pip install setuptools Cython==3.0.4
...
Successfully installed Cython-3.0.4 setuptools-68.2.2
(__venv__) [reedsolomon ((v1.7.0))]$ python setup.py --cythonize build
Cython is installed, building creedsolo module
[1/1] Cythonizing creedsolo.pyx

Error compiling Cython file:
------------------------------------------------------------
...

    # For each possible value in the galois field 2^8, we will pre-compute the logarithm and anti-logarithm (exponential) of this value
    # To do that, we generate the Galois Field F(2^p) by building a list starting with the element 0 followed by the (p-1) successive powers of the generator a : 1, a, a^1, a^2, ..., a^(p-1).
    x = 1
    for i in xrange(field_charac): # we could skip index 255 which is equal to index 0 because of modulo: g^255==g^0
        gf_exp[i] = x # compute anti-log for this value and store it in a table
              ^
------------------------------------------------------------

creedsolo.pyx:202:14: Compiler crash in AnalyseExpressionsTransform

ModuleNode.body = StatListNode(creedsolo.pyx:86:0)
StatListNode.stats[13] = StatListNode(creedsolo.pyx:181:0)
StatListNode.stats[0] = DefNode(creedsolo.pyx:181:0,
    doc = "Precompute the logarithm and anti-log tables for faster computation later, using the provided primitive polynomial.\n    These tables are used for multiplication/division since addition/substraction are simple XOR operations inside GF of characteristic 2.\n    The basic idea is quite simple: since b**(log_b(x), log_b(y)) == x * y given any number b (the base or generator of the logarithm), then we can use any number b to precompute logarithm and anti-log (exponentiation) tables to use for multiplying two numbers x and y.\n    That's why when we use a different base/generator number, the log and anti-log tables are drastically different, but the resulting computations are the same given any such tables.\n    For more infos, see https://en.wikipedia.org/wiki/Finite_field_arithmetic#Implementation_tricks\n    ",
    is_cyfunction = True,
    modifiers = [...]/0,
    name = 'init_tables',
    np_args_idx = [...]/0,
    outer_attrs = [...]/2,
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
File 'Nodes.py', line 397, in analyse_expressions: StatListNode(creedsolo.pyx:182:4,
    is_terminator = True)
File 'Nodes.py', line 7556, in analyse_expressions: ForInStatNode(creedsolo.pyx:201:4)
File 'Nodes.py', line 397, in analyse_expressions: StatListNode(creedsolo.pyx:202:8)
File 'Nodes.py', line 6026, in analyse_expressions: SingleAssignmentNode(creedsolo.pyx:202:20)
File 'Nodes.py', line 6164, in analyse_types: SingleAssignmentNode(creedsolo.pyx:202:20)
File 'ExprNodes.py', line 4085, in analyse_target_types: IndexNode(creedsolo.pyx:202:14,
    is_subscript = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 4123, in analyse_base_and_index_types: IndexNode(creedsolo.pyx:202:14,
    is_subscript = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 4349, in analyse_as_buffer_operation: IndexNode(creedsolo.pyx:202:14,
    is_subscript = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 4968, in analyse_types: MemoryViewIndexNode(creedsolo.pyx:202:14,
    is_memview_index = True,
    memslice_index = True,
    result_is_used = True,
    use_managed_ref = True,
    writable_needed = True)

Compiler crash traceback from this point on:
  File ".../reedsolomon/__venv__/lib64/python3.12/site-packages/Cython/Compiler/ExprNodes.py", line 4968, in analyse_types
    performance_hint(index.pos, "Index should be typed for more efficient access")
TypeError: performance_hint() missing 1 required positional argument: 'env'
Traceback (most recent call last):
  File ".../reedsolomon/setup.py", line 32, in <module>
    extensions = cythonize([ Extension('creedsolo', ['creedsolo.pyx']) ], force=True)  # this may fail hard if Cython is installed but there is no C compiler for current Python version, and we have no way to know. Alternatively, we could supply exclude_failures=True , but then for those who really want the cythonized compiled extension, it would be much harder to debug
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../reedsolomon/__venv__/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
    cythonize_one(*args)
  File ".../reedsolomon/__venv__/lib64/python3.12/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: creedsolo.pyx

As a thing to paste:

git clone git@github.com:tomerfiliba-org/reedsolomon.git
cd reedsolomon/
git switch -d v1.7.0
python3.12 -m venv __venv__
. __venv__/bin/activate
pip install setuptools Cython==3.0.4
python setup.py --cythonize build

Expected behaviour

With 3.0.2 and 3.0.3 this compiles fine.

OS

Linux

Python version

3.12.0

Cython version

3.0.4

Additional context

No response

@hroncok
Copy link
Contributor Author

hroncok commented Oct 23, 2023

Looks like a dupe of #5771, sorry for the noise.

@hroncok hroncok closed this as completed Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant