Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious warning "Unused entry 'genexpr'" #1699

Open
mhsmith opened this issue May 6, 2017 · 18 comments
Open

Spurious warning "Unused entry 'genexpr'" #1699

mhsmith opened this issue May 6, 2017 · 18 comments

Comments

@mhsmith
Copy link

mhsmith commented May 6, 2017

Cython version 0.25.2

l = [1,2,3]

def f():
    print(x for x in l)
$ cython -Wextra -Werror foo.pyx

Error compiling Cython file:
------------------------------------------------------------
...
l = [1,2,3]

def f():
    print(x for x in l)
           ^
------------------------------------------------------------

foo.pyx:4:12: Unused entry 'genexpr'

This only happens when the generator expression is in a function, rather than at the module level. Surrounding the expression with parentheses doesn't help. Converting it to a list comprehension does remove the warning, but will make performance worse, especially if the generator expression would not have run to completion.

I like to use -Wextra -Werror because the "unused" warnings often indicate a genuine error in my code.

@LordAro
Copy link
Contributor

LordAro commented Dec 14, 2018

Still getting this with 0.29.1, any chance of anyone looking at it? Having to wrap expressions in [] is unfortunate...

@DrissiReda
Copy link

DrissiReda commented Jul 2, 2019

Haven't there been fixes for this issue since? Still having it on 0.29.10

@wkschwartz
Copy link

wkschwartz commented Aug 27, 2019

Getting this warning in 0.29.12

@scoder
Copy link
Contributor

scoder commented Aug 28, 2019

Probably not difficult to do. Might be enough to mark the Entry of the generator expression as entry.used = True.

@scoder
Copy link
Contributor

scoder commented Aug 28, 2019

Test could possibly be implemented by setting -Werror for the generator expression tests – unless that triggers other 'fatal' warnings.

@shssf
Copy link

shssf commented Aug 23, 2020

I have the same issue.
I didn't understand - what is the workaround?

@da-woods
Copy link
Contributor

I have the same issue.
I didn't understand - what is the workaround?

There isn't a workaround (except for the obvious of not using -Werror). There's an untested suggestion for what a potential patch might be.

@LordAro
Copy link
Contributor

LordAro commented Aug 23, 2020

You can use lists explicitly instead of generator expressions. Obviously this doesn't work in all cases, and can significantly increase memory usage, but it will work some of the time...

print(x for x in foo)

->

print([x for x in foo])

@scoder
Copy link
Contributor

scoder commented Aug 23, 2020

Note also that short running list comprehensions can be much faster than the corresponding generator expression.

@scoder
Copy link
Contributor

scoder commented Aug 23, 2020

I tried enabling warning tracking for the entire test suite in #3788 and didn't see this specific warning anywhere. Weird. The OP's test case is simple enough to be covered.

@da-woods
Copy link
Contributor

The OP's test case is simple enough to be covered.

The one issue with OP's test case is that it doesn't actually run the generator expression - it just prints <generator object at 0xSOME_ADDRESS> so it's not likely to be tested. With that said, I see the same warning with the more useful print(list(x for x in l)) (with -Wexta on the command-line)

@WillDaSilva
Copy link

This issue is still present in Cython 3.0a6

@scoder scoder closed this as completed in da26b4a May 20, 2021
@scoder scoder added this to the 0.29.24 milestone May 20, 2021
@scoder
Copy link
Contributor

scoder commented May 20, 2021

I pushed a proposed fix, but without a test, since this is so long standing. Please try out the 0.29.x and master branches to see if it resolves the issue on your side.

@WillDaSilva
Copy link

I created a fresh virtual environment on Python 3.9.2 as follows:

will@will-desktop ~ $ python -V
Python 3.9.2
will@will-desktop ~ $ python -m venv venv
will@will-desktop ~ $ source venv/bin/activate

Then installed Cython checked out onto the commit with the fix:

(venv) will@will-desktop ~ $ cd cython/
(venv) will@will-desktop ~/cython $ git rev-parse HEAD
da26b4a2131993df9e68c34213ca688b43b5970c
(venv) will@will-desktop ~/cython $ python setup.py install

<install output trimmed>

Installed /home/will/venv/lib/python3.9/site-packages/Cython-0.29.23-py3.9-linux-x86_64.egg
Processing dependencies for Cython==0.29.23
Finished processing dependencies for Cython==0.29.23
(venv) will@will-desktop ~/cython $

I then changed directories to the project I was working in where the warnings were showing up, and ran setup.py build_ext:

(venv) will@will-desktop ~/pyproject $ python setup.py build_ext
Traceback (most recent call last):
  File "/home/will/pyproject/setup.py", line 24, in <module>
    from Cython.Build import cythonize
  File "/home/will/venv/lib/python3.9/site-packages/Cython-0.29.23-py3.9-linux-x86_64.egg/Cython/Build/__init__.py", line 1, in <module>
    from .Dependencies import cythonize
  File "/home/will/venv/lib/python3.9/site-packages/Cython-0.29.23-py3.9-linux-x86_64.egg/Cython/Build/Dependencies.py", line 48, in <module>
    from ..Compiler.Main import Context, CompilationOptions, default_options
  File "/home/will/venv/lib/python3.9/site-packages/Cython-0.29.23-py3.9-linux-x86_64.egg/Cython/Compiler/Main.py", line 28, in <module>
    from .Scanning import PyrexScanner, FileSourceDescriptor
  File "Cython/Compiler/Scanning.py", line 1, in init Cython.Compiler.Scanning
  File "/home/will/venv/lib/python3.9/site-packages/Cython-0.29.23-py3.9-linux-x86_64.egg/Cython/Plex/__init__.py", line 36, in <module>
    from .Lexicons import Lexicon, State
  File "/home/will/venv/lib/python3.9/site-packages/Cython-0.29.23-py3.9-linux-x86_64.egg/Cython/Plex/Lexicons.py", line 14, in <module>
    from . import DFA
  File "Cython/Plex/DFA.py", line 1, in init Cython.Plex.DFA
AttributeError: module 'Cython.Plex.Actions' has no attribute 'Method'
(venv) will@will-desktop ~/pyproject $

@scoder
Copy link
Contributor

scoder commented May 21, 2021

That error makes no sense at all to me. Something must have gone wrong with the installation. Did you already retry this a second time? Could you also try pip install . instead to install Cython?

@WillDaSilva
Copy link

I tried twice with python setup.py install, as that was what the INSTALL.txt said to do. I tried again (in a new virtual environment) with pip install ., but it made no difference. I then tried with Python 3.6.10, 3.7.6, 3.8.8, and 3.9.2. For each of them I created a new virtual environment, ran pip install -U pip setuptools wheel toml psutil numpy (the build dependencies for my project), then pip install . to build Cython. I then ran python setup.py build_ext for my project, and each time the Unused entry 'genexpr' warnings were still present. Maybe the previous error was due to some missing/outdated build dependency? In hindsight I should've installed Cython before installing my project's build dependencies.

@mhsmith
Copy link
Author

mhsmith commented Aug 27, 2021

@scoder: I'm sorry, but this isn't fixed at all. My simple 3-line example above still fails, with almost the same message:

$ cython --version
Cython version 0.29.24
$ cython -Wextra -Werror foo.pyx
/home/smith/.virtualenvs/pypi/lib/python3.8/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/smith/tmp/foo.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
l = [1,2,3]

def f():
    print(x for x in l)
         ^
------------------------------------------------------------

foo.pyx:4:10: Unused entry 'genexpr'

Please reopen the issue.

@da-woods
Copy link
Contributor

da-woods commented Sep 3, 2021

This turns out to be a little more complicated:

else: # declare entry stub
self.declare_var(name, py_object_type, pos, visibility=visibility)
entry = self.declare_var(None, py_object_type, pos,
cname=name, visibility='private')
entry.name = EncodedString(name)
entry.qualified_name = self.qualify_name(name)
entry.signature = pyfunction_signature
entry.is_anonymous = True
return entry

There are two entries created for the GeneratorDefNode. A "stub" entry that's put into the symtab but is not returned, and an anonymous entry that is returned. The "stub" entry is the one that's discovered and reported unused, while the other one is the one that GeneratorDefNode has access to.

I'm not really sure I understand the purpose of this scheme so I'm reluctant to mess with it. But that's the rough outline

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants