New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"usage of Generator.Function is deprecated" error coming from PyTest code, not ours #4400

Closed
nickwilliams-eventbrite opened this Issue Nov 15, 2018 · 15 comments

Comments

Projects
None yet
3 participants
@nickwilliams-eventbrite

nickwilliams-eventbrite commented Nov 15, 2018

We upgraded one of our projects to PyTest 4.0.0, and our builds started failing with this deprecation error. We're filtering out the warning for now to get our build running, because there's nothing we can do about it. The warning isn't coming from our code. As you can see from the stacktrace below, the warning is coming from PyTest code.

self = <CallInfo when='collect' exception: usage of Generator.Function is deprecated, please use pytest.Function instead>
func = <function <lambda> at 0x7fa771266d70>, when = 'collect'
treat_keyboard_interrupt_as_exception = False

    def __init__(self, func, when, treat_keyboard_interrupt_as_exception=False):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

../venv/lib/python2.7/site-packages/_pytest/runner.py:211: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../venv/lib/python2.7/site-packages/_pytest/runner.py:272: in <lambda>
    call = CallInfo(lambda: list(collector.collect()), "collect")
../venv/lib/python2.7/site-packages/_pytest/python.py:769: in collect
    function_class = self.Function
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = _CompatProperty(name='Function')
obj = <Generator 'test_parse_valid_tokens'>
owner = <class '_pytest.python.Generator'>

    def __get__(self, obj, owner):
        if obj is None:
            return self
    
        from _pytest.deprecated import COMPAT_PROPERTY
    
        warnings.warn(
>           COMPAT_PROPERTY.format(name=self.name, owner=owner.__name__), stacklevel=2
        )
E       RemovedInPytest4Warning: usage of Generator.Function is deprecated, please use pytest.Function instead

../venv/lib/python2.7/site-packages/_pytest/nodes.py:70: RemovedInPytest4Warning

Information from the test startup:

============================= test session starts ==============================
platform linux2 -- Python 2.7.12, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
django settings: xxxxxxxxxxxx (from ini file)
rootdir: xxxxxxxxxxxxxxx, inifile: setup.cfg
plugins: django-2.9.1, forked-0.2, pysoa-0.55.0, mock-1.10.0, cov-2.6.0, xdist-1.24.1

Here's a quick checklist in what to include:

  • Include a detailed description of the bug or suggestion
  • pip list of the virtual environment you are using (I've shared what I can without divulging confidential information)
  • pytest and operating system versions
  • Minimal example if possible (Not possible because I don't know what exactly causes this to replicate and I can't share the project)
@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Nov 15, 2018

The error happens inside pytest because when you turn errors into warnings the exception is raised by the calling code, but the cause of the exception is still because there are yield tests in your testing code. Look for the yield keyword in your tests directory and I'm sure you will find those. 👍

Closing this for now, but feel free to follow up with further questions.

@nicoddemus nicoddemus closed this Nov 15, 2018

@nickwilliams-eventbrite

This comment has been minimized.

nickwilliams-eventbrite commented Nov 15, 2018

I'm both confused and not sure I agree with your assessment. I get that the error is due to the flag turning warnings into errors. That's fine. My concern is sussing out the source of the warning in the first place. I'm confused because the warning says nothing about yield statements. The warning says that something is using Generator.Function, which is deprecated and has been replaced with pytest.Function. I'm in disagreement because the stack trace says that _pytest/python.py is the code using Generator.Function in violation of the deprecation.

@nicoddemus nicoddemus reopened this Nov 15, 2018

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Nov 15, 2018

Oh I'm sorry, I read your post on the run and assumed it was related to yield tests.

I'm still on the run so I will have to look into this with more detail later, but meanwhile perhaps other maintainers can help out.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Nov 15, 2018

Can you also post the entire traceback please?

@nickwilliams-eventbrite

This comment has been minimized.

nickwilliams-eventbrite commented Nov 15, 2018

Yes, the entire traceback is posted above in the issue body. That's all there was. There was no other traceback above or below that.

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

the broken test is called test_parse_valid_tokens

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

we also ought to take a look why its not blowing up while collecting the generator itself

@nickwilliams-eventbrite

This comment has been minimized.

nickwilliams-eventbrite commented Nov 15, 2018

What about the test test_parse_valid_tokens is broken?

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

@nickwilliams-eventbrite can you please show the output if you set up -rw - i suspect there is a incorrectly categorized deprecation which is why the real warning rror is missing

test_parse_valid_tokens is a yield based test - its a feature that has been deprecated since ages

is it possible your codebase has a warning filter that hides the deprecation about generators?

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

https://github.com/pytest-dev/pytest/blob/master/src/_pytest/python.py#L767 that one strikes me as a strange mistake on our part

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

so we have 2 bugs there

  • warning is triggered after error prone collect
  • Function is not obtained using self._getcustomclass

i feel its necessary to put this into a bugfix release even tho we completely remove it in 4.1

@nickwilliams-eventbrite

This comment has been minimized.

nickwilliams-eventbrite commented Nov 15, 2018

We have the following filters:

    # FIXME: Ignored warnings, until we can use `pytest-django` >= 3.0.0
    ignore:MarkInfo objects are deprecated.*:DeprecationWarning
    ignore:getfuncargvalue is deprecated, use getfixturevalue.*:DeprecationWarning
    # FIXME: Ignored warnings, until we fix it in soa repository
    ignore:encoding is deprecated, Use raw=False instead:PendingDeprecationWarning

I don't think any of those would filter out a generator warning.

You were indeed right. That test was a generator test:

def test_parse_valid_tokens():
    for token in VALID_TOKENS:
        yield check_parse, token

Full disclosure: I've never seen that test before, so I'm admittedly playing catch-up here. What was the expected behavior of that test back when it was supported (I've never seen a test do that before)?

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

@nickwilliams-eventbrite that particular test looks like its a really simple transform to a parameterized test

@pytest.mark.parametrize('token', VALID_TOKENS)
def test_parse_valid_token(token):
    check_parse(token)
@nickwilliams-eventbrite

This comment has been minimized.

nickwilliams-eventbrite commented Nov 15, 2018

Okay. So the test as-is is a generator of tests, and the existing behavior (which is being removed in 4.1) was that it would iterate over the generator and run each test with the supplied arguments. Hence why replacing it with parameterization is the best course of action. Do I understand that correctly?

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Nov 15, 2018

correct - you are really lucky the test was stateless ^^

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Nov 16, 2018

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Nov 16, 2018

@blueyed blueyed closed this in 950d30e Nov 17, 2018

blueyed added a commit that referenced this issue Nov 17, 2018

Merge pull request #4403 from RonnyPfannschmidt/fix-4400
Fix #4400 - rearrange the handling of yield test warnings/errors

peterbe added a commit to mozilla-services/tecken that referenced this issue Nov 26, 2018

Update pytest to 4.0.1 (#1359)
This PR updates [pytest](https://pypi.org/project/pytest) from **4.0.0** to **4.0.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 4.0.1
   ```
   =========================

Bug Fixes
---------

- `3952 &lt;https://github.com/pytest-dev/pytest/issues/3952&gt;`_: Display warnings before &quot;short test summary info&quot; again, but still later warnings in the end.


- `4386 &lt;https://github.com/pytest-dev/pytest/issues/4386&gt;`_: Handle uninitialized exceptioninfo in repr/str.


- `4393 &lt;https://github.com/pytest-dev/pytest/issues/4393&gt;`_: Do not create ``.gitignore``/``README.md`` files in existing cache directories.


- `4400 &lt;https://github.com/pytest-dev/pytest/issues/4400&gt;`_: Rearrange warning handling for the yield test errors so the opt-out in 4.0.x correctly works.


- `4405 &lt;https://github.com/pytest-dev/pytest/issues/4405&gt;`_: Fix collection of testpaths with ``--pyargs``.


- `4412 &lt;https://github.com/pytest-dev/pytest/issues/4412&gt;`_: Fix assertion rewriting involving ``Starred`` + side-effects.


- `4425 &lt;https://github.com/pytest-dev/pytest/issues/4425&gt;`_: Ensure we resolve the absolute path when the given ``--basetemp`` is a relative path.



Trivial/Internal Changes
------------------------

- `4315 &lt;https://github.com/pytest-dev/pytest/issues/4315&gt;`_: Use ``pkg_resources.parse_version`` instead of ``LooseVersion`` in minversion check.


- `4440 &lt;https://github.com/pytest-dev/pytest/issues/4440&gt;`_: Adjust the stack level of some internal pytest warnings.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>

bors bot added a commit to mozilla/normandy that referenced this issue Nov 26, 2018

Merge #1629
1629: Scheduled weekly dependency update for week 47 r=mythmon a=pyup-bot






### Update [botocore](https://pypi.org/project/botocore) from **1.12.47** to **1.12.51**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.12.51
   ```
   =======

* api-change:``amplify``: Update amplify client to latest version
* api-change:``transfer``: Update transfer client to latest version
* api-change:``snowball``: Update snowball client to latest version
* api-change:``robomaker``: Update robomaker client to latest version
* api-change:``s3``: Update s3 client to latest version
* api-change:``datasync``: Update datasync client to latest version
   ```
   
  
  
   ### 1.12.50
   ```
   =======

* api-change:``rekognition``: Update rekognition client to latest version
   ```
   
  
  
   ### 1.12.49
   ```
   =======

* api-change:``autoscaling-plans``: Update autoscaling-plans client to latest version
* api-change:``xray``: Update xray client to latest version
* api-change:``devicefarm``: Update devicefarm client to latest version
* api-change:``ssm``: Update ssm client to latest version
* api-change:``medialive``: Update medialive client to latest version
* api-change:``redshift``: Update redshift client to latest version
* api-change:``rds-data``: Update rds-data client to latest version
* api-change:``appsync``: Update appsync client to latest version
* api-change:``cloudwatch``: Update cloudwatch client to latest version
* api-change:``quicksight``: Update quicksight client to latest version
* api-change:``cloudfront``: Update cloudfront client to latest version
   ```
   
  
  
   ### 1.12.48
   ```
   =======

* api-change:``lightsail``: Update lightsail client to latest version
* api-change:``workspaces``: Update workspaces client to latest version
* api-change:``workdocs``: Update workdocs client to latest version
* api-change:``batch``: Update batch client to latest version
* api-change:``ec2``: Update ec2 client to latest version
* api-change:``devicefarm``: Update devicefarm client to latest version
* api-change:``rds``: Update rds client to latest version
* api-change:``cloudformation``: Update cloudformation client to latest version
* api-change:``lambda``: Update lambda client to latest version
* api-change:``config``: Update config client to latest version
* api-change:``cloudtrail``: Update cloudtrail client to latest version
* api-change:``mediaconvert``: Update mediaconvert client to latest version
* api-change:``iot``: Update iot client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/botocore
  - Changelog: https://pyup.io/changelogs/botocore/
  - Repo: https://github.com/boto/botocore
</details>





### Update [Pygments](https://pypi.org/project/Pygments) from **2.2.0** to **2.3.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 2.3.0
   ```
   -------------
(released Nov 25, 2018)

- Added lexers:

  * Fennel (PR783)
  * HLSL (PR675)

- Updated lexers:

  * Dockerfile (PR714)

- Minimum Python versions changed to 2.7 and 3.5
- Added support for Python 3.7 generator changes (PR772)
- Fix incorrect token type in SCSS for single-quote strings (1322)
- Use `terminal256` formatter if `TERM` contains `256` (PR666)
- Fix incorrect handling of GitHub style fences in Markdown (PR741, 1389)
- Fix `%a` not being highlighted in Python3 strings (PR727)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pygments
  - Changelog: https://pyup.io/changelogs/pygments/
  - Homepage: http://pygments.org/
</details>





### Update [boto3](https://pypi.org/project/boto3) from **1.9.47** to **1.9.51**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.9.51
   ```
   ======

* api-change:``amplify``: [``botocore``] Update amplify client to latest version
* api-change:``transfer``: [``botocore``] Update transfer client to latest version
* api-change:``snowball``: [``botocore``] Update snowball client to latest version
* api-change:``robomaker``: [``botocore``] Update robomaker client to latest version
* api-change:``s3``: [``botocore``] Update s3 client to latest version
* api-change:``datasync``: [``botocore``] Update datasync client to latest version
   ```
   
  
  
   ### 1.9.50
   ```
   ======

* api-change:``rekognition``: [``botocore``] Update rekognition client to latest version
   ```
   
  
  
   ### 1.9.49
   ```
   ======

* api-change:``autoscaling-plans``: [``botocore``] Update autoscaling-plans client to latest version
* api-change:``xray``: [``botocore``] Update xray client to latest version
* api-change:``devicefarm``: [``botocore``] Update devicefarm client to latest version
* api-change:``ssm``: [``botocore``] Update ssm client to latest version
* api-change:``medialive``: [``botocore``] Update medialive client to latest version
* api-change:``redshift``: [``botocore``] Update redshift client to latest version
* api-change:``rds-data``: [``botocore``] Update rds-data client to latest version
* api-change:``appsync``: [``botocore``] Update appsync client to latest version
* api-change:``cloudwatch``: [``botocore``] Update cloudwatch client to latest version
* api-change:``quicksight``: [``botocore``] Update quicksight client to latest version
* api-change:``cloudfront``: [``botocore``] Update cloudfront client to latest version
   ```
   
  
  
   ### 1.9.48
   ```
   ======

* api-change:``lightsail``: [``botocore``] Update lightsail client to latest version
* api-change:``workspaces``: [``botocore``] Update workspaces client to latest version
* api-change:``workdocs``: [``botocore``] Update workdocs client to latest version
* api-change:``batch``: [``botocore``] Update batch client to latest version
* api-change:``ec2``: [``botocore``] Update ec2 client to latest version
* api-change:``devicefarm``: [``botocore``] Update devicefarm client to latest version
* api-change:``rds``: [``botocore``] Update rds client to latest version
* api-change:``cloudformation``: [``botocore``] Update cloudformation client to latest version
* api-change:``lambda``: [``botocore``] Update lambda client to latest version
* api-change:``config``: [``botocore``] Update config client to latest version
* api-change:``cloudtrail``: [``botocore``] Update cloudtrail client to latest version
* api-change:``mediaconvert``: [``botocore``] Update mediaconvert client to latest version
* api-change:``iot``: [``botocore``] Update iot client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/boto3
  - Changelog: https://pyup.io/changelogs/boto3/
  - Repo: https://github.com/boto/boto3
</details>





### Update [pytest](https://pypi.org/project/pytest) from **4.0.0** to **4.0.1**.


<details>
  <summary>Changelog</summary>
  
  
   ### 4.0.1
   ```
   =========================

Bug Fixes
---------

- `3952 &lt;https://github.com/pytest-dev/pytest/issues/3952&gt;`_: Display warnings before &quot;short test summary info&quot; again, but still later warnings in the end.


- `4386 &lt;https://github.com/pytest-dev/pytest/issues/4386&gt;`_: Handle uninitialized exceptioninfo in repr/str.


- `4393 &lt;https://github.com/pytest-dev/pytest/issues/4393&gt;`_: Do not create ``.gitignore``/``README.md`` files in existing cache directories.


- `4400 &lt;https://github.com/pytest-dev/pytest/issues/4400&gt;`_: Rearrange warning handling for the yield test errors so the opt-out in 4.0.x correctly works.


- `4405 &lt;https://github.com/pytest-dev/pytest/issues/4405&gt;`_: Fix collection of testpaths with ``--pyargs``.


- `4412 &lt;https://github.com/pytest-dev/pytest/issues/4412&gt;`_: Fix assertion rewriting involving ``Starred`` + side-effects.


- `4425 &lt;https://github.com/pytest-dev/pytest/issues/4425&gt;`_: Ensure we resolve the absolute path when the given ``--basetemp`` is a relative path.



Trivial/Internal Changes
------------------------

- `4315 &lt;https://github.com/pytest-dev/pytest/issues/4315&gt;`_: Use ``pkg_resources.parse_version`` instead of ``LooseVersion`` in minversion check.


- `4440 &lt;https://github.com/pytest-dev/pytest/issues/4440&gt;`_: Adjust the stack level of some internal pytest warnings.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>





### Update [whitenoise](https://pypi.org/project/whitenoise) from **4.1.1** to **4.1.2**.


<details>
  <summary>Changelog</summary>
  
  
   ### 4.1.2
   ```
   ------

 * Add correct MIME type for WebAssembly, which is required for files to be
   executed (thanks `mdboom &lt;https://github.com/mdboom&gt;`_ ).
 * Stop accessing the FILE_CHARSET Django setting which was almost entirely
   unused and is now deprecated (thanks `timgraham
   &lt;https://github.com/timgraham&gt;`_).
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/whitenoise
  - Changelog: https://pyup.io/changelogs/whitenoise/
  - Homepage: http://whitenoise.evans.io
</details>





### Update [backoff](https://pypi.org/project/backoff) from **1.6.0** to **1.7.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.7.0
   ```
   Changed

- Support Python 3.7
- Drop support for async in Python 3.4
- Drop support for Python 2.6
- Update development dependencies
- Use poetry for dependencies and packaging
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/backoff
  - Changelog: https://pyup.io/changelogs/backoff/
  - Repo: https://github.com/litl/backoff
</details>







Co-authored-by: pyup-bot <github-bot@pyup.io>

bors bot added a commit to rehandalal/therapist that referenced this issue Nov 27, 2018

Merge #47
47: Update pytest to 4.0.1 r=rehandalal a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **4.0.0** to **4.0.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 4.0.1
   ```
   =========================

Bug Fixes
---------

- `3952 &lt;https://github.com/pytest-dev/pytest/issues/3952&gt;`_: Display warnings before &quot;short test summary info&quot; again, but still later warnings in the end.


- `4386 &lt;https://github.com/pytest-dev/pytest/issues/4386&gt;`_: Handle uninitialized exceptioninfo in repr/str.


- `4393 &lt;https://github.com/pytest-dev/pytest/issues/4393&gt;`_: Do not create ``.gitignore``/``README.md`` files in existing cache directories.


- `4400 &lt;https://github.com/pytest-dev/pytest/issues/4400&gt;`_: Rearrange warning handling for the yield test errors so the opt-out in 4.0.x correctly works.


- `4405 &lt;https://github.com/pytest-dev/pytest/issues/4405&gt;`_: Fix collection of testpaths with ``--pyargs``.


- `4412 &lt;https://github.com/pytest-dev/pytest/issues/4412&gt;`_: Fix assertion rewriting involving ``Starred`` + side-effects.


- `4425 &lt;https://github.com/pytest-dev/pytest/issues/4425&gt;`_: Ensure we resolve the absolute path when the given ``--basetemp`` is a relative path.



Trivial/Internal Changes
------------------------

- `4315 &lt;https://github.com/pytest-dev/pytest/issues/4315&gt;`_: Use ``pkg_resources.parse_version`` instead of ``LooseVersion`` in minversion check.


- `4440 &lt;https://github.com/pytest-dev/pytest/issues/4440&gt;`_: Adjust the stack level of some internal pytest warnings.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment