New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeprecationWarning when running pytest on pyramid application #384

Closed
josherickson opened this Issue Oct 18, 2018 · 14 comments

Comments

Projects
None yet
4 participants
@josherickson
Copy link

josherickson commented Oct 18, 2018

Hello,

These are warnings that I'm receiving when running pytest on my pyramid application. It appears that they are coming from this package, although it's not clear to me after looking through the source code what it is that's causing the warnings.

Warnings from pytest output:

=============================== warnings summary ===============================
/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:373: DeprecationWarning: invalid escape sequence \`
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:526: DeprecationWarning: invalid escape sequence \`
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:577: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1488: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1594: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1703: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1855: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2456: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2560: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2664: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2833: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3034: DeprecationWarning: invalid escape sequence \*
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3469: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3574: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3680: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3854: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5035: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5140: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5245: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/byterange.py:5: DeprecationWarning: invalid escape sequence \d
_rx_range = re.compile('bytes *= *(\d*) *- *(\d*)', flags=re.I)

/usr/local/lib/python3.6/site-packages/webtest/forms.py:408: DeprecationWarning: invalid escape sequence \s
'''<label\s+(?:[^>]*)for=(?:"|')([a-z0-9_\-]+)(?:"|')(?:[^>]*)>''',

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=============== 1 failed, 1 passed, 21 warnings in 6.84 seconds ================

pytest code:

from unittest import TestCase

from pyramid.paster import get_appsettings
from webtest import TestApp as _TestApp

from app import main


class FunctionalTests(TestCase):
    def setUp(self):
        settings = get_appsettings('pytest.ini', name='main')
        app_ = main({}, **settings)
        self.test_app = _TestApp(app_)

    def tearDown(self):
        self.base_tear_down()

    def test_success(self):
        self.assertTrue(True)

    def test_failure(self):
        self.assertTrue(False)

Is this something that I can fix, or is this an issue with the library?

@mmerickel

This comment has been minimized.

Copy link
Member

mmerickel commented Oct 18, 2018

which version of webob?

@josherickson

This comment has been minimized.

Copy link

josherickson commented Oct 18, 2018

which version of webob?

pyramid==1.9.2
pytest==3.91
WebOb==1.8.2
WebTest==2.0.32

By the way, I am not installing and importing WebOb myself. I imagine this occurs through my use of WebTest.

@mmerickel

This comment has been minimized.

Copy link
Member

mmerickel commented Oct 18, 2018

Can you try webob 1.8.3 as it was just released this week and report back. I'm not expecting it to fix things but I want to be sure.

@mmerickel

This comment has been minimized.

Copy link
Member

mmerickel commented Oct 18, 2018

Below is my test case using the same versions you mentioned. I had to come up with one since yours is not actually runnable. Everything passes here without warnings. What OS and version of Python are you running?

from pyramid.config import Configurator
import unittest
from webtest import TestApp as _TestApp

class FunctionalTests(unittest.TestCase):
    def setUp(self):
        with Configurator() as config:
            config.add_view(lambda r: 'foo', renderer='string')
            app = config.make_wsgi_app()
        self.test_app = _TestApp(app)

    def test_success(self):
        response = self.test_app.get('/')
        self.assertEqual(response.text, 'foo')
❯ env/bin/pytest test_webob_384.py
==================================== test session starts ====================================
platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.7.0, pluggy-0.8.0
rootdir: /Users/michael/work/oss/pyramid, inifile:
collected 1 item

test_webob_384.py .                                                                   [100%]

================================= 1 passed in 0.15 seconds ==================================
@josherickson

This comment has been minimized.

Copy link

josherickson commented Oct 18, 2018

Yep, looks to be the same. This is with WebOb==1.8.3.

=============================== warnings summary ===============================
/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:373: DeprecationWarning: invalid escape sequence \`
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:526: DeprecationWarning: invalid escape sequence \`
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:577: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1488: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1594: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1703: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1855: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2456: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2560: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2664: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2833: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3034: DeprecationWarning: invalid escape sequence \*
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3469: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3574: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3680: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3854: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5035: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5140: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5245: DeprecationWarning: invalid escape sequence \ 
"""

/usr/local/lib/python3.6/site-packages/webob/byterange.py:5: DeprecationWarning: invalid escape sequence \d
_rx_range = re.compile('bytes *= *(\d*) *- *(\d*)', flags=re.I)

/usr/local/lib/python3.6/site-packages/webtest/forms.py:408: DeprecationWarning: invalid escape sequence \s
'''<label\s+(?:[^>]*)for=(?:"|')([a-z0-9_\-]+)(?:"|')(?:[^>]*)>''',

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=============== 1 failed, 1 passed, 21 warnings in 6.89 seconds ================
@mmerickel

This comment has been minimized.

Copy link
Member

mmerickel commented Oct 18, 2018

Does my example above repro the issue for you? Can you edit it to repro the issue?

@mmerickel

This comment has been minimized.

Copy link
Member

mmerickel commented Oct 18, 2018

Okay I think we found some invalid backslashes, I'm annoyed that I can't get my box to output the warnings though.

@bertjwregeer

This comment has been minimized.

Copy link
Member

bertjwregeer commented Oct 18, 2018

I've cleaned these up in ce914a1

However neither @mmerickel or I are about to reproduce the output you saw, even while running the WebOb test suite.

Would you be so kind as to paste the whole of your pytest output? Specifically I am looking for the lines that look like:

platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.7.0, pluggy-0.7.1
rootdir: /Users/xistence/Projects/Pylons/webob, inifile: setup.cfg
plugins: xdist-1.23.2, forked-0.2, cov-2.6.0
@bertjwregeer

This comment has been minimized.

Copy link
Member

bertjwregeer commented Oct 18, 2018

I've been able to reproduce the errors you are seeing (though not easily, and the above fix did fix it). Do you have PYTHONDONTWRITEBYTECODE=1 or -B enabled for your Python interpreter?

How did you install Pyramid/WebOb?

@josherickson

This comment has been minimized.

Copy link

josherickson commented Oct 19, 2018

Hey guys, sorry for the delay on this. Here's a full solution that gives the warnings. Copy each bit of code into the filename I've put at the top. Note that the first file is in an "app" directory, and everything else is in the root directory.

Build and run via Docker using the following commands:
docker build -t warnings-test . && docker run -t warnings-test

@bertjwregeer:
I am not setting PYTHONDONTWRITEBYTECODE=1 or -B that I know of. Originally I was not installing WebOb, I believe it was being installed as part of WebTest. However, now I'm installing everything including WebOb via pip in this example, to make sure I'm getting the specified version of WebOb.

# app/__init__.py
from paste.translogger import TransLogger
from pyramid.config import Configurator


def main(_global_config, **settings):
    config = Configurator(settings=settings)
    config.scan()
    app = config.make_wsgi_app()
    return TransLogger(app, setup_console_handler=False)
# app.ini
[server:main]
use = egg:waitress#main
listen = 0.0.0.0:80
url_scheme = http

[app:main]
use = egg:warnings_test

[loggers]
keys = root

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = INFO
handlers = console

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s.%(msecs)03d %(levelname)-8s [%(name)s] %(message)s
datefmt = %H:%M:%S
# Dockerfile
FROM python:3.6-alpine
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY ./ ./
RUN pip3 install -e .
RUN pip3 freeze
CMD ["pytest", "-c", "test.ini", "-s", "--cov-report=term-missing", "--cov=app", "test.py"]
# requirements.txt
paste==2.0.3
pyramid==1.9.2
pytest==3.9.1
pytest-cov==2.6.0
waitress==1.1.0
WebOb==1.8.3
WebTest==2.0.32
# setup.py
from setuptools import setup, find_packages


setup(
    name='warnings_test',
    packages=find_packages(),
    entry_points={
        'paste.app_factory': [
            'main = app:main',
        ],
    },
)
# test.ini
[pytest]
testpaths = .
python_files = *.py
# test.py
from unittest import TestCase

from pyramid.paster import get_appsettings
from webtest import TestApp as _TestApp

from app import main


class FunctionalTests(TestCase):
    def setUp(self):
        settings = get_appsettings('app.ini', name='main')
        app_ = main({}, **settings)
        self.test_app = _TestApp(app_)

    def tearDown(self):
        pass

    def test_success(self):
        self.assertTrue(True)

    def test_failure(self):
        self.assertTrue(False)

The output of this for me includes:

platform linux -- Python 3.6.6, pytest-3.9.1, py-1.7.0, pluggy-0.8.0
rootdir: /, inifile: test.ini
plugins: cov-2.6.0

and

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:373: DeprecationWarning: invalid escape sequence \`
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:526: DeprecationWarning: invalid escape sequence \`
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:577: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1488: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1594: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1703: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:1855: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2456: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2560: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2664: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:2833: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3034: DeprecationWarning: invalid escape sequence \*
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3469: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3574: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3680: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:3854: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5035: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5140: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/acceptparse.py:5245: DeprecationWarning: invalid escape sequence \ 
  """

/usr/local/lib/python3.6/site-packages/webob/byterange.py:5: DeprecationWarning: invalid escape sequence \d
  _rx_range = re.compile('bytes *= *(\d*) *- *(\d*)', flags=re.I)

/usr/local/lib/python3.6/site-packages/webtest/forms.py:408: DeprecationWarning: invalid escape sequence \s
  '''<label\s+(?:[^>]*)for=(?:"|')([a-z0-9_\-]+)(?:"|')(?:[^>]*)>''',

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================================ 1 failed, 1 passed, 21 warnings in 7.39 seconds
@josherickson

This comment has been minimized.

Copy link

josherickson commented Oct 19, 2018

Also, not sure if you guys are on the WebTest team as well, but looks like the last warning there is from WebTest.

/usr/local/lib/python3.6/site-packages/webtest/forms.py:408: DeprecationWarning: invalid escape sequence \s '''<label\s+(?:[^>]*)for=(?:"|')([a-z0-9_\-]+)(?:"|')(?:[^>]*)>'''

@bertjwregeer

This comment has been minimized.

Copy link
Member

bertjwregeer commented Oct 19, 2018

@josherickson yeah, the only time those errors are shown is when Python runs the compile step to turn the file into a *.pyc, and then never again.

This is why both @mmerickel and I weren't able to replicate it, as pip should during the installation phase turn the sdist into a wheel, and then install it, that last step I believe will also compile the .py files.

Which is why I was asking if you were turning bytecode compilation/saving off because after the first run you wouldn't see it again. I've fixed the issues in WebOb, and will get them back ported. I am going to try your steps too and see if I can replicate the issue and understand why you are not getting the .py files compiled and thus seeing the error each time you run your tests.

@kiddick

This comment has been minimized.

Copy link

kiddick commented Oct 31, 2018

same here

@bertjwregeer

This comment has been minimized.

Copy link
Member

bertjwregeer commented Nov 11, 2018

I've back ported this change to the 1.8-branch and will be releasing a new version shortly.

bertjwregeer added a commit that referenced this issue Nov 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment