Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some tests fail if all tests are run together #284

Closed
yan12125 opened this issue Oct 16, 2020 · 7 comments · Fixed by #286
Closed

Some tests fail if all tests are run together #284

yan12125 opened this issue Oct 16, 2020 · 7 comments · Fixed by #286
Assignees
@lepture
Labels
bug

Comments

@yan12125
Copy link
Contributor

Describe the bug

Some tests in tests/core/test_oauth2/test_rfc8414.py and tests/core/test_oidc/test_discovery.py fail if they are run with Django or Flask tests.

Error Stacks

pytest logs 
============================= test session starts ==============================
platform linux -- Python 3.8.6, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
django: settings: tests.django.settings (from ini)
rootdir: /build/python-authlib/src/authlib, configfile: setup.cfg
plugins: asyncio-0.14.0, django-3.10.0
collected 503 items

tests/django/test_client/test_oauth_client.py .............              [  2%]
tests/django/test_oauth1/test_authorize.py ....                          [  3%]
tests/django/test_oauth1/test_resource_protector.py ....                 [  4%]
tests/django/test_oauth1/test_token_credentials.py .....                 [  5%]
tests/django/test_oauth2/test_authorization_code_grant.py .......        [  6%]
tests/django/test_oauth2/test_client_credentials_grant.py .....          [  7%]
tests/django/test_oauth2/test_implicit_grant.py ...                      [  8%]
tests/django/test_oauth2/test_password_grant.py .....                    [  9%]
tests/django/test_oauth2/test_refresh_token.py ......                    [ 10%]
tests/django/test_oauth2/test_resource_protector.py .....                [ 11%]
tests/django/test_oauth2/test_revocation_endpoint.py ....                [ 12%]
tests/core/test_jose/test_jwe.py ...............                         [ 15%]
tests/core/test_jose/test_jwk.py ...................                     [ 18%]
tests/core/test_jose/test_jws.py ..............                          [ 21%]
tests/core/test_jose/test_jwt.py ...............                         [ 24%]
tests/core/test_oauth2/test_rfc6749_misc.py .....                        [ 25%]
tests/core/test_oauth2/test_rfc7591.py ......                            [ 26%]
tests/core/test_oauth2/test_rfc7662.py ......                            [ 28%]
tests/core/test_oauth2/test_rfc8414.py ....F..F..FF..F..F....F...        [ 33%]
tests/core/test_oidc/test_core.py ..........                             [ 35%]
tests/core/test_oidc/test_discovery.py ............F..........           [ 39%]
tests/core/test_requests_client/test_assertion_session.py ..             [ 40%]
tests/core/test_requests_client/test_oauth1_session.py ................  [ 43%]
tests/core/test_requests_client/test_oauth2_session.py ................. [ 46%]
...........                                                              [ 48%]
tests/flask/test_client/test_oauth_client.py ...................         [ 52%]
tests/flask/test_client/test_user_mixin.py ......                        [ 53%]
tests/flask/test_oauth1/test_authorize.py ......                         [ 55%]
tests/flask/test_oauth1/test_resource_protector.py ........              [ 56%]
tests/flask/test_oauth1/test_temporary_credentials.py ................   [ 59%]
tests/flask/test_oauth1/test_token_credentials.py ......                 [ 61%]
tests/flask/test_oauth2/test_authorization_code_grant.py ............    [ 63%]
tests/flask/test_oauth2/test_client_credentials_grant.py .....           [ 64%]
tests/flask/test_oauth2/test_client_registration_endpoint.py .........   [ 66%]
tests/flask/test_oauth2/test_code_challenge.py ............              [ 68%]
tests/flask/test_oauth2/test_device_code_grant.py .........              [ 70%]
tests/flask/test_oauth2/test_implicit_grant.py ......                    [ 71%]
tests/flask/test_oauth2/test_introspection_endpoint.py ....              [ 72%]
tests/flask/test_oauth2/test_jwt_bearer_client_auth.py .......           [ 73%]
tests/flask/test_oauth2/test_jwt_bearer_grant.py .....                   [ 74%]
tests/flask/test_oauth2/test_oauth2_server.py .......                    [ 76%]
tests/flask/test_oauth2/test_openid_code_grant.py ........               [ 77%]
tests/flask/test_oauth2/test_openid_hybrid_grant.py ..........           [ 79%]
tests/flask/test_oauth2/test_openid_implict_grant.py ........            [ 81%]
tests/flask/test_oauth2/test_password_grant.py .......                   [ 82%]
tests/flask/test_oauth2/test_refresh_token.py ..........                 [ 84%]
tests/flask/test_oauth2/test_revocation_endpoint.py ....                 [ 85%]
tests/py3/test_httpx_client/test_assertion_client.py ..                  [ 85%]
tests/py3/test_httpx_client/test_async_assertion_client.py ..            [ 86%]
tests/py3/test_httpx_client/test_async_oauth1_client.py .......          [ 87%]
tests/py3/test_httpx_client/test_async_oauth2_client.py ................ [ 90%]
....                                                                     [ 91%]
tests/py3/test_httpx_client/test_oauth1_client.py .......                [ 93%]
tests/py3/test_httpx_client/test_oauth2_client.py ...................    [ 96%]
tests/py3/test_starlette_client/test_oauth_client.py ...........         [ 99%]
tests/py3/test_starlette_client/test_user_mixin.py .....                 [100%]

=================================== FAILURES ===================================
_____ AuthorizationServerMetadataTest.test_validate_authorization_endpoint _____

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_authorization_endpoint>

    def test_validate_authorization_endpoint(self):
        # https
        metadata = AuthorizationServerMetadata({
            'authorization_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_authorization_endpoint()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:93: AttributeError
_____ AuthorizationServerMetadataTest.test_validate_introspection_endpoint _____

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_introspection_endpoint>

    def test_validate_introspection_endpoint(self):
        metadata = AuthorizationServerMetadata()
        metadata.validate_introspection_endpoint()
    
        # https
        metadata = AuthorizationServerMetadata({
            'introspection_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_introspection_endpoint()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:430: AttributeError
_____________ AuthorizationServerMetadataTest.test_validate_issuer _____________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_issuer>

    def test_validate_issuer(self):
        #: missing
        metadata = AuthorizationServerMetadata({})
        with self.assertRaises(ValueError) as cm:
            metadata.validate()
            self.assertEqual('"issuer" is required', str(cm.exception))
    
        #: https
        metadata = AuthorizationServerMetadata({
            'issuer': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_issuer()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:63: AttributeError
____________ AuthorizationServerMetadataTest.test_validate_jwks_uri ____________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_jwks_uri>

    def test_validate_jwks_uri(self):
        # can missing
        metadata = AuthorizationServerMetadata()
        metadata.validate_jwks_uri()
    
        metadata = AuthorizationServerMetadata({
            'jwks_uri': 'http://authlib.org/jwks.json'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_jwks_uri()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:150: AttributeError
_____ AuthorizationServerMetadataTest.test_validate_registration_endpoint ______

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_registration_endpoint>

    def test_validate_registration_endpoint(self):
        metadata = AuthorizationServerMetadata()
        metadata.validate_registration_endpoint()
    
        metadata = AuthorizationServerMetadata({
            'registration_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_registration_endpoint()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:166: AttributeError
______ AuthorizationServerMetadataTest.test_validate_revocation_endpoint _______

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_revocation_endpoint>

    def test_validate_revocation_endpoint(self):
        metadata = AuthorizationServerMetadata()
        metadata.validate_revocation_endpoint()
    
        # https
        metadata = AuthorizationServerMetadata({
            'revocation_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_revocation_endpoint()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:368: AttributeError
_________ AuthorizationServerMetadataTest.test_validate_token_endpoint _________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_token_endpoint>

    def test_validate_token_endpoint(self):
        # implicit
        metadata = AuthorizationServerMetadata({
            'grant_types_supported': ['implicit']
        })
        metadata.validate_token_endpoint()
    
        # missing
        metadata = AuthorizationServerMetadata()
        with self.assertRaises(ValueError) as cm:
            metadata.validate_token_endpoint()
            self.assertIn('required', str(cm.exception))
    
        # https
        metadata = AuthorizationServerMetadata({
            'token_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_token_endpoint()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oauth2/test_rfc8414.py:132: AttributeError
______________ OpenIDProviderMetadataTest.test_validate_jwks_uri _______________

self = <test_oidc.test_discovery.OpenIDProviderMetadataTest testMethod=test_validate_jwks_uri>

    def test_validate_jwks_uri(self):
        # required
        metadata = OpenIDProviderMetadata()
        with self.assertRaises(ValueError) as cm:
            metadata.validate_jwks_uri()
            self.assertEqual('"jwks_uri" is required', str(cm.exception))
    
        metadata = OpenIDProviderMetadata({
            'jwks_uri': 'http://authlib.org/jwks.json'
        })
        with self.assertRaises(ValueError) as cm:
            metadata.validate_jwks_uri()
>           self.assertIn('https', str(cm.exception))
E           AttributeError: '_AssertRaisesContext' object has no attribute 'exception'

tests/core/test_oidc/test_discovery.py:48: AttributeError
=============================== warnings summary ===============================
../../../../usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1230
  /usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1230: PytestConfigWarning: Unknown config option: python_paths
  
    self._warn_or_fail_if_strict("Unknown config option: {}\n".format(key))

tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_nonce_replay
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_prompt
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_pure_code_flow
tests/flask/test_oauth2/test_openid_code_grant.py::RSAOpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::JWKSOpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::ECOpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::PEMOpenIDCodeTest::test_authorize_token
  /build/python-authlib/src/authlib/authlib/integrations/flask_oauth2/authorization_server.py:73: AuthlibDeprecationWarning: Define "get_jwt_config" in OpenID Connect grants
  It will be compatible before version 1.0.
    deprecate('Define "get_jwt_config" in OpenID Connect grants', '1.0')

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_authorization_endpoint
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_introspection_endpoint
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_issuer
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_jwks_uri
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_registration_endpoint
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_revocation_endpoint
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_token_endpoint
FAILED tests/core/test_oidc/test_discovery.py::OpenIDProviderMetadataTest::test_validate_jwks_uri
================== 8 failed, 495 passed, 9 warnings in 6.37s ===================

To Reproduce

  1. Clone 0.15.1
  2. python -m pytest

Expected behavior

All tests pass

Environment:

  • OS: Arch Linux
  • Python Version: 3.8.6
  • Authlib Version: 0.15.1

Additional context

I'm interested in updating the python-authlib package for Arch Llinux for compatibility with newer httpx. That package runs all tests together [1], which worked for authlib 0.14.3.

Do you suggest that packagers for Linux distributions should test different parts separately like the official GitHub workflows [2] do?

[1] https://github.com/archlinux/svntogit-community/blob/2b480fcd73e2ef5fa6ad6d42bd2fdda8fa9a5606/trunk/PKGBUILD#L42
[2]

authlib/.github/workflows/python.yml

Lines 42 to 44 in 84ba75a

env:
TOXENV: py,flask,django,starlette
run: tox

@yan12125 yan12125 added the bug label Oct 16, 2020
@lepture
Copy link
Owner

lepture commented Oct 16, 2020 •
edited

I think #279 has fixed the problem. Can you test with master code.

@lepture
Copy link
Owner

lepture commented Oct 16, 2020

@yan12125 if you can confirm that fixed the problem, I'll cherry pick the PR into maintain-0.15 code, then release a 0.15.2 version.

@yan12125
Copy link
Contributor Author

Thanks for the pointer. On master (84ba75a), tests fail with different errors:

$ DJANGO_SETTINGS_MODULE=tests.django.settings python -m pytest
============================================================================ test session starts ============================================================================
platform linux -- Python 3.8.6, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
django: settings: tests.django.settings (from env)
rootdir: /home/yen/tmp/authlib, configfile: setup.cfg
plugins: asyncio-0.14.0, django-3.10.0
collected 504 items                                                                                                                                                         

tests/django/test_client/test_oauth_client.py .............                                                                                                           [  2%]
tests/django/test_oauth1/test_authorize.py ....                                                                                                                       [  3%]
tests/django/test_oauth1/test_resource_protector.py ....                                                                                                              [  4%]
tests/django/test_oauth1/test_token_credentials.py .....                                                                                                              [  5%]
tests/django/test_oauth2/test_authorization_code_grant.py .......                                                                                                     [  6%]
tests/django/test_oauth2/test_client_credentials_grant.py .....                                                                                                       [  7%]
tests/django/test_oauth2/test_implicit_grant.py ...                                                                                                                   [  8%]
tests/django/test_oauth2/test_password_grant.py .....                                                                                                                 [  9%]
tests/django/test_oauth2/test_refresh_token.py ......                                                                                                                 [ 10%]
tests/django/test_oauth2/test_resource_protector.py .....                                                                                                             [ 11%]
tests/django/test_oauth2/test_revocation_endpoint.py ....                                                                                                             [ 12%]
tests/core/test_jose/test_jwe.py ...............                                                                                                                      [ 15%]
tests/core/test_jose/test_jwk.py ...................                                                                                                                  [ 18%]
tests/core/test_jose/test_jws.py ................                                                                                                                     [ 22%]
tests/core/test_jose/test_jwt.py ..............                                                                                                                       [ 24%]
tests/core/test_oauth2/test_rfc6749_misc.py .....                                                                                                                     [ 25%]
tests/core/test_oauth2/test_rfc7591.py ......                                                                                                                         [ 26%]
tests/core/test_oauth2/test_rfc7662.py ......                                                                                                                         [ 28%]
tests/core/test_oauth2/test_rfc8414.py ....F..F..FF..F..F....F...                                                                                                     [ 33%]
tests/core/test_oidc/test_core.py ..........                                                                                                                          [ 35%]
tests/core/test_oidc/test_discovery.py ............F..........                                                                                                        [ 39%]
tests/core/test_requests_client/test_assertion_session.py ..                                                                                                          [ 40%]
tests/core/test_requests_client/test_oauth1_session.py ................                                                                                               [ 43%]
tests/core/test_requests_client/test_oauth2_session.py ............................                                                                                   [ 49%]
tests/flask/test_client/test_oauth_client.py ...................                                                                                                      [ 52%]
tests/flask/test_client/test_user_mixin.py ......                                                                                                                     [ 53%]
tests/flask/test_oauth1/test_authorize.py ......                                                                                                                      [ 55%]
tests/flask/test_oauth1/test_resource_protector.py ........                                                                                                           [ 56%]
tests/flask/test_oauth1/test_temporary_credentials.py ................                                                                                                [ 59%]
tests/flask/test_oauth1/test_token_credentials.py ......                                                                                                              [ 61%]
tests/flask/test_oauth2/test_authorization_code_grant.py ............                                                                                                 [ 63%]
tests/flask/test_oauth2/test_client_credentials_grant.py .....                                                                                                        [ 64%]
tests/flask/test_oauth2/test_client_registration_endpoint.py .........                                                                                                [ 66%]
tests/flask/test_oauth2/test_code_challenge.py ............                                                                                                           [ 68%]
tests/flask/test_oauth2/test_device_code_grant.py .........                                                                                                           [ 70%]
tests/flask/test_oauth2/test_implicit_grant.py ......                                                                                                                 [ 71%]
tests/flask/test_oauth2/test_introspection_endpoint.py ....                                                                                                           [ 72%]
tests/flask/test_oauth2/test_jwt_bearer_client_auth.py .......                                                                                                        [ 73%]
tests/flask/test_oauth2/test_jwt_bearer_grant.py .....                                                                                                                [ 74%]
tests/flask/test_oauth2/test_oauth2_server.py .......                                                                                                                 [ 76%]
tests/flask/test_oauth2/test_openid_code_grant.py ........                                                                                                            [ 77%]
tests/flask/test_oauth2/test_openid_hybrid_grant.py ..........                                                                                                        [ 79%]
tests/flask/test_oauth2/test_openid_implict_grant.py ........                                                                                                         [ 81%]
tests/flask/test_oauth2/test_password_grant.py .......                                                                                                                [ 82%]
tests/flask/test_oauth2/test_refresh_token.py ..........                                                                                                              [ 84%]
tests/flask/test_oauth2/test_revocation_endpoint.py ....                                                                                                              [ 85%]
tests/starlette/test_client/test_oauth_client.py ...........                                                                                                          [ 87%]
tests/starlette/test_client/test_user_mixin.py .....                                                                                                                  [ 88%]
tests/starlette/test_httpx_client/test_assertion_client.py ..                                                                                                         [ 89%]
tests/starlette/test_httpx_client/test_async_assertion_client.py ..                                                                                                   [ 89%]
tests/starlette/test_httpx_client/test_async_oauth1_client.py .......                                                                                                 [ 90%]
tests/starlette/test_httpx_client/test_async_oauth2_client.py ....................                                                                                    [ 94%]
tests/starlette/test_httpx_client/test_oauth1_client.py .......                                                                                                       [ 96%]
tests/starlette/test_httpx_client/test_oauth2_client.py ...................                                                                                           [100%]

================================================================================= FAILURES ==================================================================================
___________________________________________________ AuthorizationServerMetadataTest.test_validate_authorization_endpoint ____________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_authorization_endpoint>

    def test_validate_authorization_endpoint(self):
        # https
        metadata = AuthorizationServerMetadata({
            'authorization_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_authorization_endpoint()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:92: AssertionError
___________________________________________________ AuthorizationServerMetadataTest.test_validate_introspection_endpoint ____________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_introspection_endpoint>

    def test_validate_introspection_endpoint(self):
        metadata = AuthorizationServerMetadata()
        metadata.validate_introspection_endpoint()
    
        # https
        metadata = AuthorizationServerMetadata({
            'introspection_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_introspection_endpoint()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:429: AssertionError
___________________________________________________________ AuthorizationServerMetadataTest.test_validate_issuer ____________________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_issuer>

    def test_validate_issuer(self):
        #: missing
        metadata = AuthorizationServerMetadata({})
        with self.assertRaises(ValueError) as cm:
            metadata.validate()
        self.assertEqual('"issuer" is required', str(cm.exception))
    
        #: https
        metadata = AuthorizationServerMetadata({
            'issuer': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_issuer()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:62: AssertionError
__________________________________________________________ AuthorizationServerMetadataTest.test_validate_jwks_uri ___________________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_jwks_uri>

    def test_validate_jwks_uri(self):
        # can missing
        metadata = AuthorizationServerMetadata()
        metadata.validate_jwks_uri()
    
        metadata = AuthorizationServerMetadata({
            'jwks_uri': 'http://authlib.org/jwks.json'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_jwks_uri()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:149: AssertionError
____________________________________________________ AuthorizationServerMetadataTest.test_validate_registration_endpoint ____________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_registration_endpoint>

    def test_validate_registration_endpoint(self):
        metadata = AuthorizationServerMetadata()
        metadata.validate_registration_endpoint()
    
        metadata = AuthorizationServerMetadata({
            'registration_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_registration_endpoint()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:165: AssertionError
_____________________________________________________ AuthorizationServerMetadataTest.test_validate_revocation_endpoint _____________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_revocation_endpoint>

    def test_validate_revocation_endpoint(self):
        metadata = AuthorizationServerMetadata()
        metadata.validate_revocation_endpoint()
    
        # https
        metadata = AuthorizationServerMetadata({
            'revocation_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_revocation_endpoint()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:367: AssertionError
_______________________________________________________ AuthorizationServerMetadataTest.test_validate_token_endpoint ________________________________________________________

self = <test_oauth2.test_rfc8414.AuthorizationServerMetadataTest testMethod=test_validate_token_endpoint>

    def test_validate_token_endpoint(self):
        # implicit
        metadata = AuthorizationServerMetadata({
            'grant_types_supported': ['implicit']
        })
        metadata.validate_token_endpoint()
    
        # missing
        metadata = AuthorizationServerMetadata()
        with self.assertRaises(ValueError) as cm:
            metadata.validate_token_endpoint()
        self.assertIn('required', str(cm.exception))
    
        # https
        metadata = AuthorizationServerMetadata({
            'token_endpoint': 'http://authlib.org/'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_token_endpoint()
E           AssertionError: ValueError not raised

tests/core/test_oauth2/test_rfc8414.py:131: AssertionError
_____________________________________________________________ OpenIDProviderMetadataTest.test_validate_jwks_uri _____________________________________________________________

self = <test_oidc.test_discovery.OpenIDProviderMetadataTest testMethod=test_validate_jwks_uri>

    def test_validate_jwks_uri(self):
        # required
        metadata = OpenIDProviderMetadata()
        with self.assertRaises(ValueError) as cm:
            metadata.validate_jwks_uri()
        self.assertEqual('"jwks_uri" is required', str(cm.exception))
    
        metadata = OpenIDProviderMetadata({
            'jwks_uri': 'http://authlib.org/jwks.json'
        })
        with self.assertRaises(ValueError) as cm:
>           metadata.validate_jwks_uri()
E           AssertionError: ValueError not raised

tests/core/test_oidc/test_discovery.py:47: AssertionError
============================================================================= warnings summary ==============================================================================
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_nonce_replay
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_prompt
tests/flask/test_oauth2/test_openid_code_grant.py::OpenIDCodeTest::test_pure_code_flow
tests/flask/test_oauth2/test_openid_code_grant.py::RSAOpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::JWKSOpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::ECOpenIDCodeTest::test_authorize_token
tests/flask/test_oauth2/test_openid_code_grant.py::PEMOpenIDCodeTest::test_authorize_token
  /home/yen/tmp/authlib/authlib/integrations/flask_oauth2/authorization_server.py:73: AuthlibDeprecationWarning: Define "get_jwt_config" in OpenID Connect grants
  It will be compatible before version 1.0.
    deprecate('Define "get_jwt_config" in OpenID Connect grants', '1.0')

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================== short test summary info ==========================================================================
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_authorization_endpoint - AssertionError: ValueError not raised
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_introspection_endpoint - AssertionError: ValueError not raised
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_issuer - AssertionError: ValueError not raised
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_jwks_uri - AssertionError: ValueError not raised
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_registration_endpoint - AssertionError: ValueError not raised
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_revocation_endpoint - AssertionError: ValueError not raised
FAILED tests/core/test_oauth2/test_rfc8414.py::AuthorizationServerMetadataTest::test_validate_token_endpoint - AssertionError: ValueError not raised
FAILED tests/core/test_oidc/test_discovery.py::OpenIDProviderMetadataTest::test_validate_jwks_uri - AssertionError: ValueError not raised
================================================================= 8 failed, 496 passed, 8 warnings in 5.10s =================================================================

@lepture
Copy link
Owner

lepture commented Oct 18, 2020

  1. those error happens because you have set AUTHLIB_INSECURE_TRANSPORT environment variable.
  2. because test_rfc8414 request not setting AUTHLIB_INSECURE_TRANSPORT
  3. while tests in flask and django oauth provider requires AUTHLIB_INSECURE_TRANSPORT

So, it is impossible to test them together.

@lepture lepture closed this as completed Oct 18, 2020
archlinux-github pushed a commit to archlinux/svntogit-community that referenced this issue Oct 18, 2020
upgpkg: python-authlib 0.15.2-1; update for compatibility with the la…
cdbcb1c 
…test httpx; add workaround for an upstream issue [1] in check()

[1] lepture/authlib#284

git-svn-id: file:///srv/repos/svn-community/svn@726103 9fca08f4-af9d-4005-b8df-a31f2cc04f65
archlinux-github pushed a commit to archlinux/svntogit-community that referenced this issue Oct 18, 2020
upgpkg: python-authlib 0.15.2-1; update for compatibility with the la…
3e3ccf5 
…test httpx; add workaround for an upstream issue [1] in check()

[1] lepture/authlib#284


git-svn-id: file:///srv/repos/svn-community/svn@726103 9fca08f4-af9d-4005-b8df-a31f2cc04f65
@yan12125
Copy link
Contributor Author

Thanks for the explanation! I seperate those tests in the latest Arch Linux package.

@eli-schwartz
Copy link

I don't see any reason why this should be a problem... just use https://docs.pytest.org/en/latest/monkeypatch.html to set the env var only in the tests that need it.

@yan12125 yan12125 mentioned this issue Oct 20, 2020
Merged
8 tasks
@yan12125
Copy link
Contributor Author

just use https://docs.pytest.org/en/latest/monkeypatch.html to set the env var

Thanks for the hint. I tried a different approach at #286 as Django and Flask tests appear to use unittest infrastructure instead of the pytest one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lepture lepture

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow tests to be run together in a pytest session
3 participants
@lepture @yan12125 @eli-schwartz