Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

T36032 User.username should be unique #154

Merged
merged 3 commits into from
Aug 17, 2022
Merged

T36032 User.username should be unique #154

merged 3 commits into from
Aug 17, 2022

Conversation

JenySadadia
Copy link
Collaborator

Create an index on User model to make sure that the username is unique for every user.

gctucker
gctucker reviewed Jun 29, 2022
View reviewed changes
api/db.py Outdated Show resolved Hide resolved
api/db.py Outdated Show resolved Hide resolved
@JenySadadia JenySadadia force-pushed the unique-username branch 4 times, most recently from 1ddb2cc to 316b87d Compare June 30, 2022 12:13
gctucker
gctucker reviewed Jun 30, 2022
View reviewed changes
api/main.py Outdated Show resolved Hide resolved
api/models.py Outdated Show resolved Hide resolved
api/models.py Show resolved Hide resolved
test/test_subscribe_handler.py Outdated
@@ -16,7 +16,7 @@


def test_subscribe_endpoint(mock_get_current_user, mock_init_sub_id,
mock_subscribe):
mock_subscribe, mock_db_create_indexes):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why add it to these various test cases? Maybe we should just have one test for creating indexes and leave the existing tests unchanged. Does it have any side-effect to call mock_db_create_indexes for these test cases?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I do not add this mocked function here, the test case fails with pytest command:

_______________________________________________________________ test_subscribe_endpoint ________________________________________________________________

mock_get_current_user = <AsyncMock id='139717843878384'>, mock_init_sub_id = <AsyncMock id='139717843878000'>
mock_subscribe = <AsyncMock id='139717843880448'>

    def test_subscribe_endpoint(mock_get_current_user, mock_init_sub_id,
                                mock_subscribe):
        """
        Test Case : Test KernelCI API /subscribe endpoint
        Expected Result :
            HTTP Response Code 200 OK
            JSON with 'id' and 'channel' keys
        """
        subscribe = Subscription(id=1, channel='abc')
        mock_subscribe.return_value = subscribe
    
>       with TestClient(app) as client:

test/test_subscribe_handler.py:29: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:105: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError

If I run pytest -v test/test_subscribe_handler.py , it works. I am not sure why!

================================================================= test session starts ==================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /home/jeny/kernelCI/kernelci-api-env/bin/python3
cachedir: .pytest_cache
rootdir: /home/jeny/kernelCI/kernelci-api-env/src/kernelci-api
plugins: anyio-3.6.1, asyncio-0.16.0, mock-3.6.1
collected 1 item                                                                                                                                       

test/test_subscribe_handler.py::test_subscribe_endpoint PASSED                                                                                   [100%]

=================================================================== warnings summary ===================================================================
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
  /home/jeny/kernelCI/kernelci-api-env/lib/python3.8/site-packages/aiofiles/os.py:10: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def run(*args, loop=None, executor=None, **kwargs):

-- Docs: https://docs.pytest.org/en/stable/warnings.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, I didn't realise the tests were starting the actual service and therefore calling the initialisation functions in main.py including the one to create database indexes. So yes I guess that needs to be mocked, but maybe it should be enabled in all the tests by default then.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I guess this will need more investigation, I haven't spent too much time looking at the tests yet so I'm not sure what's causing this issue or the timeout issue seen earlier.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also tried to use the mocked function in conftest.py to have it enabled in all the tests, but it didn't work. I will look into it again.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

async tests from test_pubsub.py were closing the event loop for further test cases. Fixed it by adding event_loop fixture. I would like your suggestion on this @mgalka.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JenySadadia
Tests in test_pubsub.py utilize @pytest.mark.asyncio decorator which adds an event loop fixture to every test case. I'd say that it seems like an expected behavior that the event loop is closed when the test case finishes.

I don't fully understand which tests you have in mind, if they use async code, maybe it's worth to add the @pytest.mark.asyncio there?

Can you maybe provide an example what is failing and what should be the expected result?

Copy link
Collaborator Author

@JenySadadia JenySadadia Aug 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run tests without the changes from the last commit that implements event_loop fixture, the output is:

================================================================= test session starts ==================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/jeny/kernelCI/kernelci-api-env/src/kernelci-api
plugins: anyio-3.6.1, asyncio-0.16.0, mock-3.6.1
collected 29 items                                                                                                                                     

test/test_listen_handler.py ...                                                                                                                  [ 10%]
test/test_me_handler.py .                                                                                                                        [ 13%]
test/test_node_handler.py ..........                                                                                                             [ 48%]
test/test_pubsub.py .....                                                                                                                        [ 65%]
test/test_root_handler.py .                                                                                                                      [ 68%]
test/test_subscribe_handler.py F                                                                                                                 [ 72%]
test/test_token_handler.py ...                                                                                                                   [ 82%]
test/test_unsubscribe_handler.py FF                                                                                                              [ 89%]
test/test_user_handler.py FFF                                                                                                                    [100%]

======================================================================= FAILURES =======================================================================
_______________________________________________________________ test_subscribe_endpoint ________________________________________________________________

mock_get_current_user = <AsyncMock id='140352823824240'>, mock_init_sub_id = <AsyncMock id='140352823823328'>
mock_subscribe = <AsyncMock id='140352823823808'>

    def test_subscribe_endpoint(mock_get_current_user, mock_init_sub_id,
                                mock_subscribe):
        """
        Test Case : Test KernelCI API /subscribe endpoint
        Expected Result :
            HTTP Response Code 200 OK
            JSON with 'id' and 'channel' keys
        """
        subscribe = Subscription(id=1, channel='abc')
        mock_subscribe.return_value = subscribe
    
>       with TestClient(app) as client:

test/test_subscribe_handler.py:29: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:111: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError
______________________________________________________________ test_unsubscribe_endpoint _______________________________________________________________

mock_get_current_user = <AsyncMock id='140352694383664'>, mock_init_sub_id = <AsyncMock id='140352823286032'>
mock_unsubscribe = <AsyncMock id='140352694383712'>

    def test_unsubscribe_endpoint(mock_get_current_user,
                                  mock_init_sub_id, mock_unsubscribe):
        """
        Test Case : Test KernelCI API /unsubscribe endpoint positive path
        Expected Result :
            HTTP Response Code 200 OK
        """
>       with TestClient(app) as client:

test/test_unsubscribe_handler.py:24: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:111: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError
_______________________________________________________ test_unsubscribe_endpoint_empty_response _______________________________________________________

mock_get_current_user = <AsyncMock id='140352691678896'>, mock_init_sub_id = <AsyncMock id='140352691679040'>

    def test_unsubscribe_endpoint_empty_response(mock_get_current_user,
                                                 mock_init_sub_id):
        """
        Test Case : Test KernelCI API /unsubscribe endpoint negative path
        Expected Result :
            HTTP Response Code 404 Not Found
            JSON with 'detail' key
        """
>       with TestClient(app) as client:

test/test_unsubscribe_handler.py:42: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:111: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError
_______________________________________________________________ test_create_regular_user _______________________________________________________________

mock_init_sub_id = <AsyncMock id='140352692245648'>, mock_get_current_admin_user = <AsyncMock id='140352692245456'>
mock_db_create = <AsyncMock id='140352692245360'>, mock_publish_cloudevent = <AsyncMock id='140352692245024'>

    def test_create_regular_user(mock_init_sub_id, mock_get_current_admin_user,
                                 mock_db_create, mock_publish_cloudevent):
        """
        Test Case : Test KernelCI API /user endpoint to create regular user
        when requested with admin user's bearer token
        Expected Result :
            HTTP Response Code 200 OK
            JSON with '_id', 'username', 'hashed_password'
            'active', and 'is_admin' keys
        """
        user = User(username='test', hashed_password="$2b$12$Whi.dpTC.\
    HR5UHMdMFQeOe1eD4oXaP08oW7ogYqyiNziZYNdUHs8i", active=True)
        mock_db_create.return_value = user
    
>       with TestClient(app) as client:

test/test_user_handler.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:111: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError
________________________________________________________________ test_create_admin_user ________________________________________________________________

mock_init_sub_id = <AsyncMock id='140352692021616'>, mock_get_current_admin_user = <AsyncMock id='140352823953488'>
mock_db_create = <AsyncMock id='140352692021904'>, mock_publish_cloudevent = <AsyncMock id='140352833182304'>

    def test_create_admin_user(mock_init_sub_id, mock_get_current_admin_user,
                               mock_db_create, mock_publish_cloudevent):
        """
        Test Case : Test KernelCI API /user endpoint to create admin user
        when requested with admin user's bearer token
        Expected Result :
            HTTP Response Code 200 OK
            JSON with '_id', 'username', 'hashed_password'
            'active', and 'is_admin' keys
        """
        user = User(username='test_admin', hashed_password="$2b$12$Whi.dpTC.\
    HR5UHMdMFQeOe1eD4oXaP08oW7ogYqyiNziZYNdUHs8i", active=True, is_admin=True)
        mock_db_create.return_value = user
    
>       with TestClient(app) as client:

test/test_user_handler.py:63: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:111: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError
__________________________________________________________ test_create_user_endpoint_negative __________________________________________________________

mock_init_sub_id = <AsyncMock id='140352692056560'>, mock_get_current_user = <AsyncMock id='140352692058144'>
mock_publish_cloudevent = <AsyncMock id='140352692057472'>

    def test_create_user_endpoint_negative(mock_init_sub_id, mock_get_current_user,
                                           mock_publish_cloudevent):
        """
        Test Case : Test KernelCI API /user endpoint when requested
        with regular user's bearer token
        Expected Result :
            HTTP Response Code 401 Unauthorized
            JSON with 'detail' key denoting 'Access denied' error
        """
        mock_get_current_user.return_value = None, "Access denied"
    
>       with TestClient(app) as client:

test/test_user_handler.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../lib/python3.8/site-packages/starlette/testclient.py:459: in __enter__
    loop.run_until_complete(self.wait_startup())
/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../lib/python3.8/site-packages/starlette/testclient.py:485: in wait_startup
    self.task.result()
../../lib/python3.8/site-packages/starlette/testclient.py:469: in lifespan
    await self.app(scope, self.receive_queue.get, self.send_queue.put)
../../lib/python3.8/site-packages/fastapi/applications.py:208: in __call__
    await super().__call__(scope, receive, send)
../../lib/python3.8/site-packages/starlette/applications.py:112: in __call__
    await self.middleware_stack(scope, receive, send)
../../lib/python3.8/site-packages/starlette/middleware/errors.py:146: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/exceptions.py:58: in __call__
    await self.app(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:569: in __call__
    await self.lifespan(scope, receive, send)
../../lib/python3.8/site-packages/starlette/routing.py:540: in lifespan
    async for item in self.lifespan_context(app):
../../lib/python3.8/site-packages/starlette/routing.py:481: in default_lifespan
    await self.startup()
../../lib/python3.8/site-packages/starlette/routing.py:516: in startup
    await handler()
api/main.py:44: in create_indexes
    await db.create_indexes()
api/db.py:39: in create_indexes
    model.create_indexes(col)
api/models.py:111: in create_indexes
    collection.create_index("username", unique=True)
../../lib/python3.8/site-packages/motor/metaprogramming.py:73: in method
    return framework.run_on_executor(loop,
../../lib/python3.8/site-packages/motor/frameworks/asyncio/__init__.py:73: in run_on_executor
    return loop.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py:774: in run_in_executor
    self._check_closed()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>

    def _check_closed(self):
        if self._closed:
>           raise RuntimeError('Event loop is closed')
E           RuntimeError: Event loop is closed

/usr/lib/python3.8/asyncio/base_events.py:508: RuntimeError
=================================================================== warnings summary ===================================================================
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
../../lib/python3.8/site-packages/aiofiles/os.py:10
  /home/jeny/kernelCI/kernelci-api-env/lib/python3.8/site-packages/aiofiles/os.py:10: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def run(*args, loop=None, executor=None, **kwargs):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============================================================== short test summary info ================================================================
FAILED test/test_subscribe_handler.py::test_subscribe_endpoint - RuntimeError: Event loop is closed
FAILED test/test_unsubscribe_handler.py::test_unsubscribe_endpoint - RuntimeError: Event loop is closed
FAILED test/test_unsubscribe_handler.py::test_unsubscribe_endpoint_empty_response - RuntimeError: Event loop is closed
FAILED test/test_user_handler.py::test_create_regular_user - RuntimeError: Event loop is closed
FAILED test/test_user_handler.py::test_create_admin_user - RuntimeError: Event loop is closed
FAILED test/test_user_handler.py::test_create_user_endpoint_negative - RuntimeError: Event loop is closed
======================================================= 6 failed, 23 passed, 6 warnings in 1.09s =======================================================

If we exclude test_pubsub.py from running, the rest of the tests are passing.
So the issue is event loop is being closed by async tests from pubsub.

Copy link
Collaborator Author

@JenySadadia JenySadadia Aug 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mgalka The issue is reported here: pytest-dev/pytest-asyncio#371
There are also similar kinds of issues: https://github.com/pytest-dev/pytest-asyncio/issues

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, It looks then that in this case we can leave a fixture that inserts an event loop to the test case as a work around. Maybe it's worth to mention why we use this fixture in the docstring and even paste the link to the issue.

@JenySadadia JenySadadia force-pushed the unique-username branch 3 times, most recently from 0103d14 to cd96e0d Compare July 1, 2022 09:47
@JenySadadia JenySadadia requested a review from mgalka July 1, 2022 09:47
gctucker
gctucker reviewed Aug 3, 2022
View reviewed changes
api/models.py Outdated Show resolved Hide resolved
Jeny Sadadia added 2 commits August 15, 2022 14:56
api: create database indexes on startup
0ecb0d5 
Implement method Database.create_indexes to create
indexes on model collections. Implement
User.create_index to bind unique constraint
to username field.

Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>
api.main: catch 'DuplicateKeyError' exception
1d23e72 
Need to catch an exception and return proper
error message when '/user' endpoint receives
a username that has already been taken.

Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>
mgalka
mgalka reviewed Aug 15, 2022
View reviewed changes
test/conftest.py Outdated
@@ -35,6 +36,15 @@ def client():
return TestClient(app)


@pytest.fixture
def event_loop():
"""Create an instance of the default event loop for each test case"""
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a description why we need to insert an event loop to each test case and paste the bug URL.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the docstring.

test: add 'event_loop' fixture
482ef5e 
Default event loop is being closed after running
`async` tests from test_pubsub.py. This fails
subscribe and unsubscribe handler tests.
Need to create an instance for default event
loop for all the async tests.

Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>
mgalka
mgalka approved these changes Aug 17, 2022
View reviewed changes
Copy link

@mgalka mgalka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mgalka mgalka merged commit 732d6b9 into kernelci:main Aug 17, 2022
@JenySadadia JenySadadia deleted the unique-username branch August 17, 2022 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gctucker gctucker gctucker left review comments

@mgalka mgalka mgalka approved these changes

Assignees
No one assigned
Labels
None yet
Projects
KernelCI API & Pipeline
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@JenySadadia @mgalka @gctucker