PoC: `fastapi-users` #363

JenySadadia · 2023-09-25T10:57:56Z

This is a PoC of fastapi-users package for user management tools in API.
This is intended for investigation and review only. Improvements are needed for the actual integration.

Add `fastapi-users` package with `beanie` and `oauth` tools for user management. The specific `10.4.0` version has been selected to make it compatible with `fastapi 0.68.1` package version. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

Add `TestUser` model and extend `User` model provided by `fastapi-users`. Add an extra field `username` and create `unique` index on it. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

Also added a method to get database instance `Database._db` to use it for initializing Beanie on API startup. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

`fastapi-users` uses `Beanie` ODM to provide tools to work with MongoDB. We need to initialize Beanie with database and user model on the application startup. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

JenySadadia · 2023-09-25T11:45:09Z

Below are the example commands for requesting routes provided by the package:

Register a user

$ curl -X 'POST' \
  'http://localhost:8001/latest/auth/register' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "email": "user@example.com",
  "password": "test",
  "username": "test"
}'
{"id":"65116ac07e1037b63f946c08", "email": "user@example.com", "password": "test", "is_active": true,  "is_superuser": false, "is_verified": false, "username": "test"}

Verify email
Request token for verification:

curl -X 'POST' \
  'http://localhost:8001/latest/auth/request-verify-token' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "email": "user@example.com"
}'

Reponse: null

API logs:

kernelci-api | Verification requested for user 650bead8a4de36b741e3fbcf. Verification token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJlbWFpbCI6Implbnkuc2FkYWRpYUBnbWFpbC5jb20iLCJhdWQiOiJmYXN0YXBpLXVzZXJzOnZlcmlmeSIsImV4cCI6MTY5NTM4MDEwMn0.0byhPu0WTvdpli1IuEcCYY_b40-VjqqI-7TtOgnbwHo

Provide the verification token to /verify endpoint.

curl -X 'POST' \
  'http://localhost:8001/latest/auth/verify' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJlbWFpbCI6Implbnkuc2FkYWRpYUBnbWFpbC5jb20iLCJhdWQiOiJmYXN0YXBpLXVzZXJzOnZlcmlmeSIsImV4cCI6MTY5NTM4MDEwMn0.0byhPu0WTvdpli1IuEcCYY_b40-VjqqI-7TtOgnbwHo"
}'
{"id": null, "email": "user@example.com", "is_active": true, "is_superuser": false, "is_verified": true, "username": "test"}

Note: need to investigate why id is null in the response.
We can add logic to get the token in email using on_after_request_verify handler from UserManager.

Login

curl -X 'POST'   'http://localhost:8001/latest/auth/jwt/login'   -H 'accept: application/json'   -H 'Content-Type: application/x-www-form-urlencoded'   -d 'grant_type=&username=user@example.com&password=test&scope=&client_id=&client_secret='
{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl19.a1D8r_z3L0MyIp83n88cHMtIv3DqxCDcH7wvSSwDeSo",
"token_type":"bearer"}

Note: It supports tokens with OAuth scopes.

Logout

$ curl -X 'POST' \
  'http://localhost:8001/latest/auth/jwt/logout' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWU0MzBlZjA5M2U2YmVlZDg3OGUiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl0sImV4cCI6MTY5NTI5MjMwN30.iYYMxxHf5F_qrMUbNkfom5lMevE5lPgrNf2NO_U7wVw'
Reponse: null

Note: JWT logout is not implemented as the authentication backend was configured with token expiry time. Please see fastapi-users/fastapi-users#154 (comment)

Reset password

Get token for resetting password:
$ curl -X 'POST' \
  'http://localhost:8001/latest/auth/forgot-password' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "email": "user@example.com"
}'
Reponse: null

API logs:

kernelci-api | User 650bead8a4de36b741e3fbcf has forgot their password. Reset token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJwYXNzd29yZF9mZ3B0IjoiJDJiJDEyJHQxUHI4R1hFTGRSWEFBS0g4NFVpLk8zRXdidTZtN1g5TWN6RU4zT2l5LlNiLzJqd2dob2VtIiwiYXVkIjoiZmFzdGFwaS11c2VyczpyZXNldCIsImV4cCI6MTY5NTMwNDYwMn0.R-Iew4EBTZqA_qtpYgUINcnpnacBVHB7Gdatb4-ZU9Q

Reset the password using token:

$ curl -X 'POST' \
  'http://localhost:8001/latest/auth/reset-password' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJwYXNzd29yZF9mZ3B0IjoiJDJiJDEyJEZYeS9OVWZ4TGRIdlMxMVUvNmMzNGVDOG9nNTQxZWgwU1VYZGRRYmR3YWMxZGY5MGtKSjIuIiwiYXVkIjoiZmFzdGFwaS11c2VyczpyZXNldCIsImV4cCI6MTY5NTMwNTEyNn0.HKwIR-rN4qgJz-Fw1tLE4fbz7EA5SANFlBQPg3egN3I",
  "password": "jeny"
}'

Note: Sending tokens via emails can be done by adding logic to UserManager.on_after_forgot_password handler.

Get user

$ curl -X 'GET'   'http://localhost:8001/latest/users/me'   -H 'accept: application/json'   -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl19.a1D8r_z3L0MyIp83n88cHMtIv3DqxCDcH7wvSSwDeSo'
{"id":"650bead8a4de36b741e3fbcf","email":"user@example.com","is_active":true,"is_superuser":false,"is_verified":true,"username":"test"}

Update user

$ curl -X 'PATCH'   'http://localhost:8001/latest/users/me'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWFkOGE0ZGUzNmI3NDFlM2ZiY2YiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl19.a1D8r_z3L0MyIp83n88cHMtIv3DqxCDcH7wvSSwDeSo' -d '{
  "password": "test",
  "email": "user1@example.com",
  "username": "test"
}'
{"id":"650bead8a4de36b741e3fbcf","email":"user1@example.com","is_active":true,"is_superuser":false,"is_verified":true,"username":"test"}

Get user by id (accessible by superuser only)

$ curl -X 'GET'   'http://localhost:8001/latest/users/650bead8a4de36b741e3fbcf'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBkODZkZmQ2NDU2YTNiYTM2NDNiMDYiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl19.Nb-u2UMAY6cXScO2MziGjZzYGN-H81Gj_HnV_32nThE'
{"id":"650bead8a4de36b741e3fbcf","email":"user@example.com","is_active":true,"is_superuser":false,"is_verified":false,"username":"test"}

Update user by id (accessible by superuser only)

$ curl -X 'PATCH'   'http://localhost:8001/latest/users/650be1de418d00f4cc61cd2e'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBkODZkZmQ2NDU2YTNiYTM2NDNiMDYiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl19.Nb-u2UMAY6cXScO2MziGjZzYGN-H81Gj_HnV_32nThE' -d '{"username": "test"}'
{"id":"650be1de418d00f4cc61cd2e","email":"user@example.com","is_active":true,"is_superuser":false,"is_verified":false,"username":"test"}

Delete user by id (accessible by superuser only)

$ curl -X 'DELETE'   'http://localhost:8001/latest/users/650d85d6d6456a3ba3643b05'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBkODZkZmQ2NDU2YTNiYTM2NDNiMDYiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl19.Nb-u2UMAY6cXScO2MziGjZzYGN-H81Gj_HnV_32nThE'
Returns HTTP 204 No Content

Authenticate user

$ curl -X 'GET' 'http://localhost:8001/latest/authenticated-route' -H 'accept: application/json'  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2NTBiZWU0MzBlZjA5M2U2YmVlZDg3OGUiLCJhdWQiOlsiZmFzdGFwaS11c2VyczphdXRoIl0sImV4cCI6MTY5NTI5MjMwN30.iYYMxxHf5F_qrMUbNkfom5lMevE5lPgrNf2NO_U7wVw'
{"message":"Hello test!"}

gctucker · 2023-09-25T11:49:21Z

Looks like a great initial investigation. I shall give it a try to get a clearer idea of how this would work and try to provide some feedback.

On a side note, I see this uses PATCH queries for updating users. I wonder if it's better than PUT which is what we're using with other endpoints currently. Maybe PATCH is better for updating some data whereas PUT is normally expected to be used when replacing files? I would need to read the standard definitions again to be sure.

JenySadadia · 2023-09-25T12:17:23Z

On a side note, I see this uses PATCH queries for updating users. I wonder if it's better than PUT which is what we're using with other endpoints currently. Maybe PATCH is better for updating some data whereas PUT is normally expected to be used when replacing files? I would need to read the standard definitions again to be sure.

Yes, PATCH is for partial updates, and PUT is for replacing objects.
I agree that PATCH is better here as with it we don't need to provide fields that are not changing.

Extend `BaseUserManager` provided by `fastapi-users` to have customized core logic for user management. Add database adapter `get_user_db` to create a link between user model from `fastapi-users` and API database configuration. Inject `UserManager` at a runtime in a database session. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

Create an authentication backend with JWT as a strategy and bearer transport. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

Create an instance of `FastAPIUsers` to bind `UserManager` with authentication backend. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

Add pydantic models for validating schema and serialize response for `fastapi-users` routers. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

Register different routers provided by the package to our application. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

gctucker

Looks great to me :) Just a few comments but overall it seems like we're ready to upgrade this PR to a full implementation.

We might need to defer some things such as password reset via email and user moderation into follow-up PRs to discuss a few security issues with the sysadmins first.

@nuclearcat @VinceHillier ^ What do you guys think?

gctucker · 2023-10-04T17:03:35Z

api/models.py

+from beanie import Indexed, Document
+from fastapi_users.db import BeanieBaseUser
+from fastapi_users import schemas
+from beanie import PydanticObjectId, Indexed
+from typing import Optional


I wonder if these should rather be in a separate module to avoid "polluting" the whole api.models namespace with fastapi_users imports.

Yes, we should have a separate model file just like we have for pagination related models.

gctucker · 2023-10-04T17:05:39Z

api/models.py

        collection.create_index("profile.username", unique=True)


+class TestUser(BeanieBaseUser, Document):


Is the class called TestUser just because this is a PoC?

Yes, exactly.

gctucker · 2023-10-04T17:06:18Z

api/models.py

+
+
+class UserUpdate(schemas.BaseUserUpdate):
+    username: Optional[Indexed(str, unique=True)]


Out of interest, why is this one optional whereas it's not in the other models?

That's to allow users to only send fields that are updating. If I remove the Optional, the endpoint would expect username field in the request even if the user wants it to be unchanged.
If you look at the definition of the parent class BaseUserUpdate, you'll see all the fields are optional.

class BaseUserUpdate(CreateUpdateDictModel): password: Optional[str] email: Optional[EmailStr] is_active: Optional[bool] is_superuser: Optional[bool] is_verified: Optional[bool]

Right, I think this brings us back to the idea I mentioned a while ago about having a different model for the API and for the database.

gctucker · 2023-10-04T17:09:17Z

api/main.py

+    await init_beanie(
+        database=db.db,  
+        document_models=[
+            TestUser,  
+        ],
+    )


Maybe this call could be moved to api.auth with a method in the Database class to avoid exposing the underlying .db object? That way only the Database class would follow the object-oriented principle of accessing its own data. We already know we need to import the beanie modules in api.auth so it wouldn't be adding extra dependencies there.

The reason I kept this call in main.py was that init_beanie needs an actual instantiated database object and not just the class reference. Still, I'll take another look if I can move things around.

gctucker · 2023-10-04T17:12:29Z

api/auth.py

+    async def on_after_forgot_password(
+        self, user: TestUser, token: str, request: Optional[Request] = None
+    ):
+        print(f"User {user.id} has forgot their password. Reset token: {token}")


So I guess this is where an email would be sent with the reset token in a URL, and then the user would then come back via the on_after_request_verify method?

We could probably test this on the command line using an API token initially although this shouldn't be allowed in production as any leaked token should not enable changing the user's password imho. One issue of course is it's not possible to invalidate a JWT token, it just has a built-in expiry date. I wonder what's the standard way to deal with this in other APIs.

So I guess this is where an email would be sent with the reset token in a URL, and then the user would then come back via the on_after_request_verify method?

Yes, that is true.

gctucker · 2023-10-04T17:15:35Z

Ah also I think this doesn't enable new users to sing up with just an email address, so that's one less security thing to consider here. We'll need to work this out properly for a full production deployment though.

JenySadadia · 2023-10-07T06:41:26Z

Completed PoC. Moving forward with the actual integration.

Jeny Sadadia added 2 commits September 25, 2023 12:22

api.models: extend models from fastapi-users

01b6570

Add `TestUser` model and extend `User` model provided by `fastapi-users`. Add an extra field `username` and create `unique` index on it. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

JenySadadia mentioned this pull request Sep 25, 2023

Investigate fastapi-users #309

Closed

4 tasks

Jeny Sadadia added 2 commits September 25, 2023 16:42

api.db: get database instance

c831920

Also added a method to get database instance `Database._db` to use it for initializing Beanie on API startup. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

api.main: initialize Beanie on app startup

8c47587

`fastapi-users` uses `Beanie` ODM to provide tools to work with MongoDB. We need to initialize Beanie with database and user model on the application startup. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

JenySadadia force-pushed the fastapi-users-poc branch from 996d35a to 86a6b7d Compare September 25, 2023 11:16

JenySadadia added the staging-skip Don't test automatically on staging.kernelci.org label Sep 26, 2023

Jeny Sadadia added 5 commits September 27, 2023 17:16

api.auth: create authentication backend

6209bad

Create an authentication backend with JWT as a strategy and bearer transport. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

api.auth: create an instance of FastAPIUsers

51ed081

Create an instance of `FastAPIUsers` to bind `UserManager` with authentication backend. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

api.models: add models for fastapi-users routers

faf2d00

Add pydantic models for validating schema and serialize response for `fastapi-users` routers. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

api.main: include routers provided by fastapi-users

a51e911

Register different routers provided by the package to our application. Signed-off-by: Jeny Sadadia <jeny.sadadia@collabora.com>

JenySadadia force-pushed the fastapi-users-poc branch from 86a6b7d to a51e911 Compare September 27, 2023 11:47

JenySadadia marked this pull request as draft October 3, 2023 10:24

JenySadadia requested a review from gctucker October 4, 2023 09:59

gctucker reviewed Oct 4, 2023

View reviewed changes

JenySadadia closed this Oct 7, 2023

JenySadadia deleted the fastapi-users-poc branch October 16, 2023 04:40

		collection.create_index("profile.username", unique=True)


		class TestUser(BeanieBaseUser, Document):



		class UserUpdate(schemas.BaseUserUpdate):
		username: Optional[Indexed(str, unique=True)]

PoC: fastapi-users #363

PoC: fastapi-users #363

Uh oh!

Conversation

JenySadadia commented Sep 25, 2023

Uh oh!

JenySadadia commented Sep 25, 2023

Uh oh!

gctucker commented Sep 25, 2023

Uh oh!

JenySadadia commented Sep 25, 2023

Uh oh!

gctucker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gctucker commented Oct 4, 2023

Uh oh!

JenySadadia commented Oct 7, 2023

Uh oh!

Uh oh!

PoC: `fastapi-users` #363

PoC: `fastapi-users` #363