[16.0] FastAPI Integration #291

lmignon · 2022-09-28T06:58:44Z

manage exceptions and transaction rollback (through the use of a dedicated starlette middleware???)
manage languages
works with extendable-pydantic
depends on [16.0] endpoint_route_handler: migration to v16 web-api#17
improves UI to alert the user that the routing map is out of sync and must be synchronized when one element involved into the routing definition is updated or a new app is created.
rework the documentation to address the security aspect from the start and to evolve its implementation in the examples given in the following topics.

lmignon · 2022-10-28T15:31:10Z

@Julien00859 Here it's a usage of the odoo.http.Dispatcherclass to register a new kind of query....

lmignon · 2022-12-10T16:51:04Z

@sebastienbeau @sbidoul @simahawk @StefanRijnhart I took some time to write some documentation explaining what writing a REST api is and how to do it with this new addon. I still have to understand why the tests are failing in our CI but not on my laptop. Your comments and suggestions are welcome.

sbidoul

This is seriously great !

fastapi/depends.py

sbidoul · 2022-12-10T17:19:21Z

fastapi/depends.py

+def authenticated_partner_from_basic_auth_user(
+    user: Users = Depends(basic_auth_user),  # noqa: B008
+) -> Partner:
+    return user.partner_id


Use env["res.partner"].browse(user.partner_id), so the partner is not sudo?

sbidoul · 2022-12-10T17:27:13Z

fastapi/readme/ROADMAP.rst

+because the integration of the fastapi is based on the use of a specific middleware
+that convert the WSGI request consumed by odoo to a ASGI request. The question
+is to know if it is also possible to develop the same kind of bridge for the
+WebSockets.


I suppose the same challenge applies to streaming large responses in the asgi-wsgi bridge?

sbidoul · 2022-12-10T17:27:44Z

fastapi/readme/USAGE.rst

+Before you start, we must define some terms:
+
+* **App**: A FastAPI app is a collection of routes, dependencies, and other
+  components that can be used to build a web application.


also an ASGI app?

sbidoul · 2022-12-10T17:34:51Z

fastapi/readme/USAGE.rst

+
+* **'odoo_env'**: Returns the current odoo environment.
+* **'fastapi_endpoint'**: Returns the current fastapi endpoint model instance.
+* **'authenticated_partner'**: Returns the authenticated partner.


Mention authenticated_partner_env and when to use it vs odoo_env?

sbidoul · 2022-12-10T17:48:37Z

fastapi/readme/USAGE.rst

+* You can change the implementation of the route handler.
+* You can override the dependencies of the route handler.
+* You can add a new route handler.
+* You can extend the model used as parameter or as response of the route handler.


Elaborate this? One can add optional fields (or required fields with default values) to the request.
In the response, one can add fields, or make optional fields required, but not make required fields optional.
One can remove optional fields from a response (depending on what optional means exactly - null vs absent).

sbidoul · 2022-12-10T17:49:42Z

fastapi/readme/USAGE.rst

+      _name = "demo.fastapi.endpoint"
+      _description = "Demo Endpoint"
+
+      def echo(self, message: str) -> str:


@api.model?

sbidoul · 2022-12-10T17:55:04Z

fastapi/readme/USAGE.rst

+
+  The liskov substitution principle has also to be respected. That means that
+  if you extend a model, you must add new required fields or you must provide
+  default values for the new optional fields.


sbidoul · 2022-12-10T18:00:16Z

fastapi/readme/USAGE.rst

+
+* Use plural for the name of a service. For example, if you provide a service
+  that allows you to manage the sale orders, you must use the name 'sale_orders'
+  and not 'sale_order'.


Mention the concept of collection of resources which must be named with a plural (/sale_orders/{id}?

sbidoul · 2022-12-10T18:01:01Z

fastapi/readme/USAGE.rst

+
+
+
+* ... and many more.


Say a word about input validation. Be paranoid about what clients send, assume they will send invalid id's, or attempt to access forbidden data.

sebastienbeau · 2023-01-04T00:54:53Z

fastapi/models/fastapi_endpoint_demo.py

+        app.dependency_overrides[
+            authenticated_partner_impl
+        ] = authenticated_partner_impl_override


Suggested change

app.dependency_overrides[

authenticated_partner_impl

] = authenticated_partner_impl_override

app.dependency_overrides[

authenticated_partner_impl

] = authenticated_partner_impl_override

Indentation is wrong, so when self.app is different then "demo" it raise an error as authenticated_partner_impl_override is not defined

…pi service excution

* add default empty method to use as dependency to get the authenticated partner * improves the demo app to illustrate the way the dependency overrides mechanism can be used to provide the right implementation to use to retrieve the authenticated partner according to the security method configured on the app * add tests for the demo app to show how the TestClient class and the dependey overrides functianality should be used to easily write tests

This method can be used to get access to the fastapi.endpoint record into your router's methods

ensure transation is rolled back in case of error and allows override / extension of the error handling globally or by app

…ndlers

flachica · 2023-02-24T13:34:40Z

fastapi/models/fastapi_endpoint.py

+    def _get_app_exception_handlers(
+        self,
+    ) -> Dict[
+        int | Type[Exception],


Suggested change

int | Type[Exception],

Union[int, Type[Exception]],

With this notation fastapi support python 3.7+ as you can see documentation section union. Otherwise we would be locked to python 3.10

Thank you @flachica
Happy to see someone making a try with this draft PR. I plan to work on it in the coming days to publish a first release.

I am thoroughly studying this module and related technology. I intend to use it extensively in the very short term. I understand that it is in an initial state and I will do my best to contribute according to my possibilities and assuming that changes will come and I will have to adapt. Thank you very much for all the effort, especially in the documentation.

I've two issues to solve to be complete.

~~The management of the languages to return the translated info into the requested language if available. This will be based on the accept-language http header~~ done in 1a750d4

The use of extendable-pydantic is not yet completely functional. For some processes, the fastapi lib use internal methods from the pydantic lib without going through the model class directly. This means that the extension mechanisms implemented by the extendable-fastapi library at the class definition level are bypassed in these cases. I have identified at this stage 2 places that need to be adapted so that the schema for an extended model is correctly generated in the documentation and the serialization is based on the extended class definition and not the base class. Fixed into the last release of extenable-pydantic (0.0.2)

We'll probably find others issues but I'm confident that nothing should call into question the approach, which seems to me to be validated from a technical point of view. I'll start to use it for my own projects in the coming weeks.

@flachica The integration of extendable-pydantic and fastapi is now fixed into the last version of extendable-pydantic (>=0.0.2) I've to adapt the documentation to specify that an extra odoo addon is required to 'extendable_fastapi'. I'll add the management of the languages tomorrow.

flachica · 2023-03-03T16:32:13Z

fastapi/readme/USAGE.rst

+        """
+        # TODO we should declare a technical user with read access only on the
+        # fastapi.endpoint model
+        return env["fastapi.endpoint"].sudo().browse(_id)


Suggested change

return env["fastapi.endpoint"].sudo().browse(_id)

return env["fastapi.endpoint"].browse(_id)

In endpoint sudo not must be used

flachica · 2023-03-03T16:41:48Z

I have built my enpoint and it works perfectly. Now it turns out that I've done the tests for that endpoint and it doesn't work. If the enpoint only returns a message it works fine, but if it has to access the database it doesn't throw the exception, nor does it exit self.env[model].search().

I don't know what I'm doing wrong. I'm waiting for the writing of that part of documentation of security that you mention that is pending.

lmignon · 2023-03-06T07:25:49Z

@flachica About security you've 2 point to take into account.

The user specified on endpoint definition. This user is the user that will be used to run the logic. This user should be put in a specific security group without any access.
The authentication mechanism. The authentication mechanism is responsible to provide the partner requesting the partner. In the same time, the system will inject into the evaluation context of the record rules the authenticated_partner_id . This will allow you write the required record rules to secure your services and link these rule to the specific security group of your technical user running the services.

The principles are explained here https://github.com/OCA/rest-framework/pull/291/files#diff-d864270e7f736cc705f667b331dd7242c7841de1783c06d10550a27ede6d6ed3R803

In the same time, applies the security guidelines for the demo app

lmignon · 2023-03-10T10:59:43Z

@flachica In my last commit I improved the documentation about security aspects and provides now a basic group to use when creating your own security group for your app.

flachica · 2023-03-10T11:12:43Z

@flachica In my last commit I improved the documentation about security aspects and provides now a basic group to use when creating your own security group for your app.

I have read it. It's great

guewen · 2023-03-11T07:40:15Z

fastapi/models/fastapi_endpoint_demo.py

+        selection_add=[("demo", "Demo Endpoint")], ondelete={"demo": "cascade"}
+    )
+    demo_auth_method = fields.Selection(
+        selection=[("api_key", "Api Key"), ("http_basic", "HTTP Bacic")],


Suggested change

selection=[("api_key", "Api Key"), ("http_basic", "HTTP Bacic")],

selection=[("api_key", "Api Key"), ("http_basic", "HTTP Basic")],

florian-dacosta · 2023-03-13T09:36:50Z

fastapi/readme/USAGE.rst

+  <record id="my_demo_app_group" model="res.groups">
+    <field name="name">My Demo Endpoint Group</field>
+    <field name="users" eval="[(4, ref('my_demo_app_user'))]" />
+    <field name="implied_ids" eval="[(4, ref('fast_api.group_fastapi_endpoint_runner'))]" />


fast_api -> fastapi

jbaudoux · 2023-03-27T11:08:24Z

fastapi/readme/USAGE.rst

+
+.. code-block:: python
+
+    from ..depends import odoo_env


Suggested change

from ..depends import odoo_env

from odoo.addons.fastapi.depends import odoo_env

jbaudoux · 2023-03-27T11:09:13Z

fastapi/readme/USAGE.rst

+
+.. code-block:: python
+
+    from ..depends import odoo_env


Suggested change

from ..depends import odoo_env

from odoo.addons.fastapi.depends import odoo_env

linhhonblade · 2023-04-24T07:07:42Z

May I know the fastapi version you used? I tried to run these code recently but got error "cannot import name 'APIRouter' from 'fastapi'"

lmignon · 2023-04-24T07:29:30Z

May I know the fastapi version you used? I tried to run these code recently but got error "cannot import name 'APIRouter' from 'fastapi'"

AFAIK Tha APIRouter is in all versions of fastapi https://github.com/tiangolo/fastapi/blob/c81e136d75f5ac4252df740b35551cf2afb4c7f1/fastapi/routing.py#L478

flachica · 2023-05-04T12:15:16Z

Dear @lmignon, while reviewing the security issue of the fastapi module, I realized that it may be necessary to adapt the base_rest_auth_user_service module to depend on it and not on base_rest. (And make it work with Odoo OAuth if it is possible)

The tests I have done have been from Postman and skipping the need to provide security to the system. But that time has passed.

The question is this: Is there any plan to move the base_rest dependent modules that are already in 16.0 to fastapi? Is there any progress in this regard? It is not to duplicate work.

I would like to contribute and I think that in order to maximize the value it is good to reach an agreement.

When a field is of an extended type, the assemebled class is not part of the know classes computed by fastapi from the fastapi endepoint definition. Is such a case, we must add the extented type to the list of knwon classes

lmignon · 2023-06-07T15:25:52Z

/ocabot merge nobump

OCA-git-bot · 2023-06-07T15:26:14Z

What a great day to merge this nice PR. Let's do it!
Prepared branch 16.0-ocabot-merge-pr-291-by-lmignon-bump-nobump, awaiting test results.

OCA-git-bot · 2023-06-07T15:32:07Z

Congratulations, your PR was merged at db3e878. Thanks a lot for contributing to OCA. ❤️

sbidoul · 2023-06-08T10:29:48Z

fastapi/depends.py

+    to declare the way your partner will be provided. In some case, this
+    partner will come from the authentication mechanism (ex jwt token) in other cases
+    it could comme from a lookup on an email received into an HTTP header ...
+    See the fastapi_endpoint_demo for an example"""


@lmignon should this not raise NotImplementedError ?

Ok, I'll do that in the fastapi_auth_jwt PR.

lmignon · 2023-06-08T11:28:11Z

Yes, it should be.

…

On Thu, Jun 8, 2023 at 12:30 PM Stéphane Bidoul ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In fastapi/depends.py <#291 (comment)>: > +from .schemas import Paging + +if TYPE_CHECKING: + from .models.fastapi_endpoint import FastapiEndpoint + + +def odoo_env() -> Environment: + yield odoo_env_ctx.get() + + +def authenticated_partner_impl() -> Partner: + """This method has to be overriden when you create your fastapi app + to declare the way your partner will be provided. In some case, this + partner will come from the authentication mechanism (ex jwt token) in other cases + it could comme from a lookup on an email received into an HTTP header ... + See the fastapi_endpoint_demo for an example""" @lmignon <https://github.com/lmignon> should this not raise NotImplementedError ? — Reply to this email directly, view it on GitHub <#291 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEE2WXPP2XWDBZ4TQ2YECTXKGSSVANCNFSM6AAAAAAQXOT7K4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Josesosa07 · 2023-06-13T19:58:23Z

Hi, I am using in the manifest: ["base", "base_rest_auth_api_key", "base_rest_datamodel",]
but I got a warning saying: rest-framework/base_rest/init.py: base_rest is deprecated and not fully supported anymore on Odoo 16. Please migrate to the FastAPI migration module. See #291.
I am in a project in odoo 16, if I continue using the rest-api in this way, will I get issues in the project?

Do I need to use fastapi? If so, how should it be implemented?

sbidoul · 2023-06-14T06:22:16Z

@Josesosa07 base_rest is deemed deprecated because it's main maintainers (@lmignon and ACSONE) have decided to move towards a FastAPI integration which is much more powerful and simplier, delegating a lot of the heavy lifting to the fastapi library.

So if you are starting a new project, it is better to start with the fastapi addon in this repo, which also has extensive documentation on how to get started. If you are porting existing base_rest services from a previous version and don't want to convert them to FastAPI, base_rest is still just fine.

lmignon mentioned this pull request Sep 28, 2022

Update our tooling to support Odoo 16.0 OCA/maintainer-tools#537

Closed

22 tasks

lmignon force-pushed the 16.0-fastapi branch 4 times, most recently from 23aabb2 to e54b56a Compare October 9, 2022 09:46

lmignon force-pushed the 16.0-fastapi branch 3 times, most recently from 6503eb6 to bb4c85d Compare December 10, 2022 16:07

sbidoul reviewed Dec 10, 2022

View reviewed changes

sebastienbeau reviewed Jan 4, 2023

View reviewed changes

lmignon force-pushed the 16.0-fastapi branch from cac1747 to df2f6ed Compare February 12, 2023 09:01

lmignon and others added 15 commits February 12, 2023 10:07

[ADD] Allows use of fastapi into Odoo!

d071de2

[IMP] fastapi: Uses contextvars to hold and provide odoo env on fasta…

f2f56ac

…pi service excution

[IMP] fastapi: renames fastapi.app into fastapi.endpoint

e7b7b3b

[IMP] fastapi: code cleanup

aa3bcd7

[IMP] fastapi: Add dependency method

a02ecd9

This method can be used to get access to the fastapi.endpoint record into your router's methods

[IMP] fastapi: Implement error handling

9445431

ensure transation is rolled back in case of error and allows override / extension of the error handling globally or by app

[IMP] fastapi: Allows to define the user executing the api calls

3835a31

[IMP] fastapi: Add authenticated_partner_id into ir.rule eval context

615467f

[IMP] fastapi: Add helper code to ease development of search route ha…

d11cda1

…ndlers

[IMP] fastapi: documentation

b90df58

[FIX] add test dependency for fastapi

73173b5

Fix a few typos

ab34529

Fix typo

790d23f

little fixes

dca1b58

lmignon force-pushed the 16.0-fastapi branch from df2f6ed to dca1b58 Compare February 12, 2023 09:30

flachica suggested changes Feb 24, 2023

View reviewed changes

flachica suggested changes Mar 3, 2023

View reviewed changes

[IMP] fastapi: Improves documentation on security aspects

c121741

In the same time, applies the security guidelines for the demo app

guewen reviewed Mar 11, 2023

View reviewed changes

florian-dacosta reviewed Mar 13, 2023

View reviewed changes

jbaudoux reviewed Mar 27, 2023

View reviewed changes

lmignon added 2 commits April 24, 2023 09:26

[IMP] fastapi: Declared as Beta

f1cfc32

[IMP] fastapi: Typos

6ea99a9

lmignon marked this pull request as ready for review June 6, 2023 11:47

lmignon added 2 commits June 7, 2023 12:22

Fix schema generation

ee0c889

When a field is of an extended type, the assemebled class is not part of the know classes computed by fastapi from the fastapi endepoint definition. Is such a case, we must add the extented type to the list of knwon classes

[FIX] fastapi: odoo-addon-endpoint-route-handler is merged

118742a

OCA-git-bot added the bot is merging ⏳ label Jun 7, 2023

OCA-git-bot merged commit e54d646 into OCA:16.0 Jun 7, 2023
5 checks passed

OCA-git-bot added merged 🎉 and removed bot is merging ⏳ labels Jun 7, 2023

sbidoul deleted the 16.0-fastapi branch June 7, 2023 20:28

sbidoul reviewed Jun 8, 2023

View reviewed changes

	return env["fastapi.endpoint"].sudo().browse(_id)
	return env["fastapi.endpoint"].browse(_id)

	selection=[("api_key", "Api Key"), ("http_basic", "HTTP Bacic")],
	selection=[("api_key", "Api Key"), ("http_basic", "HTTP Basic")],

	from ..depends import odoo_env
	from odoo.addons.fastapi.depends import odoo_env

[16.0] FastAPI Integration #291

[16.0] FastAPI Integration #291

Conversation

lmignon commented Sep 28, 2022 • edited

lmignon commented Oct 28, 2022

lmignon commented Dec 10, 2022

sbidoul left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lmignon Feb 26, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flachica commented Mar 3, 2023

lmignon commented Mar 6, 2023

lmignon commented Mar 10, 2023

flachica commented Mar 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linhhonblade commented Apr 24, 2023

lmignon commented Apr 24, 2023

flachica commented May 4, 2023

lmignon commented Jun 7, 2023

OCA-git-bot commented Jun 7, 2023

OCA-git-bot commented Jun 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lmignon commented Jun 8, 2023 via email

Josesosa07 commented Jun 13, 2023 • edited

sbidoul commented Jun 14, 2023

lmignon commented Sep 28, 2022 •

edited

lmignon Feb 26, 2023 •

edited

Josesosa07 commented Jun 13, 2023 •

edited