Skip to content

Commit

Permalink
docs: middleware refresh
Browse files Browse the repository at this point in the history
  • Loading branch information
wolovim committed Mar 6, 2024
1 parent 9859afb commit c873446
Show file tree
Hide file tree
Showing 4 changed files with 129 additions and 134 deletions.
103 changes: 0 additions & 103 deletions docs/internals.rst
Original file line number Diff line number Diff line change
Expand Up @@ -188,109 +188,6 @@ provider instance.
w3 = Web3(HTTPProvider(endpoint_uri="...", retry_configuration=None)
.. _internals__middlewares:
Middlewares
-----------
.. note:: The Middleware API in web3 borrows from the Django middleware API introduced
in version 1.10.0
Middlewares provide a simple yet powerful api for implementing layers of business logic
for web3 requests. Writing middleware is simple and extending from the base
``Web3Middleware`` class allows for overriding only the parts of the middleware that
make sense for your use case. If all you need to do is modify the
params before the request is made, you can override the ``request_processor`` method,
make the necessary tweaks to the params, and pass the arguments to the next element in
the middleware stack. If processing the response is the only concern, you only need to
override the ``response_processor`` method and return the response.
.. code-block:: python
from web3.middlewares import Web3Middleware
class SimpleMiddleware(Web3Middleware):
def request_processor(self, method, params):
# Pre-request processing goes here before passing to the next middleware.
return (method, params)
def response_processor(self, method, response):
# Response processing goes here before passing to the next middleware.
return response
# If your provider is asynchronous, override the async methods instead
async def async_request_processor(self, method, params):
return (method, params)
async def async_response_processor(self, method, response):
return response
Wrapping the ``make_request`` method of a provider is possible. If you wish to prevent
making a call under certain conditions, for example, you can override the
``wrap_make_request`` method. This allows for defining pre-request processing,
skipping or making the request under certain conditions, as well as response
processing before passing it to the next middleware. The order of operations still
passes all pre-request processing down the middlewares before making the request,
and then passes the response back up the middleware stack for processing. The next
middleware on the stack is essentially the "make_request" method until we reach
the end of the middlewares and the request is made by the actual ``make_request``
method of the provider. The response is then passed back up the middleware stack for
processing before being returned to the user.
.. code-block:: python
from web3.middlewares import Web3Middleware
class SimpleMiddleware(Web3Middleware):
def wrap_make_request(self, make_request):
def middleware(method, params):
# pre-request processing goes here
response = make_request(method, params) # make the request
# response processing goes here
return response
# If your provider is asynchronous, override the async method instead
async def async_wrap_make_request(self, make_request):
async def middleware(method, params):
# pre-request processing goes here
response = await make_request(method, params)
# response processing goes here
return response
return middleware
The ``RequestManager`` object exposes the ``middleware_onion`` object to manage
middlewares. It is also exposed on the ``Web3`` object for convenience. That API is
detailed in :ref:`Modifying_Middleware`.
Middlewares are added to the middleware stack as the class itself. The ``name`` kwarg is
optional.
.. code-block:: python
from web3 import Web3
from my_module import (
SimpleMiddleware,
)
w3 = Web3(HTTPProvider(endpoint_uri="..."))
# add the middleware to the stack as the class
w3.middleware_onion.add(SimpleMiddleware, name="simple_middleware")
Managers
--------
Expand Down
157 changes: 127 additions & 30 deletions docs/middleware.rst
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
.. _middleware_internals:

Middleware
==========

Web3 manages layers of middlewares by default. They sit between the public Web3 methods and the
:doc:`providers`, which handle native communication with the Ethereum client. Each layer
can modify the request and/or response. Some middlewares are enabled by default, and
others are available for optional use.
``Web3`` is instantiated with layers of middleware by default. They sit between the public
``Web3`` methods and the :doc:`providers`, and are used to perform sanity checks, convert data
types, enable ENS support, and more. Each layer can modify the request and/or response.
While several middleware are enabled by default, others are available for optional use,
and you're free to create your own!

Each middleware layer gets invoked before the request reaches the provider, and then
processes the result after the provider returns, in reverse order. However, it is
possible for a middleware to return early from a
call without the request ever getting to the provider (or even reaching the middlewares
that are in deeper layers).

More information is available in the "Internals: :ref:`internals__middlewares`" section.
possible for a middleware to return early from a call without the request ever getting
to the provider (or even reaching the middleware that are in deeper layers).


.. _default_middleware:

Default Middleware
------------------

The following middlewares are added by default if you don't add any:
The following middleware are included by default:

* ``gas_price_strategy``
* ``ens_name_to_address``
* ``attrdict``
* ``validation``
* ``gas_estimate``

The defaults are defined in the ``default_middlewares()`` method in ``web3/manager.py``.
The defaults are defined in the ``get_default_middlewares()`` method in ``web3/manager.py``.

AttributeDict
~~~~~~~~~~~~~
Expand Down Expand Up @@ -101,7 +101,7 @@ can name the middleware for later reference.
Middleware Order
~~~~~~~~~~~~~~~~

Think of the middlewares as being layered in an onion, where you initiate a web3.py request at
Think of the middleware as being layered in an onion, where you initiate a web3.py request at
the outermost layer of the onion, and the Ethereum node (like geth) receives and responds
to the request inside the innermost layer of the onion. Here is a (simplified) diagram:

Expand Down Expand Up @@ -158,14 +158,15 @@ to the request inside the innermost layer of the onion. Here is a (simplified) d
Returned value in web3.py
The middlewares are maintained in ``Web3.middleware_onion``. See below for the API.
The middleware are maintained in ``Web3.middleware_onion``. See below for the API.

When specifying middlewares in a list, or retrieving the list of middlewares, they will
When specifying middleware in a list, or retrieving the list of middleware, they will
be returned in the order of outermost layer first and innermost layer last. In the above
example, that means that ``w3.middleware_onion.middlewares`` would return the middlewares in
the order of: ``[2, 1, 0]``.
example, that means that ``w3.middleware_onion.middlewares`` would return the middleware
in the order of: ``[2, 1, 0]``.


See "Internals: :ref:`internals__middlewares`" for a deeper dive to how middlewares work.
.. _middleware_stack_api:

Middleware Stack API
~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -237,7 +238,7 @@ To add or remove items in different layers, use the following API:
.. py:method:: Web3.middleware_onion.clear()
Empty all the middlewares, including the default ones.
Empty all the middleware, including the default ones.

.. code-block:: python
Expand All @@ -247,15 +248,15 @@ To add or remove items in different layers, use the following API:
.. py:attribute:: Web3.middleware_onion.middlewares
Return all the current middlewares for the ``Web3`` instance in the appropriate order for importing into a new
Return all the current middleware for the ``Web3`` instance in the appropriate order for importing into a new
``Web3`` instance.

.. code-block:: python
>>> w3_1 = Web3(...)
# add uniquely named middleware:
>>> w3_1.middleware_onion.add(web3.middleware.GasPriceStrategyMiddleware, 'test_middleware')
# export middlewares from first w3 instance
# export middleware from first w3 instance
>>> middlewares = w3_1.middleware_onion.middlewares
# import into second instance
Expand All @@ -264,22 +265,27 @@ To add or remove items in different layers, use the following API:
>>> assert w3_2.middleware_onion.get('test_middleware')
Optional Middleware
-------------------

Web3 ships with non-default middleware, for your custom use. In addition to the other ways of
:ref:`Modifying_Middleware`, you can specify a list of middleware when initializing Web3, with:
Instantiate with Custom Middleware
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Instead of working from the default list, you can specify a custom list of
middleware when initializing Web3:

.. code-block:: python
Web3(middlewares=[my_middleware1, my_middleware2])
.. warning::
This will *replace* the default middlewares. To keep the default functionality,
either use ``middleware_onion.add()`` from above, or add the default middlewares to
your list of new middlewares.
This will *replace* the default middleware. To keep the default functionality,
either use ``middleware_onion.add()`` from above, or add the default middleware to
your list of new middleware.

Below is a list of available middlewares which are not enabled by default.

Optional Middleware
-------------------

``Web3`` includes optional middleware for common use cases. Below is a list of available
middleware which are not enabled by default.

Stalecheck
~~~~~~~~~~~~
Expand Down Expand Up @@ -339,7 +345,7 @@ middleware is:
# confirm that the connection succeeded
>>> w3.client_version
'Geth/v1.7.3-stable-4bb3c89d/linux-amd64/go1.9'
'Geth/v1.13.11-stable-4bb3c89d/linux-amd64/go1.20.2'
This example connects to a local ``geth --dev`` instance on Linux with a
unique IPC location and loads the middleware:
Expand Down Expand Up @@ -495,3 +501,94 @@ A legacy transaction still works in the same way as it did before EIP-1559 was i
... 'gasPrice': 123456, # optional - if not provided, gas_price_strategy (if exists) or eth_gasPrice is used
... }
>>> w3.eth.send_transaction(legacy_transaction)
Create Middleware
-----------------

To write your own middleware, create a class and extend from the base ``Web3Middleware``
class, then override only the parts of the middleware that make sense for your use case.

.. note:: The Middleware API borrows from the Django middleware API introduced
in version 1.10.0.

If all you need is to modify the params before a request is made, you can override
the ``request_processor`` method, make the necessary tweaks to the params, and pass the
arguments to the next element in the middleware stack. Need to do some processing on the
response? Override the ``response_processor`` method and return the modified response.

The pattern:

.. code-block:: python
from web3.middleware import Web3Middleware
class CustomMiddleware(Web3Middleware):
def request_processor(self, method, params):
# Pre-request processing goes here before passing to the next middleware.
return (method, params)
def response_processor(self, method, response):
# Response processing goes here before passing to the next middleware.
return response
# If your provider is asynchronous, override the async methods instead:
async def async_request_processor(self, method, params):
# Pre-request processing goes here before passing to the next middleware.
return (method, params)
async def async_response_processor(self, method, response):
# Response processing goes here before passing to the next middleware.
return response
If you wish to prevent making a call under certain conditions, you can override the
``wrap_make_request`` method. This allows for defining pre-request processing,
skipping or making the request under certain conditions, as well as response
processing before passing it to the next middleware.


.. code-block:: python
from web3.middleware import Web3Middleware
class CustomMiddleware(Web3Middleware):
def wrap_make_request(self, make_request):
def middleware(method, params):
# pre-request processing goes here
response = make_request(method, params) # make the request
# response processing goes here
return response
return middleware
# If your provider is asynchronous, override the async method instead:
async def async_wrap_make_request(self, make_request):
async def middleware(method, params):
# pre-request processing goes here
response = await make_request(method, params)
# response processing goes here
return response
return middleware
Custom middleware can be added to the stack via the class itself, using the
:ref:`middleware_stack_api`. The ``name`` kwarg is optional. For example:

.. code-block:: python
from web3 import Web3
from my_module import (
CustomMiddleware,
)
w3 = Web3(HTTPProvider(endpoint_uri="..."))
# add the middleware to the stack as the class
w3.middleware_onion.add(CustomMiddleware, name="custom_middleware")
2 changes: 1 addition & 1 deletion docs/v7_migration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ a simpler, clearer interface for defining middleware, gives more flexibility for
asynchronous operations and also paves the way for supporting batch requests - included in
the roadmap for web3.py.

The new middleware model is documented in the :ref:`internals__middlewares` section.
The new middleware model is documented in the :ref:`middleware_internals` section.


Middleware Renaming and Removals
Expand Down
1 change: 1 addition & 0 deletions newsfragments/3266.docs.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Refresh of the middleware docs

0 comments on commit c873446

Please sign in to comment.