Skip to content

Latest commit

 

History

History
566 lines (404 loc) · 22.6 KB

providers.rst

File metadata and controls

566 lines (404 loc) · 22.6 KB
Raw

Providers

The provider is how web3 talks to the blockchain. Providers take JSON-RPC requests and return the response. This is normally done by submitting the request to an HTTP or IPC socket based server.

Note

web3.py supports one provider per instance. If you have an advanced use case that requires multiple providers, create and configure a new web3 instance per connection.

If you are already happily connected to your Ethereum node, then you can skip the rest of the Providers section.

Choosing How to Connect to Your Node

Most nodes have a variety of ways to connect to them. If you have not decided what kind of node to use, head on over to choosing_node

The most common ways to connect to your node are:

  1. IPC (uses local filesystem: fastest and most secure)
  2. WebSocket (works remotely, faster than HTTP)
  3. HTTP (more nodes support it)

If you're not sure how to decide, choose this way:

  • If you have the option of running web3.py on the same machine as the node, choose IPC.
  • If you must connect to a node on a different computer, use WebSocket.
  • If your node does not support WebSocket, use HTTP.

Most nodes have a way of "turning off" connection options. We recommend turning off all connection options that you are not using. This provides a safer setup: it reduces the number of ways that malicious hackers can try to steal your ether.

Once you have decided how to connect, you specify the details using a Provider. Providers are web3.py classes that are configured for the kind of connection you want.

See:

  • ~web3.providers.rpc.HTTPProvider
  • ~web3.providers.ipc.IPCProvider
  • ~web3.providers.async_rpc.AsyncHTTPProvider
  • ~web3.providers.persistent.AsyncIPCProvider (Persistent Connection Provider)
  • ~web3.providers.persistent.WebSocketProvider (Persistent Connection Provider)
  • ~web3.providers.legacy_websocket.LegacyWebSocketProvider (Deprecated)

Each provider above should link to the documentation on how to properly initialize the provider. Once you have reviewed the relevant documentation for the provider of your choice, you are ready to get started with web3.py<first_w3_use>.

Provider via Environment Variable

Alternatively, you can set the environment variable WEB3_PROVIDER_URI before starting your script, and web3 will look for that provider first.

Valid formats for this environment variable are:

  • file:///path/to/node/rpc-json/file.ipc
  • http://192.168.1.2:8545
  • https://node.ontheweb.com
  • ws://127.0.0.1:8546

Auto-initialization Provider Shortcuts

Geth dev Proof of Authority

To connect to a geth --dev Proof of Authority instance with the POA middleware loaded by default:

>>> from web3.auto.gethdev import w3

# confirm that the connection succeeded
>>> w3.is_connected()
True

Or, connect to an async web3 instance:

>>> from web3.auto.gethdev import async_w3
>>> await async_w3.provider.connect()

# confirm that the connection succeeded
>>> await async_w3.is_connected()
True

Built In Providers

Web3 ships with the following providers which are appropriate for connecting to local and remote JSON-RPC servers.

HTTPProvider

IPCProvider

AsyncHTTPProvider

Persistent Connection Providers

Persistent Connection Base Class

Note

This class is not meant to be used directly. If your provider class inherits from this class, look to these docs for additional configuration options.

)

This is a base provider class, inherited by the following providers:

  • ~web3.providers.persistent.WebSocketProvider
  • ~web3.providers.persistent.AsyncIPCProvider

It handles interactions with a persistent connection to a JSON-RPC server. Among its configuration, it houses all of the ~web3.providers.persistent.request_processor.RequestProcessor logic for handling the asynchronous sending and receiving of requests and responses. See the internals__persistent_connection_providers section for more details on the internals of persistent connection providers.

  • request_timeout is the timeout in seconds, used when sending data over the connection and waiting for a response to be received from the listener task. Defaults to 50.0.
  • subscription_response_queue_size is the size of the queue used to store subscription responses, defaults to 500. While messages are being consumed, this queue should never fill up as it is a transient queue and meant to handle asynchronous receiving and processing of responses. When in sync with the socket stream, this queue should only ever store 1 to a few messages at a time.
  • silence_listener_task_exceptions is a boolean that determines whether exceptions raised by the listener task are silenced. Defaults to False, raising any exceptions that occur in the listener task.

AsyncIPCProvider

WebSocketProvider

Using Persistent Connection Providers

The AsyncWeb3 class may be used as a context manager, utilizing the async with syntax, when instantiating with a ~web3.providers.persistent.PersistentConnectionProvider. This will automatically close the connection when the context manager exits and is the recommended way to initiate a persistent connection to the provider.

A similar example using a websockets connection as an asynchronous context manager can be found in the websockets connection docs.

>>> import asyncio
>>> from web3 import AsyncWeb3
>>> from web3.providers.persistent import (
...     AsyncIPCProvider,
...     WebSocketProvider,
... )

>>> LOG = True  # toggle debug logging
>>> if LOG:
...     import logging
...     # logger = logging.getLogger("web3.providers.AsyncIPCProvider")  # for the AsyncIPCProvider
...     logger = logging.getLogger("web3.providers.WebSocketProvider")  # for the WebSocketProvider
...     logger.setLevel(logging.DEBUG)
...     logger.addHandler(logging.StreamHandler())

>>> async def context_manager_subscriptions_example():
...     #  async with AsyncWeb3(AsyncIPCProvider("./path/to.filename.ipc") as w3:  # for the AsyncIPCProvider
...     async with AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546")) as w3:  # for the WebSocketProvider
...         # subscribe to new block headers
...         subscription_id = await w3.eth.subscribe("newHeads")
...
...         async for response in w3.socket.process_subscriptions():
...             print(f"{response}\n")
...             # handle responses here
...
...             if some_condition:
...                 # unsubscribe from new block headers and break out of
...                 # iterator
...                 await w3.eth.unsubscribe(subscription_id)
...                 break
...
...         # still an open connection, make any other requests and get
...         # responses via send / receive
...         latest_block = await w3.eth.get_block("latest")
...         print(f"Latest block: {latest_block}")
...
...         # the connection closes automatically when exiting the context
...         # manager (the `async with` block)

>>> asyncio.run(context_manager_subscription_example())

The AsyncWeb3 class may also be used as an asynchronous iterator, utilizing the async for syntax, when instantiating with a ~web3.providers.persistent.PersistentConnectionProvider. This may be used to set up an indefinite websocket connection and reconnect automatically if the connection is lost.

A similar example using a websockets connection as an asynchronous iterator can be found in the websockets connection docs.

>>> import asyncio
>>> import websockets
>>> from web3 import AsyncWeb3
>>> from web3.providers.persistent import (
...     AsyncIPCProvider,
...     WebSocketProvider,
... )

>>> async def subscription_iterator_example():
...     # async for w3 in AsyncWeb3(AsyncIPCProvider("./path/to/filename.ipc")):  # for the AsyncIPCProvider
...     async for w3 in AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546")):  # for the WebSocketProvider
...         try:
...             ...
...         except websockets.ConnectionClosed:
...             continue

# run the example
>>> asyncio.run(subscription_iterator_example())

Awaiting the instantiation with a ~web3.providers.persistent.PersistentConnectionProvider, or instantiating and awaiting the connect() method is also possible. Both of these examples are shown below.

>>> async def await_instantiation_example():
...     # w3 = await AsyncWeb3(AsyncIPCProvider("./path/to/filename.ipc"))  # for the AsyncIPCProvider
...     w3 = await AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546"))  # for the WebSocketProvider
...
...     # some code here
...
...     # manual cleanup
...     await w3.provider.disconnect()

# run the example
>>> asyncio.run(await_instantiation_example)
>>> async def await_provider_connect_example():
...     # w3 = AsyncWeb3(AsyncIPCProvider("./path/to/filename.ipc"))  # for the AsyncIPCProvider
...     w3 = AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546"))  # for the WebSocketProvider
...     await w3.provider.connect()
...
...     # some code here
...
...     # manual cleanup
...     await w3.provider.disconnect()

# run the example
>>> asyncio.run(await_provider_connect_example)

~web3.providers.persistent.PersistentConnectionProvider classes use the ~web3.providers.persistent.request_processor.RequestProcessor class under the hood to sync up the receiving of responses and response processing for one-to-one and one-to-many request-to-response requests. Refer to the ~web3.providers.persistent.request_processor.RequestProcessor documentation for details.

AsyncWeb3 with Persistent Connection Providers

When an AsyncWeb3 class is connected to a ~web3.providers.persistent.PersistentConnectionProvider, some attributes and methods become available.

Interacting with the Persistent Connection

LegacyWebSocketProvider

Warning

LegacyWebSocketProvider is deprecated and is likely to be removed in a future major release. Please use WebSocketProvider instead.

AutoProvider

~web3.providers.auto.AutoProvider is the default used when initializing web3.Web3 without any providers. There's rarely a reason to use it explicitly.

EthereumTesterProvider

Warning

Experimental: This provider is experimental. There are still significant gaps in functionality. However it is being actively developed and supported.

Note

To install the needed dependencies to use EthereumTesterProvider, you can install the pip extras package that has the correct interoperable versions of the eth-tester and py-evm dependencies needed to do testing: e.g. pip install web3[tester]