[HaClient] Asyncua ha client #367
Conversation
It exposes an interface similar to Client. You'll find some examples
and tests to guide you through this new feature.
Under the hood, it starts a keepalive task to monitor the status and
service level for each server.
It also includes an HaManager task which promotes a primary and
reconnect unhealthy clients. Unhealthy clients are detected based
on the socket status and keepalive data collected.
The requests received by users as well as the HaManager decisions
produce an ideal state called the "ideal_map". Regularly, the reconciliator task
kicks in, lock this configuration, and applies the "ideal_map" configuration.
It then stores the actual subscriptions status into the "real_map" using the
lower level components of the library.
Current limits:
- We only support HA WARM mode and datachange notifications (no event/status_change).
_ We support multiple subscriptions, but a node should only be subscribed once.
Component details:
- HaClient:
- Configuration based on dataclasses: HaConfig (required) and HaConfigSecurity (optional)
- Responsible to start up sides tasks: keepalive, hamanager, reconciliator
- Mutates the ideal_map via VirtualSubscription
- Generic hooks (i.e: to monitor subscription performance)
- KeepAlive (task):
- Regularly hits the server to check its service_level / status.
- HaManager (task):
- Promote primary / secondaries
- Reconnect disconnected/unhealthy client based on the keepalive
feedback.
- Reconciliator (task):
- Applies the ideal configuration
- Perform checks on the call responses
- Mutates the real_map
- VirtualSubscription:
- Key component of the ideal and real maps.
- Exposes an interface similar to the subscription but only
store the settings.
|
Thanks. Obviously you have done a lot of work and the code looks very clean. |
ZuZuD
force-pushed
the
ha_client_asyncua_latest
branch
3 times, most recently
from
63abfa7
to
dacdd1d
Compare
CancelledError and TimeoutError have moved from concurrent.futures to asyncio. Note that CancelledError now inherits from BaseException and not Exception anymore. See https://bugs.python.org/issue32528 for the details. Finally, fix pytest_yield_fixture deprecation
ZuZuD
force-pushed
the
ha_client_asyncua_latest
branch
from
dacdd1d
to
c91d68a
Compare
Thanks for your honest feedback @oroulet, and I totally understand your point. To give you more context, I currently use this HaClient in production with more than 20 OPC-UA "clusters". This is just to highlight that this is more than a PoC, and that I'll likely stay an active contributor should you merge this. I may also know other contributors that are interested or already testing this feature. I will check with them if they can share their experience here to build up more support. |
|
I closed it by mistake ;-) |
| class KeepAlive: | ||
| """ | ||
| Ping the server status regularly to check its health | ||
| """ |
There was a problem hiding this comment.
Isnt't that KeepAlive clas a (better) duplicate of the KeepAlive class in client.py? Could we merge them?
There was a problem hiding this comment.
There was a KeepAlive thread in python-opcua which was replaced by an async task. Since its role is to renew the secure_channel, its frequency depends on the session_timeout and secure_channel_timeout. However, users of the HaClient could want to check the server status more frequently. Keeping those tasks apart feels like the right SoC.
asyncua/client/ha/reconciliator.py
Outdated
| await asyncio.gather(*tasks_add, return_exceptions=True) | ||
| await asyncio.gather(*tasks_del, return_exceptions=True) | ||
|
|
||
| async def update_nodes(self, real_map, ideal_map, targets: Set[str]): |
There was a problem hiding this comment.
several methods in that class are too long, they will be hard to maintain
There was a problem hiding this comment.
Refactored in the following commit.
ZuZuD
force-pushed
the
ha_client_asyncua_latest
branch
from
4733189
to
c0d250b
Compare
* [HaClient] Asyncua client wrapper
It exposes an interface similar to Client. You'll find some examples
and tests to guide you through this new feature.
Under the hood, it starts a keepalive task to monitor the status and
service level for each server.
It also includes an HaManager task which promotes a primary and
reconnect unhealthy clients. Unhealthy clients are detected based
on the socket status and keepalive data collected.
The requests received by users as well as the HaManager decisions
produce an ideal state called the "ideal_map". Regularly, the reconciliator task
kicks in, lock this configuration, and applies the "ideal_map" configuration.
It then stores the actual subscriptions status into the "real_map" using the
lower level components of the library.
Current limits:
- We only support HA WARM mode and datachange notifications (no event/status_change).
_ We support multiple subscriptions, but a node should only be subscribed once.
Component details:
- HaClient:
- Configuration based on dataclasses: HaConfig (required) and HaConfigSecurity (optional)
- Responsible to start up sides tasks: keepalive, hamanager, reconciliator
- Mutates the ideal_map via VirtualSubscription
- Generic hooks (i.e: to monitor subscription performance)
- KeepAlive (task):
- Regularly hits the server to check its service_level / status.
- HaManager (task):
- Promote primary / secondaries
- Reconnect disconnected/unhealthy client based on the keepalive
feedback.
- Reconciliator (task):
- Applies the ideal configuration
- Perform checks on the call responses
- Mutates the real_map
- VirtualSubscription:
- Key component of the ideal and real maps.
- Exposes an interface similar to the subscription but only
store the settings.
* [HaClient] fix test for py3.8/py3.9
CancelledError and TimeoutError have moved from concurrent.futures to
asyncio. Note that CancelledError now inherits from BaseException and
not Exception anymore. See https://bugs.python.org/issue32528 for the
details.
Finally, fix pytest_yield_fixture deprecation
* [HaClient] address the comments (utils + refacto reconciliator)
Co-authored-by: oroulet <oroulet@users.noreply.github.com>
See discussion #314.
This adds a Client interface named HaClient which supports two or more servers for high availability purposes.
You'll find some examples and tests to guide you through this new feature.
Under the hood, it starts a keepalive task to monitor the status and service level for each server.
It also includes an HaManager task which promotes a primary and reconnect unhealthy clients. Unhealthy clients are detected based on the socket status and keepalive data collected.
The requests received by users as well as the HaManager decisions produce an ideal state called the "ideal_map". Regularly, the reconciliator task kicks in, lock this configuration, and applies the "ideal_map" configuration.
It then stores the actual subscriptions status into the "real_map" using the lower level components of the library.
Current limits:
Component details:
HaClient:
KeepAlive (task):
HaManager (task):
feedback.
Reconciliator (task):
VirtualSubscription:
store the settings.