Skip to content

Commit

Permalink
refactor: merge dryrun into ping (#5151)
Browse files Browse the repository at this point in the history 
* refactor: merge dryrun into ping

* style: fix overload and cli autocomplete

* refactor: merge dryrun into ping

* refactor: merge dryrun into ping

* refactor: merge dryrun into ping

* refactor: merge dryrun into ping

Co-authored-by: Jina Dev Bot <dev-bot@jina.ai>
  • Loading branch information
@hanxiao @jina-bot
hanxiao and jina-bot committed Sep 8, 2022
1 parent 0a71009 commit 273fda5
Show file tree
Hide file tree
Showing 21 changed files with 512 additions and 272 deletions.
449 changes: 340 additions & 109 deletions docs/fundamentals/client/client.md
View file

Large diffs are not rendered by default.

97 changes: 71 additions & 26 deletions docs/fundamentals/flow/health-check.md
View file
Original file line number Diff line number Diff line change
@@ -1,29 +1,32 @@
# Readiness & health check
Every Jina {class}`~jina.Flow` consists of a {ref}`number of microservices <architecture-overview>`,
A Jina {class}`~jina.Flow` consists of {ref}`a Gateway and Executors<architecture-overview>`,
each of which have to be healthy before the Flow is ready to receive requests.

Each Flow microservice provides a health check in the form of a [standardized gRPC endpoint](https://github.com/grpc/grpc/blob/master/doc/health-checking.md) that exposes this information to the outside world.
A Flow is marked as "ready", when all its Executors and its Gateway are fully loaded and ready.

Each Executor provides a health check in the form of a [standardized gRPC endpoint](https://github.com/grpc/grpc/blob/master/doc/health-checking.md) that exposes this information to the outside world.
This means that health checks can automatically be performed by Jina itself as well as external tools like Docker Compose, Kubernetes service meshes, or load balancers.


## Readiness of a Flow

In most cases, it is most useful to check if an entire Flow is ready to accept requests.
To enable this readiness check, the Jina Gateway can aggregate health check information from all services and provides
a readiness check endpoint for the complete Flow.

## Readiness of complete Flow

A lot of times, it is useful to know if a Flow, as a complete set of microservices, is ready to receive requests. This is why the Gateway
exposes an endpoint for each of the supported protocols to know the health and readiness of the entire Flow.
<!-- start flow-ready -->

Jina {class}`~jina.Flow` and {class}`~jina.Client` offer a convenient API to query these readiness endpoints. You can call `flow.dry_run()` or `client.dry_run()`, which will return `True` if the Flow is healthy and ready, and `False` otherwise:
{class}`~jina.Client` offer a convenient API to query these readiness endpoints. You can call {meth}`~jina.clients.mixin.HealthCheckMixin.is_flow_ready` or {meth}`~jina.Flow.is_flow_ready`, it will return `True` if the Flow is ready, and `False` when it is not.

````{tab} via Flow
```python
from jina import Flow
with Flow().add() as f:
print(f.dry_run())
print(f.is_flow_ready())
print(f.dry_run())
print(f.is_flow_ready())
```
```text
True
Expand All @@ -41,28 +44,75 @@ with Flow(port=12345).add() as f:
from jina import Client
client = Client(port=12345)
print(client.dry_run())
print(client.is_flow_ready())
```
```text
True
```
````

````{tab} via CLI
`````{tab} via CLI
```python
from jina import Flow
with Flow(port=12345).add() as f:
f.block()
```
```bash
jina dryrun grpc://localhost:12345
jina ping flow grpc://localhost:12345
```
````{tab} Success
```text
dry run successful
INFO JINA@92877 ping grpc://localhost:12345 at 0 round... [09/08/22 12:58:13]
INFO JINA@92877 ping grpc://localhost:12345 at 0 round takes 0 seconds (0.04s)
INFO JINA@92877 ping grpc://localhost:12345 at 1 round... [09/08/22 12:58:14]
INFO JINA@92877 ping grpc://localhost:12345 at 1 round takes 0 seconds (0.01s)
INFO JINA@92877 ping grpc://localhost:12345 at 2 round... [09/08/22 12:58:15]
INFO JINA@92877 ping grpc://localhost:12345 at 2 round takes 0 seconds (0.01s)
INFO JINA@92877 avg. latency: 24 ms [09/08/22 12:58:16]
```
````
````{tab} Failure
```text
INFO JINA@92986 ping grpc://localhost:12345 at 0 round... [09/08/22 12:59:00]
ERROR GRPCClient@92986 Error while getting response from grpc server <AioRpcError of RPC that terminated with: [09/08/22 12:59:00]
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused"
debug_error_string = "UNKNOWN:Failed to pick subchannel {created_time:"2022-09-08T12:59:00.518707+02:00", children:[UNKNOWN:failed to
connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused {grpc_status:14,
created_time:"2022-09-08T12:59:00.518706+02:00"}]}"
>
WARNI… JINA@92986 not responding, retry (1/3) in 1s
INFO JINA@92986 ping grpc://localhost:12345 at 0 round takes 0 seconds (0.01s)
INFO JINA@92986 ping grpc://localhost:12345 at 1 round... [09/08/22 12:59:01]
ERROR GRPCClient@92986 Error while getting response from grpc server <AioRpcError of RPC that terminated with: [09/08/22 12:59:01]
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused"
debug_error_string = "UNKNOWN:Failed to pick subchannel {created_time:"2022-09-08T12:59:01.537293+02:00", children:[UNKNOWN:failed to
connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused {grpc_status:14,
created_time:"2022-09-08T12:59:01.537291+02:00"}]}"
>
WARNI… JINA@92986 not responding, retry (2/3) in 1s
INFO JINA@92986 ping grpc://localhost:12345 at 1 round takes 0 seconds (0.01s)
INFO JINA@92986 ping grpc://localhost:12345 at 2 round... [09/08/22 12:59:02]
ERROR GRPCClient@92986 Error while getting response from grpc server <AioRpcError of RPC that terminated with: [09/08/22 12:59:02]
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused"
debug_error_string = "UNKNOWN:Failed to pick subchannel {created_time:"2022-09-08T12:59:02.557195+02:00", children:[UNKNOWN:failed to
connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused {grpc_status:14,
created_time:"2022-09-08T12:59:02.557193+02:00"}]}"
>
WARNI… JINA@92986 not responding, retry (3/3) in 1s
INFO JINA@92986 ping grpc://localhost:12345 at 2 round takes 0 seconds (0.02s)
WARNI… JINA@92986 message lost 100% (3/3)
```
````
`````

<!-- end flow-ready -->

### Flow status using third-party clients

You can check the status of a Flow using any gRPC/HTTP/Websocket client, not just Jina's Client implementation.
Expand Down Expand Up @@ -185,31 +235,26 @@ Then by doing the same check, you will see that the call returns an error:
```

(health-check-microservices)=
## Health check of individual microservices
## Health check of an Executor

In addition to a performing a readiness check for the entire Flow, it is also possible to check every individual microservice in said Flow,
In addition to a performing a readiness check for the entire Flow, it is also possible to check every individual Executor in said Flow,
by utilizing a [standardized gRPC health check endpoint](https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
In most cases this is not necessary, since such checks are performed by Jina, a Kubernetes service mesh or a load balancer under the hood.
Nevertheless, it is possible to perform these checks as a user.

When performing these checks, you can expect on of the following `ServingStatus` responses:
- **`UNKNOWN` (0)**: The health of the microservice could not be determined
- **`SERVING` (1)**: The microservice is healthy and ready to receive requests
- **`NOT_SERVING` (2)**: The microservice is *not* healthy and *not* ready to receive requests
- **`SERVICE_UNKNOWN` (3)**: The health of the microservice could not be determined while performing streaming
- **`UNKNOWN` (0)**: The health of the Executor could not be determined
- **`SERVING` (1)**: The Executor is healthy and ready to receive requests
- **`NOT_SERVING` (2)**: The Executor is *not* healthy and *not* ready to receive requests
- **`SERVICE_UNKNOWN` (3)**: The health of the Executor could not be determined while performing streaming

````{admonition} See Also
:class: seealso
To learn more about these status codes, and how health checks are performed with gRPC, see [here](https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
````

(health-check-executor)=
### Health check of an Executor

{class}`~jina.Executor`s run as microservices exposing gRPC endpoints, and they expose one endpoint for a health and readiness check.

To see how to use it, you can start a Flow inside a terminal and block it to accept requests:
You can start a Flow inside a terminal and block it to accept requests:

```python
from jina import Flow
Expand All @@ -233,9 +278,9 @@ docker run --network='host' fullstorydev/grpcurl -plaintext 127.0.0.1:12346 grpc
```

(health-check-gateway)=
### Health check of the Gateway
## Health check of the Gateway

Just like each individual Executor, the Gateway also acts as a microservice, and as such it exposes a health check endpoint.
Just like each individual Executor, the Gateway also exposes a health check endpoint.

In contrast to Executors however, a Gateway can use gRPC, HTTP, or Websocket, and the health check endpoint changes accordingly.

Expand Down
24 changes: 12 additions & 12 deletions docs/fundamentals/flow/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,18 @@ In particular, each Flow also launches a *Gateway* service, which can expose all

The most important methods of the `Flow` object are the following:

| Method | Description |
|-------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------|
| {meth}`~jina.Flow.add` | Add an Executor to the Flow |
| {meth}`~jina.Flow.start()` | Starts the Flow. This will start all its Executors and check if they are ready to be used. |
| {meth}`~jina.Flow.close()` | Stops and closes the Flow. This will stop and shutdown all its Executors. |
| `with` context manager | Use the Flow as a context manager. It will automatically start and stop your Flow. | |
| {meth}`~jina.Flow.plot()` | Visualizes the Flow. Helpful for building complex pipelines. |
| {meth}`~jina.clients.mixin.PostMixin.post()` | Sends requests to the Flow API. |
| {meth}`~jina.Flow.block()` | Blocks execution until the program is terminated. This is useful to keep the Flow alive so it can be used from other places (clients, etc). |
| {meth}`~jina.Flow.to_docker_compose_yaml()` | Generates a Docker-Compose file listing all its Executors as Services. |
| {meth}`~jina.Flow.to_kubernetes_yaml()` | Generates the Kubernetes configuration files in `<output_directory>`. Based on your local Jina version, Jina Hub may rebuild the Docker image during the YAML generation process. If you do not wish to rebuild the image, set the environment variable `JINA_HUB_NO_IMAGE_REBUILD`. |
| {meth}`~jina.clients.mixin.HealthCheckMixin.dry_run()` | Calls the dry run endpoint of the Flow to check if the Flow is ready to process requests. Returns a boolean indicating the readiness |
| Method | Description |
|--------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| {meth}`~jina.Flow.add` | Add an Executor to the Flow |
| {meth}`~jina.Flow.start()` | Starts the Flow. This will start all its Executors and check if they are ready to be used. |
| {meth}`~jina.Flow.close()` | Stops and closes the Flow. This will stop and shutdown all its Executors. |
| `with` context manager | Use the Flow as a context manager. It will automatically start and stop your Flow. | |
| {meth}`~jina.Flow.plot()` | Visualizes the Flow. Helpful for building complex pipelines. |
| {meth}`~jina.clients.mixin.PostMixin.post()` | Sends requests to the Flow API. |
| {meth}`~jina.Flow.block()` | Blocks execution until the program is terminated. This is useful to keep the Flow alive so it can be used from other places (clients, etc). |
| {meth}`~jina.Flow.to_docker_compose_yaml()` | Generates a Docker-Compose file listing all its Executors as Services. |
| {meth}`~jina.Flow.to_kubernetes_yaml()` | Generates the Kubernetes configuration files in `<output_directory>`. Based on your local Jina version, Jina Hub may rebuild the Docker image during the YAML generation process. If you do not wish to rebuild the image, set the environment variable `JINA_HUB_NO_IMAGE_REBUILD`. |
| {meth}`~jina.clients.mixin.HealthCheckMixin.is_flow_ready()` | Check if the Flow is ready to process requests. Returns a boolean indicating the readiness |

## Why should you use a Flow?

Expand Down
34 changes: 16 additions & 18 deletions docs/get-started/migrate.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,14 @@ docs = nested_docs()
print(docs.traverse_flat('r,c').texts)
```
```
>>> ['root1', 'rooot2', 'chunk11', 'chunk12', 'chunk21', 'chunk22']
```console
['root1', 'rooot2', 'chunk11', 'chunk12', 'chunk21', 'chunk22']
```
```python
print(docs.flatten().texts)
```
```
>>> ['chunk11', 'chunk12', 'root1', 'chunk21', 'chunk22', 'root2']
```console
['chunk11', 'chunk12', 'root1', 'chunk21', 'chunk22', 'root2']
```
````
Expand All @@ -111,15 +110,14 @@ docs = nested_docs()
print(docs['@r,c'].texts)
```
```
>>> ['root1', 'rooot2', 'chunk11', 'chunk12', 'chunk21', 'chunk22']
```console
['root1', 'rooot2', 'chunk11', 'chunk12', 'chunk21', 'chunk22']
```
```python
print(docs[...].texts)
```
```
>>> ['chunk11', 'chunk12', 'root1', 'chunk21', 'chunk22', 'root2']
```console
['chunk11', 'chunk12', 'root1', 'chunk21', 'chunk22', 'root2']
```
````
Expand Down Expand Up @@ -183,15 +181,15 @@ from jina import Document, DocumentArray
d = Document()
print(d.text)
```
```
>>> ''
```console
''
```
```python
docs = DocumentArray([d, d])
print(docs.texts)
```
```
>>> ['', '']
```console
['', '']
```
````
Expand All @@ -204,15 +202,15 @@ from docarray import Document, DocumentArray
d = Document()
print(d.text)
```
```
>>> ''
```console
''
```
```python
docs = DocumentArray([d, d])
print(docs.texts)
```
```
>>> None
```console
None
```
````
Expand Down
38 changes: 6 additions & 32 deletions jina/checker.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,21 @@ def __init__(self, args: 'argparse.Namespace'):

import time

from jina import Client
from jina.logging.profile import TimeContext
from jina.serve.runtimes.worker import WorkerRuntime

ctrl_addr = f'{args.host}:{args.port}'
try:
total_time = 0
total_success = 0
for j in range(args.retries):
with TimeContext(
f'ping {ctrl_addr} at {j} round', default_logger
f'ping {args.host} at {j} round', default_logger
) as tc:
r = WorkerRuntime.is_ready(ctrl_addr)
if args.target == 'executor':
r = WorkerRuntime.is_ready(args.host)
elif args.target == 'flow':
r = Client(host=args.host).is_flow_ready(timeout=args.timeout)
if not r:
default_logger.warning(
'not responding, retry (%d/%d) in 1s'
Expand Down Expand Up @@ -55,32 +58,3 @@ def __init__(self, args: 'argparse.Namespace'):

# returns 1 (anomaly) when it comes to here
exit(1)


def dry_run_checker(args: 'argparse.Namespace'):
"""
call dry run on the given endpoint
:param args: args provided by the CLI.
"""
# No retry mechanism for dry run since it is built in the Flow

from jina import Client

client = Client(host=args.host)

try:

if client.dry_run(timeout=args.timeout):
default_logger.info('dry run successful')
exit(0)
else:
default_logger.warning('dry run failed')
exit(1)

except KeyboardInterrupt:
pass

exit(1)


# returns 1 (anomaly) when it comes to here
Loading

0 comments on commit 273fda5

Please sign in to comment.