Skip to content

Commit

Permalink
docs: add binder snippet (#1673)
Browse files Browse the repository at this point in the history
* docs: add binder snippet
  • Loading branch information
hanxiao committed Jan 12, 2021
1 parent d40aee5 commit 7b60b8e
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 4 deletions.
16 changes: 12 additions & 4 deletions README.md
Expand Up @@ -102,6 +102,7 @@ jina hello-world --help
| 🚀 | [Customize Encoder](#customize-encoder)[Test Encoder in Flow](#test-encoder-in-flow)[Parallelism & Batching](#parallelism--batching)[Add Data Indexer](#add-data-indexer)[Compose Flow from YAML](#compose-flow-from-yaml)[Search](#search)[Evaluation](#evaluation)[REST Interface](#rest-interface) |

#### Create
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-create-flow.ipynb)

Jina provides a high-level [Flow API](https://github.com/jina-ai/jina/tree/master/docs/chapters/101#flow) to simplify building search/index workflows. To create a new Flow:

Expand All @@ -113,6 +114,7 @@ f = Flow().add()
This creates a simple Flow with one [Pod](https://github.com/jina-ai/jina/tree/master/docs/chapters/101#pods). You can chain multiple `.add()`s in a single Flow.

#### Visualize
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-visualize-a-flow.ipynb)

To visualize the Flow, simply chain it with `.plot('my-flow.svg')`. If you are using a Jupyter notebook, the Flow object will be automatically displayed inline *without* `plot`:

Expand All @@ -121,6 +123,7 @@ To visualize the Flow, simply chain it with `.plot('my-flow.svg')`. If you are u
`Gateway` is the entrypoint of the Flow.

#### Feed Data
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-feed-data.ipynb)

Let's create some random data and index it:

Expand All @@ -138,6 +141,7 @@ with Flow().add() as f:
To use a Flow, open it using the `with` context manager, like you would a file in Python. You can call `index` and `search` with nearly all types of data. The whole data stream is asynchronous and efficient.

#### Fetch Result
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-fetch-result.ipynb)

Once a request is done, callback functions are fired. Jina Flow implements Promise-like interface, you can add callback functions `on_done`, `on_error`, `on_always` to hook different event. In the example below, our Flow passes the message then prints the result when success. If something wrong, it beeps. Finally, the result is written to `output.txt`.

Expand All @@ -154,6 +158,7 @@ with Flow().add() as f, open('output.txt', 'w') as fp:


#### Construct Document
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-construct-document.ipynb)

`Document` is [Jina's primitive data type](https://hanxiao.io/2020/11/22/Primitive-Data-Types-in-Neural-Search-System/#primitive-types). It can contain text, image, array, embedding, URI, and accompanied by rich meta information. It can be recurred both vertically and horizontally to have nested documents and matched documents. To construct a Document, one can use:

Expand Down Expand Up @@ -222,22 +227,24 @@ Interested readers can refer to [`jina-ai/example`: how to build a multimodal se
</details>

#### Add Logic
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-add-logic.ipynb)

To add logic to the Flow, use the `uses` parameter to attach a Pod with an [Executor](https://github.com/jina-ai/jina/tree/master/docs/chapters/101#executors). `uses` accepts multiple value types including class name, Docker image, (inline) YAML or built-in shortcut.


```python
f = (Flow().add(uses='MyBertEncoder') # class name of a Jina Executor
.add(uses='docker://jinahub/pretrained-cnn:latest') # the container name
.add(uses='myencoder.yaml') # YAML serialization of a Jina Executor
.add(uses='docker://jinahub/pretrained-cnn:latest') # the image name
.add(uses='myencoder.yml') # YAML serialization of a Jina Executor
.add(uses='!WaveletTransformer | {freq: 20}') # inline YAML config
.add(uses='_pass')) # built-in shortcut executor
.add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}}) # dict config object with __cls keyword
.add(uses='_pass') # built-in shortcut executor
.add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}})) # dict config object with __cls keyword
```

The power of Jina lies in its decentralized architecture: each `add` creates a new Pod, and these Pods can be run as a local thread/process, a remote process, inside a Docker container, or even inside a remote Docker container.

#### Inter & Intra Parallelism
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-inter-intra-parallelism.ipynb)

Chaining `.add()`s creates a sequential Flow. For parallelism, use the `needs` parameter:

Expand All @@ -262,6 +269,7 @@ f = (Flow().add(name='p1', needs='gateway')
<img src="https://github.com/jina-ai/jina/blob/master/.github/simple-plot4.svg?raw=true"/>

#### Asynchronous Flow
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jina-ai/jupyter-notebooks/main?filepath=basic-inter-intra-parallelism.ipynb)

Synchronous from outside, Jina runs asynchronously underneath: it manages the eventloop(s) for scheduling the jobs. In some scenario, user wants more control over the eventloop, then `AsyncFlow` comes to use. In the example below, Jina is part of the integration where another heavy-lifting job is running concurrently:

Expand Down
9 changes: 9 additions & 0 deletions jina/__init__.py
Expand Up @@ -161,5 +161,14 @@ def _set_nofile(nofile_atleast=4096):
from jina.clients import Client
from jina.clients.asyncio import AsyncClient

# Executor
from jina.executors.classifiers import BaseClassifier as Classifier
from jina.executors.crafters import BaseCrafter as Crafter
from jina.executors.encoders import BaseEncoder as Encoder
from jina.executors.evaluators import BaseEvaluator as Evaluator
from jina.executors.indexers import BaseIndexer as Indexer
from jina.executors.rankers import BaseRanker as Ranker
from jina.executors.segmenters import BaseSegmenter as Segmenter

__all__ = [_s for _s in dir() if not _s.startswith('_')]
__all__.extend([_s for _s in _names_with_underscore])

0 comments on commit 7b60b8e

Please sign in to comment.