Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: add docs and remove connect function #596

Merged
merged 5 commits into from
Nov 4, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Add support for CSV files to the `fit` function. [#592](https://github.com/jina-ai/finetuner/pull/592)
- Add support for CSV files to the `fit` function. ([#592](https://github.com/jina-ai/finetuner/pull/592))

### Removed

- Remove `connect` function. ([#596](https://github.com/jina-ai/finetuner/pull/596))

### Changed

- Enhance documentation of login functionalities. ([#596](https://github.com/jina-ai/finetuner/pull/596))

### Fixed

- Fix links to functions in the documentation. ([#596](https://github.com/jina-ai/finetuner/pull/596))

### Docs


Expand Down
2 changes: 1 addition & 1 deletion docs/get-started/how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ From an engineering perspective,
we have hidden all the complexity of machine learning algorithms and resource configuration (such as GPUs).
All you need to do is decide on your backbone model and prepare your training data.

Once you have logged in to the Jina Ecosystem with {meth}`~finetuner.login()` or `~finetuner.notebook_login()`,
Once you have logged in to the Jina Ecosystem with {meth}`~finetuner.login()` or {func}`~finetuner.notebook_login()`,
Finetuner will push your training data into our *Cloud Artifact Storage* (only visible to you).
At the same time, we will spin-up an isolated computational resource
with proper memory, CPU, GPU dedicated to your fine-tuning job.
Expand Down
6 changes: 3 additions & 3 deletions docs/notebooks/image_to_image.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@
"but recommended in order to retrieve your run easily and have some context about it.\n",
"* Furthermore, we had to provide names of the `train_data`.\n",
"* We set `TripletMarginLoss`.\n",
"* Additionally, we use {class}`~finetuner.callback.EvaluationCallback` for evaluation.\n",
"* Additionally, we use `finetuner.callback.EvaluationCallback` for evaluation.\n",
"* Lastly, we set the number of `epochs` and provide a `learning_rate`."
],
"metadata": {
Expand Down Expand Up @@ -224,8 +224,8 @@
"cell_type": "markdown",
"source": [
"## Evaluating\n",
"Currently, we don't have a user-friendly way to get evaluation metrics from the `~finetuner.callback.EvaluationCallback` we initialized previously.\n",
"What you can do for now is to call `~finetuner.run.Run.logs()` in the end of the run and see evaluation results:\n",
"Currently, we don't have a user-friendly way to get evaluation metrics from the `finetuner.callback.EvaluationCallback` we initialized previously.\n",
"What you can do for now is to call `run.logs()` in the end of the run and see evaluation results:\n",
"\n",
"```bash\n",
" Training [5/5] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76/76 0:00:00 0:03:15 • loss: 0.003\n",
Expand Down
6 changes: 3 additions & 3 deletions docs/notebooks/image_to_image.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ Let's understand what this piece of code does:
but recommended in order to retrieve your run easily and have some context about it.
* Furthermore, we had to provide names of the `train_data`.
* We set `TripletMarginLoss`.
* Additionally, we use {class}`~finetuner.callback.EvaluationCallback` for evaluation.
* Additionally, we use `finetuner.callback.EvaluationCallback` for evaluation.
* Lastly, we set the number of `epochs` and provide a `learning_rate`.
<!-- #endregion -->

Expand Down Expand Up @@ -142,8 +142,8 @@ You can continue monitoring the runs by checking the status - `finetuner.run.Run

<!-- #region id="BMpQxydypeZ3" -->
## Evaluating
Currently, we don't have a user-friendly way to get evaluation metrics from the `~finetuner.callback.EvaluationCallback` we initialized previously.
What you can do for now is to call `~finetuner.run.Run.logs()` in the end of the run and see evaluation results:
Currently, we don't have a user-friendly way to get evaluation metrics from the `finetuner.callback.EvaluationCallback` we initialized previously.
What you can do for now is to call `run.logs()` in the end of the run and see evaluation results:

```bash
Training [5/5] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76/76 0:00:00 0:03:15 • loss: 0.003
Expand Down
5 changes: 3 additions & 2 deletions docs/notebooks/text_to_image.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@
},
"language_info": {
"name": "python"
}
},
"accelerator": "GPU"
},
"cells": [
{
Expand Down Expand Up @@ -280,7 +281,7 @@
"cell_type": "code",
"source": [
"text_da = DocumentArray([Document(text='some text to encode')])\n",
"image_da = DocumentArray([Document(uri='my-image.png')])\n",
"image_da = DocumentArray([Document(uri='https://www.collinsdictionary.com/images/full/apple_158989157.jpg')])\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a free-use image. Try to use free-use images with known origins and stable URLs. Like this one: https://upload.wikimedia.org/wikipedia/commons/4/4e/Single_apple.png

"\n",
"clip_text_encoder = finetuner.get_model(artifact=artifact, select_model='clip-text')\n",
"clip_image_encoder = finetuner.get_model(artifact=artifact, select_model='clip-vision')\n",
Expand Down
2 changes: 1 addition & 1 deletion docs/notebooks/text_to_image.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ let's use the fine-tuned model to encode a new `Document`:

```python id="v95QsuEyzE-B"
text_da = DocumentArray([Document(text='some text to encode')])
image_da = DocumentArray([Document(uri='my-image.png')])
image_da = DocumentArray([Document(uri='https://www.collinsdictionary.com/images/full/apple_158989157.jpg')])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a free-use image. Try to use free-use images with known origins and stable URLs. Like this one: https://upload.wikimedia.org/wikipedia/commons/4/4e/Single_apple.png


clip_text_encoder = finetuner.get_model(artifact=artifact, select_model='clip-text')
clip_image_encoder = finetuner.get_model(artifact=artifact, select_model='clip-vision')
Expand Down
6 changes: 3 additions & 3 deletions docs/walkthrough/integrate-with-jina.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ It is worth noting that, while training data can be provided as a csv, data for
(integrate-with-docarray)=
## Embed DocumentArray

To embed a [DocumentArray](https://docarray.jina.ai/) with a fine-tuned model, you can get the model of your Run via the `get_model` function and embed it via the `encode` function:
To embed a [DocumentArray](https://docarray.jina.ai/) with a fine-tuned model, you can get the model of your Run via the {func}`~finetuner.get_model` function and embed it via the {func}`finetuner.encode` function:

````{tab} Artifact id and token
```python
Expand Down Expand Up @@ -121,7 +121,7 @@ executors:
````

As you can see, it's super easy!
If you did not call `save_artifact`,
If you did not call {func}`~finetuner.run.Run.save_artifact`,
you need to provide the `artifact_id` and `token`.
`FinetunerExecutor` will automatically pull your model from the Jina AI Cloud to the container.

Expand Down Expand Up @@ -169,7 +169,7 @@ models: `clip-vision` and `clip-text`.
The vision model allows you to embed images and the text model can encode text passages
into the same vector space.
To use those models, you have to provide the name of the model via an additional
`select_model` parameter to the `get_model` function.
`select_model` parameter to the {func}`~finetuner.get_model` function.


````{tab} CLIP text model
Expand Down
11 changes: 9 additions & 2 deletions docs/walkthrough/login.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Login

Since Finetuner leverages cloud resources for fine-tuning,
you are required to {meth}`~finetuner.login()` (or `~finetuner.notebook_login()`) and obtain a token from Jina before starting a fine-tuning job.
you are required to {meth}`~finetuner.login()` (or {func}`~finetuner.notebook_login()`) and obtain a token from Jina before starting a fine-tuning job.
It is as simple as:

```python
Expand All @@ -12,10 +12,17 @@ finetuner.login() # use finetuner.notebook_login() in Jupyter notebook or Googl
```

A browser window should pop up with different login options.
After {meth}`~finetuner.login()` or `~finetuner.notebook_login()` you will see this in your terminal:
After {meth}`~finetuner.login()` or {func}`~finetuner.notebook_login()` you will see this in your terminal:

```bash
🔐 Successfully logged in to Jina AI as [USER NAME]!
```

Now, an authentication token is generated which can be read with the {func}`~finetuner.get_token` function.
If you have been logged in before, the existing token will not be overwritten, however, if you want this to be happen, you can set the `force` attribute in the login function to true.

```
finetuner.login(force=True)
```

```{admonition} Why do I need to login?
Expand Down
13 changes: 8 additions & 5 deletions finetuner/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,21 @@
ft = Finetuner()


def login(force: bool = False):
def login(force: bool = False) -> None:
"""
Login to Jina AI to use cloud-based fine-tuning. Thereby, a authentication token is
guenthermi marked this conversation as resolved.
Show resolved Hide resolved
generated which can be read with the :func:`~finetuner.get_token` function.

:param force: If set to true, an existing token will be overwritten. Otherwise,
you will not login again, if a valid token already exists.
"""
ft.login(force=force)


def notebook_login(force: bool = False):
ft.notebook_login(force=force)


def connect():
ft.connect()


def list_callbacks() -> Dict[str, callback.CallbackStubType]:
"""List available callbacks."""
return {
Expand Down
6 changes: 0 additions & 6 deletions finetuner/finetuner.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,6 @@ def notebook_login(self, force: bool = False):
"""
hubble.notebook_login(force=force, post_success=self._init_state)

def connect(self):
"""Connects finetuner to Hubble without logging in again.
Use this function, if you are already logged in.
"""
self._init_state()

@staticmethod
def _get_cwd() -> str:
"""Returns current working directory."""
Expand Down