Add aqua cli documentation #420
Conversation
oracle-contributor-agreement
bot
added
the
OCA Verified
ai-quick-actions/cli.md
Outdated
|
|
||
| ### Description | ||
|
|
||
| Creates a new evaluation model using an existing Aqua model or model deployment. Currently, evaluation is only supported via model deployment as the source. |
There was a problem hiding this comment.
I think we should remove any mention of the supporting evaluating offline models. Currently we only support deployed models.
There was a problem hiding this comment.
changed to Creates a new evaluation model using an existing Aqua model deployment.
ai-quick-actions/cli.md
Outdated
|
|
||
| `--evaluation_source_id [str]` | ||
|
|
||
| The evaluation source id. Must be either model or model deployment ocid. |
ai-quick-actions/cli.md
Outdated
|
|
||
| `--dataset_path [str]` | ||
|
|
||
| The dataset path for the evaluation. Could be either a local path from notebook session or an object storage path. |
There was a problem hiding this comment.
Let's not mention the local path maybe at all in the CLI?
|
|
||
| The description of the evaluation. Defaults to None. | ||
|
|
||
| `--memory_in_gbs [float]` |
There was a problem hiding this comment.
This is only for the flexible shapes. Despite we only support flexible shapes for now, in future we will return the other shapes as well.
|
|
||
| ### Example | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
We also added supporting the rouge metric.
--metrics '[{"name": "bertscore", "args": {}}, {"name": "rouge", "args": {}}]
ai-quick-actions/cli.md
Outdated
| "frequency_penalty": 0, | ||
| "stop": [], | ||
| "shape": "VM.Standard.E3.Flex", | ||
| "dataset_path": "oci://aqua-eval-test-bucket@namespace/dataset/evaluation-sample-with-sys-message.jsonl", |
There was a problem hiding this comment.
I guess the real path should be removed?
There was a problem hiding this comment.
yea this is a made-up path as well, but I'll update it to oci://<bucket>@<namespace>/path/to/the/dataset.jsonl
ai-quick-actions/cli.md
Outdated
|
|
||
| `--report_path [str]` | ||
|
|
||
| The report path for the evaluation. Must be an object storage path. |
There was a problem hiding this comment.
Would it make sense to add supported values for every lag just below the description?
--report_path [str]
The evaluation's dataset path, which must be an Object Storage path.
Example: oci://<bucket>@<namespace>/path/to/the/dataset.jsonl
There was a problem hiding this comment.
make sense, I'll some details in the description section for each param. Maybe not for all params that are easily understood or used across oci, but the ones we've introduced.
3 participants
This PR adds documentation along with examples on how users can access Aqua resources via ads CLI.