Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/tutorials/abn/images/report.html.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
32 changes: 31 additions & 1 deletion docs/tutorials/abn/rewards.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ kubectl create deploy prometheus-mock \
--port 9090 \
-- mockoon-cli start --daemon-off \
--port 9090 \
--data https://raw.githubusercontent.com/kalantar/docs/rewards/samples/abn/model-prometheus-abn-tutorial.json
--data https://raw.githubusercontent.com/iter8-tools/docs/v0.14.6/samples/abn/model-prometheus-abn-tutorial.json
kubectl expose deploy prometheus-mock --port 9090
```

Expand Down Expand Up @@ -91,10 +91,40 @@ This experiment executes in a [loop](../../user-guide/topics/parameters.md), onc
```shell
iter8 k report
```
??? note "The text report looks like this"
```shell
Experiment summary:
*******************

Experiment completed: true
No task failures: true
Total number of tasks: 2
Number of completed tasks: 2
Number of completed loops: 2

Whether or not service level objectives (SLOs) are satisfied:
*************************************************************

SLO Conditions | version 0 | version 1
-------------- | --------- | ---------
model-prometheus/latency-mean <= 50 | true | true


Latest observed values for metrics:
***********************************

Metric | version 0 | version 1 | Best
------- | ----- | ----- | ----
model-prometheus/latency-mean | 19.00 | 43.00 | n/a
model-prometheus/profit-mean | 71.00 | 92.00 | version 1
```

=== "HTML"
```shell
iter8 k report -o html > report.html # view in a browser
```
??? note "The HTML report looks like this"
![HTML report](images/report.html.png)

Because the experiment loops, the reported results will change over time.

Expand Down
9 changes: 4 additions & 5 deletions docs/tutorials/integrations/kserve-mm/blue-green.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ metadata:
name: wisdom-0
labels:
app.kubernetes.io/name: wisdom
app.kubernetes.io/version: v1
app.kubernetes.io/version: v0
iter8.tools/watch: "true"
annotations:
serving.kserve.io/deploymentMode: ModelMesh
Expand Down Expand Up @@ -137,7 +137,7 @@ metadata:
name: wisdom-1
labels:
app.kubernetes.io/name: wisdom
app.kubernetes.io/version: v2
app.kubernetes.io/version: v1
iter8.tools/watch: "true"
annotations:
serving.kserve.io/deploymentMode: ModelMesh
Expand All @@ -152,7 +152,7 @@ EOF
```

??? note "About the candidate `InferenceService`"
The model name (`wisdom`) and version (`v2`) are recorded using the labels `app.kubernets.io/name` and `app.kubernets.io.version`.
The model name (`wisdom`) and version (`v1`) are recorded using the labels `app.kubernets.io/name` and `app.kubernets.io.version`.

In this tutorial, the model source (field `spec.predictor.model.storageUri`) is the same as for the primary version of the model. In a real world example, this would be different.

Expand Down Expand Up @@ -198,10 +198,9 @@ apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: wisdom-0
namespace: modelmesh-serving
labels:
app.kubernetes.io/name: wisdom
app.kubernetes.io/version: v2
app.kubernetes.io/version: v1
iter8.tools/watch: "true"
annotations:
serving.kserve.io/deploymentMode: ModelMesh
Expand Down
11 changes: 4 additions & 7 deletions docs/tutorials/integrations/kserve-mm/canary.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,9 @@ apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: wisdom-0
namespace: modelmesh-serving
labels:
app.kubernetes.io/name: wisdom
app.kubernetes.io/version: v1
app.kubernetes.io/version: v0
iter8.tools/watch: "true"
annotations:
serving.kserve.io/deploymentMode: ModelMesh
Expand Down Expand Up @@ -150,10 +149,9 @@ apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: wisdom-1
namespace: modelmesh-serving
labels:
app.kubernetes.io/name: wisdom
app.kubernetes.io/version: v2
app.kubernetes.io/version: v1
iter8.tools/watch: "true"
annotations:
serving.kserve.io/deploymentMode: ModelMesh
Expand All @@ -168,7 +166,7 @@ EOF
```

??? note "About the candidate `InferenceService`"
The model name (`wisdom`) and version (`v2`) are recorded using the labels `app.kubernets.io/name` and `app.kubernets.io.version`.
The model name (`wisdom`) and version (`v1`) are recorded using the labels `app.kubernets.io/name` and `app.kubernets.io.version`.

In this tutorial, the model source (field `spec.predictor.model.storageUri`) is the same as for the primary version of the model. In a real example, this would be different.

Expand All @@ -194,10 +192,9 @@ apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: wisdom-0
namespace: modelmesh-serving
labels:
app.kubernetes.io/name: wisdom
app.kubernetes.io/version: v2
app.kubernetes.io/version: v1
iter8.tools/watch: "true"
annotations:
serving.kserve.io/deploymentMode: ModelMesh
Expand Down
Binary file modified docs/tutorials/integrations/kserve-mm/images/blue-green.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/tutorials/integrations/kserve-mm/images/canary.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/tutorials/integrations/kserve-mm/images/mirror.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading