Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KServe CPU Nightly Test #2734

Merged
merged 43 commits into from
Oct 26, 2023
Merged

KServe CPU Nightly Test #2734

Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
031f6eb
kserve ci
agunapal Oct 23, 2023
dd2f429
kserve ci
agunapal Oct 23, 2023
c3c44b8
kserve ci
agunapal Oct 23, 2023
acf2c49
kserve ci
agunapal Oct 23, 2023
037a188
kserve ci
agunapal Oct 23, 2023
5d3bf11
kserve ci
agunapal Oct 23, 2023
464af5d
kserve ci
agunapal Oct 23, 2023
cfc8402
kserve ci
agunapal Oct 23, 2023
d330112
kserve ci
agunapal Oct 23, 2023
7c1ae15
test on custom runner
agunapal Oct 23, 2023
e497452
test on custom runner
agunapal Oct 23, 2023
0699a22
kserve ci
agunapal Oct 23, 2023
a14862e
kserve ci
agunapal Oct 23, 2023
dc010d5
kserve ci
agunapal Oct 23, 2023
87900d2
kserve ci
agunapal Oct 23, 2023
05a7f84
kserve ci
agunapal Oct 23, 2023
26487df
kserve ci
agunapal Oct 23, 2023
156484c
kserve ci
agunapal Oct 23, 2023
246047a
kserve ci
agunapal Oct 23, 2023
e75fcd2
kserve ci
agunapal Oct 23, 2023
476623a
kserve ci
agunapal Oct 23, 2023
520ab8b
kserve ci
agunapal Oct 23, 2023
752908b
kserve ci
agunapal Oct 23, 2023
3049766
Added V1 test
agunapal Oct 23, 2023
27baf94
Code cleanup
agunapal Oct 23, 2023
f16090a
Merge branch 'master' into feature/kserve_ci_cpu
agunapal Oct 23, 2023
05c799d
testing updated script
agunapal Oct 24, 2023
90a7072
Merge branch 'feature/kserve_ci_cpu' of https://github.com/pytorch/se…
agunapal Oct 24, 2023
1cc3600
trigger on push for testing
agunapal Oct 24, 2023
419f3b2
testing ci
agunapal Oct 24, 2023
1511810
testing ci
agunapal Oct 24, 2023
f5446e9
testing ci
agunapal Oct 24, 2023
5fb8844
testing ci
agunapal Oct 24, 2023
1e28b62
Modularized code works
agunapal Oct 25, 2023
d8eee7c
Testing KServe CI
agunapal Oct 25, 2023
7751d5c
Code cleanup
agunapal Oct 25, 2023
c659b5b
Merge branch 'master' into feature/kserve_ci_cpu
agunapal Oct 25, 2023
cdbb640
Merge branch 'master' into feature/kserve_ci_cpu
msaroufim Oct 25, 2023
b62085b
updated step name to be compatible with pytorch bot
agunapal Oct 26, 2023
c138922
Merge branch 'feature/kserve_ci_cpu' of https://github.com/pytorch/se…
agunapal Oct 26, 2023
b8591a3
update test name
agunapal Oct 26, 2023
6575c7d
update test name
agunapal Oct 26, 2023
1639a50
update test name
agunapal Oct 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/kserve_ci_cpu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: KServe CI CPU

on:
workflow_dispatch:
# runs everyday at 5:15am
schedule:
- cron: '15 5 * * *'
agunapal marked this conversation as resolved.
Show resolved Hide resolved

jobs:
kserve-ci-cpu:
runs-on: [self-hosted, regression-test-gpu]
steps:
- name: Clean up previous run
run: |
echo "Cleaning up previous run"
ls -la ./
sudo rm -rf ./* || true
sudo rm -rf ./.??* || true
ls -la ./
- name: Install minikube and kubectl
run: |
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
echo "/usr/local/bin" >> $GITHUB_PATH
- name: Setup Python 3.8
uses: actions/setup-python@v4
with:
python-version: 3.8
architecture: x64
- name: Checkout TorchServe
uses: actions/checkout@v3
- name: Checkout kserve repo
uses: actions/checkout@v4
with:
repository: kserve/kserve
path: kserve
- name: Test torchserve-kfs
run: ./kubernetes/kserve/tests/scripts/test_mnist.sh
9 changes: 9 additions & 0 deletions kubernetes/kserve/tests/configs/mnist_v1_cpu.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: "torchserve"
spec:
predictor:
pytorch:
storageUri: gs://kfserving-examples/models/torchserve/image_classifier/v1
image: pytorch/torchserve-kfs-nightly:latest-cpu
10 changes: 10 additions & 0 deletions kubernetes/kserve/tests/configs/mnist_v2_cpu.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: "torchserve-mnist-v2"
spec:
predictor:
pytorch:
protocolVersion: v2
storageUri: gs://kfserving-examples/models/torchserve/image_classifier/v2
image: pytorch/torchserve-kfs-nightly:latest-cpu
9 changes: 9 additions & 0 deletions kubernetes/kserve/tests/data/mnist_v1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@

{
"instances": [
{
"data": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAAw0lEQVR4nGNgGFggVVj4/y8Q2GOR83n+58/fP0DwcSqmpNN7oOTJw6f+/H2pjUU2JCSEk0EWqN0cl828e/FIxvz9/9cCh1zS5z9/G9mwyzl/+PNnKQ45nyNAr9ThMHQ/UG4tDofuB4bQIhz6fIBenMWJQ+7Vn7+zeLCbKXv6z59NOPQVgsIcW4QA9YFi6wNQLrKwsBebW/68DJ388Nun5XFocrqvIFH59+XhBAxThTfeB0r+vP/QHbuDCgr2JmOXoSsAAKK7bU3vISS4AAAAAElFTkSuQmCC",
"target": 0
}
]
}
40 changes: 40 additions & 0 deletions kubernetes/kserve/tests/data/mnist_v2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"id": "d3b15cad-50a2-4eaf-80ce-8b0a428bd298",
"inputs": [{
"name": "input-0",
"shape": [1, 28, 28],
"datatype": "FP32",
"data": [
[
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2392, 0.0118, 0.1647, 0.4627, 0.7569, 0.4627, 0.4627, 0.2392, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0549, 0.7020, 0.9608, 0.9255, 0.9490, 0.9961, 0.9961, 0.9961, 0.9961, 0.9608, 0.9216, 0.3294, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.5922, 0.9961, 0.9961, 0.9961, 0.8353, 0.7529, 0.6980, 0.6980, 0.7059, 0.9961, 0.9961, 0.9451, 0.1804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1686, 0.9216, 0.9961, 0.8863, 0.2510, 0.1098, 0.0471, 0.0000, 0.0000, 0.0078, 0.5020, 0.9882, 1.0000, 0.6784, 0.0667, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2196, 0.9961, 0.9922, 0.4196, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.5255, 0.9804, 0.9961, 0.2941, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2471, 0.9961, 0.6196, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8667, 0.9961, 0.6157, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.7608, 0.9961, 0.4039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.5882, 0.9961, 0.8353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1333, 0.8627, 0.9373, 0.2275, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3294, 0.9961, 0.8353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.4941, 0.9961, 0.6706, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3294, 0.9961, 0.8353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8392, 0.9373, 0.2353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3294, 0.9961, 0.8353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8392, 0.7804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3294, 0.9961, 0.8353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0431, 0.8588, 0.7804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3294, 0.9961, 0.8353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3843, 0.9961, 0.7804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.6353, 0.9961, 0.8196, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3843, 0.9961, 0.7804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2000, 0.9333, 0.9961, 0.2941, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3843, 0.9961, 0.7804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2000, 0.6471, 0.9961, 0.7647, 0.0157, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2588, 0.9451, 0.7804, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0118, 0.6549, 0.9961, 0.8902, 0.2157, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8392, 0.8353, 0.0784, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1804, 0.5961, 0.7922, 0.9961, 0.9961, 0.2471, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8392, 0.9961, 0.8000, 0.7059, 0.7059, 0.7059, 0.7059, 0.7059, 0.9216, 0.9961, 0.9961, 0.9176, 0.6118, 0.0392, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3176, 0.8039, 0.9961, 0.9961, 0.9961, 0.9961, 0.9961, 0.9961, 0.9961, 0.9882, 0.9176, 0.4706, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1020, 0.8235, 0.9961, 0.9961, 0.9961, 0.9961, 0.9961, 0.6000, 0.4078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
]
]
}]
}
119 changes: 119 additions & 0 deletions kubernetes/kserve/tests/scripts/test_mnist.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
#!/usr/bin/env bash
agunapal marked this conversation as resolved.
Show resolved Hide resolved

set -o errexit -o nounset -o pipefail

echo "MNIST KServe V2 test begin"

echo "Removing any previous kubernetes cluster "
minikube delete

echo "Starting kubernetes cluster "

minikube start

echo "Install kserve"

cd $GITHUB_WORKSPACE/kserve
./hack/quick_install.sh
echo "Waiting 5s for kserve pod to come up ..."
sleep 5

echo "Deploy the cluster"

cd $GITHUB_WORKSPACE
kubectl apply -f kubernetes/kserve/tests/configs/mnist_v2_cpu.yaml

echo "Waiting 120s for pod to come up..."
sleep 120
agunapal marked this conversation as resolved.
Show resolved Hide resolved
echo "Check status of the pod"
kubectl get pods
kubectl describe pod torchserve-mnist-v2-predictor

echo "Make cluster accessible by localhost"
MODEL_NAME=mnist
SERVICE_HOSTNAME=$(kubectl get inferenceservice torchserve-mnist-v2 -o jsonpath='{.status.url}' | cut -d "/" -f 3)
export INGRESS_HOST=localhost
export INGRESS_PORT=8080
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80 &

echo "Wait for ports to be in forwarding"
sleep 10

echo "Make inference request"

PREDICTION=$(curl -H "Content-Type: application/json" -H "Host: ${SERVICE_HOSTNAME}" http://${INGRESS_HOST}:${INGRESS_PORT}/v2/models/${MODEL_NAME}/infer -d @./kubernetes/kserve/tests/data/mnist_v2.json)

EXPECTED='{"model_name":"mnist","model_version":null,"id":"d3b15cad-50a2-4eaf-80ce-8b0a428bd298","parameters":null,"outputs":[{"name":"input-0","shape":[1],"datatype":"INT64","parameters":null,"data":[1]}]}'
echo "Creating a Service"

if [ "${PREDICTION}" = "${EXPECTED}" ]; then
echo "✓ Prediction: ${PREDICTION} (Expected ${EXPECTED})"
agunapal marked this conversation as resolved.
Show resolved Hide resolved
else
echo "✘ Test failed: Prediction: ${PREDICTION}, expected ${EXPECTED}."
exit 1
fi

echo "KServe V2 Test successful!"

echo "Delete cluster"

minikube delete

echo "MNIST KServe V1 test begin"

echo "Removing any previous kubernetes cluster "
minikube delete

echo "Starting kubernetes cluster "

minikube start

echo "Install kserve"

cd $GITHUB_WORKSPACE/kserve
./hack/quick_install.sh
echo "Waiting 5s for kserve pod to come up ..."
sleep 5

echo "Deploy the cluster"

cd $GITHUB_WORKSPACE
kubectl apply -f kubernetes/kserve/tests/configs/mnist_v1_cpu.yaml

echo "Waiting 120s for pod to come up..."
sleep 120
echo "Check status of the pod"
kubectl get pods
kubectl describe pod torchserve-predictor

echo "Make cluster accessible by localhost"
MODEL_NAME=mnist
SERVICE_HOSTNAME=$(kubectl get inferenceservice torchserve -o jsonpath='{.status.url}' | cut -d "/" -f 3)
export INGRESS_HOST=localhost
export INGRESS_PORT=8080
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80 &

echo "Wait for ports to be in forwarding"
sleep 10

echo "Make inference request"

PREDICTION=$(curl -H "Content-Type: application/json" -H "Host: ${SERVICE_HOSTNAME}" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/${MODEL_NAME}:predict -d @./kubernetes/kserve/tests/data/mnist_v1.json)

EXPECTED='{"predictions":[2]}'
echo "Creating a Service"

agunapal marked this conversation as resolved.
Show resolved Hide resolved
if [ "${PREDICTION}" = "${EXPECTED}" ]; then
echo "✓ Prediction: ${PREDICTION} (Expected ${EXPECTED})"
else
echo "✘ Test failed: Prediction: ${PREDICTION}, expected ${EXPECTED}."
exit 1
fi

echo "KServe V1 Test successful!"

echo "Delete cluster"

minikube delete
Loading