Skip to content

model deployment #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
eff-kay merged 6 commits into from
Dec 21, 2023
Merged

model deployment #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
376c38e
wip: model deployemnt
eff-kay Sep 29, 2023
3355944
finalize model deployment
eff-kay Oct 10, 2023
e52f891
update changelog
eff-kay Dec 21, 2023
874d580
remove extra files
eff-kay Dec 21, 2023
b4de0d1
remove checkppoints
eff-kay Dec 21, 2023
1561264
update changes
eff-kay Dec 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@
All notable changes to `slashml-python-client` aka `slashml` will be documented in this file.
This project adheres to [Semantic Versioning](https://semver.org/).

## 0.1.6 - 2023-12-21

### Added
- Pin some dependencies
- Add a requirements flag for model deployment


## 0.1.5 - 2023-09-17

### Added
- Unpin depenedencies

## 0.1.4 - 2023-06-09

### Added
Expand Down
8 changes: 4 additions & 4 deletions examples/deploy_custom_models/hugging_face_transformer/deploy_hugging_face_transformer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
# you might have to install transfomers and torch
from transformers import pipeline

def train_model():
# Bring in model from huggingface
return pipeline('fill-mask', model='bert-base-uncased')
# def train_model():
# # Bring in model from huggingface
# return

my_model = train_model()
my_model = pipeline('fill-mask', model='bert-base-uncased')

# Replace `API_KEY` with your SlasML API token.
model = ModelDeployment(api_key=None)
Expand Down
353 changes: 353 additions & 0 deletions examples/deploy_custom_models/hugging_face_transformer/hugging-face-test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,353 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "ea4e3e72-09e2-484c-af14-4819276a29e4",
"metadata": {},
"source": [
"## Build and Test your model locally"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "577c63e0-ce42-4755-bedd-3e6b259899ff",
"metadata": {},
"outputs": [],
"source": [
"# you might have to install transfomers and torch\n",
"import torch\n",
"from transformers import pipeline\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "7d9cceeb-8e6f-40ba-869d-ffca7c33c9fc",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'bert.pooler.dense.weight']\n",
"- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
"- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
]
}
],
"source": [
"my_model = pipeline('fill-mask', model='bert-base-uncased')"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "17c0d0a5-37b3-48bb-852c-5bae4642c854",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'score': 0.5164636969566345,\n",
" 'token': 3910,\n",
" 'token_str': 'founder',\n",
" 'sequence': 'steve jobs is the founder of apple'},\n",
" {'score': 0.36049818992614746,\n",
" 'token': 5766,\n",
" 'token_str': 'ceo',\n",
" 'sequence': 'steve jobs is the ceo of apple'},\n",
" {'score': 0.049299854785203934,\n",
" 'token': 2343,\n",
" 'token_str': 'president',\n",
" 'sequence': 'steve jobs is the president of apple'},\n",
" {'score': 0.02111203595995903,\n",
" 'token': 8543,\n",
" 'token_str': 'creator',\n",
" 'sequence': 'steve jobs is the creator of apple'},\n",
" {'score': 0.008550191298127174,\n",
" 'token': 2269,\n",
" 'token_str': 'father',\n",
" 'sequence': 'steve jobs is the father of apple'}]"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"my_model('steve jobs is the [MASK] of apple')"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "1032fb2e-dac6-48be-a3ba-94b99f5a3249",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'score': 0.31370100378990173,\n",
" 'token': 2269,\n",
" 'token_str': 'father',\n",
" 'sequence': 'elon musk is the father of tesla'},\n",
" {'score': 0.21307483315467834,\n",
" 'token': 5766,\n",
" 'token_str': 'ceo',\n",
" 'sequence': 'elon musk is the ceo of tesla'},\n",
" {'score': 0.14913086593151093,\n",
" 'token': 2365,\n",
" 'token_str': 'son',\n",
" 'sequence': 'elon musk is the son of tesla'},\n",
" {'score': 0.051770761609077454,\n",
" 'token': 2343,\n",
" 'token_str': 'president',\n",
" 'sequence': 'elon musk is the president of tesla'},\n",
" {'score': 0.050566092133522034,\n",
" 'token': 3910,\n",
" 'token_str': 'founder',\n",
" 'sequence': 'elon musk is the founder of tesla'}]"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"my_model('elon musk is the [MASK] of tesla')"
]
},
{
"cell_type": "markdown",
"id": "c9b9eef0-460e-4437-ac70-e396394ad263",
"metadata": {},
"source": [
"## Deploy the model to SlashML"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "0785f21b-2d2d-42e9-83fb-e88d77ac2c3c",
"metadata": {},
"outputs": [],
"source": [
"from slashml import ModelDeployment\n",
"import time"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "015c1d53-309f-45ae-b052-bd44d4fcfd43",
"metadata": {},
"outputs": [],
"source": [
"# Replace `API_KEY` with your SlasML API token.\n",
"client = ModelDeployment(api_key=\"68b50f46b97f8c0f98fe419df240a22af3da392f\")\n",
"# client = ModelDeployment(api_key=\"e7e33ba9a41c1426a425496319a14cab2b3f794f\")"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "9916a6d5-bcbc-4ab5-9c39-8c8a630436b8",
"metadata": {},
"outputs": [],
"source": [
"# deploy model\n",
"response = client.deploy(model_name='my_model_3', model=my_model)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "fa7c3e50-d512-41e9-9d38-c0f842b9b5f4",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'163fc6ad-fbab-4f17-b06a-860eba1f5496'"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"response.id"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "1843adac-3341-49db-83b7-88e0b7f1eb20",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'id': '163fc6ad-fbab-4f17-b06a-860eba1f5496',\n",
" 'created': '2023-09-23T22:24:23.737712Z',\n",
" 'status': 'READY',\n",
" 'name': 'my_model_3'}"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"client.status(model_version_id=response.id)"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "1263caa7-3147-4229-90b6-84bae4b0e479",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"model succeeded\n"
]
}
],
"source": [
"status = client.status(model_version_id=response.id)\n",
"\n",
"while status.status != 'READY':\n",
" print(f'status: {status.status}')\n",
" print('trying again in 5 seconds')\n",
" time.sleep(5)\n",
" status = model.status(model_version_id=response.id)\n",
"\n",
" if status.status in ['FAILED', \"ERROR\"]:\n",
" raise Exception(f'Model deployment {status}')\n",
"\n",
"print(f'model succeeded')"
]
},
{
"cell_type": "markdown",
"id": "f387ba63-28ab-4d4a-9d4f-ec86d4483441",
"metadata": {},
"source": [
"## Run predictions on the hosted model"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "9ca7d301-6b38-46c1-aad8-1a9909b1001b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'id': '163fc6ad-fbab-4f17-b06a-860eba1f5496', 'model_input': 'Steve jobs is the [MASK] of Apple.', 'model_response': {'predictions': [{'score': 0.49993884563446045, 'sequence': 'steve jobs is the founder of apple.', 'token': 3910, 'token_str': 'founder'}, {'score': 0.39265578985214233, 'sequence': 'steve jobs is the ceo of apple.', 'token': 5766, 'token_str': 'ceo'}, {'score': 0.04603331536054611, 'sequence': 'steve jobs is the president of apple.', 'token': 2343, 'token_str': 'president'}, {'score': 0.01692850887775421, 'sequence': 'steve jobs is the creator of apple.', 'token': 8543, 'token_str': 'creator'}, {'score': 0.008718223311007023, 'sequence': 'steve jobs is the chairman of apple.', 'token': 3472, 'token_str': 'chairman'}]}}\n"
]
}
],
"source": [
"input_text = 'Steve jobs is the [MASK] of Apple.'\n",
"prediction = client.predict(model_version_id=response.id, model_input=input_text)\n",
"print(prediction)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "08b95a31-10d1-457f-abdf-c4310b0d02ac",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'score': 0.49993884563446045,\n",
" 'sequence': 'steve jobs is the founder of apple.',\n",
" 'token': 3910,\n",
" 'token_str': 'founder'},\n",
" {'score': 0.39265578985214233,\n",
" 'sequence': 'steve jobs is the ceo of apple.',\n",
" 'token': 5766,\n",
" 'token_str': 'ceo'},\n",
" {'score': 0.04603331536054611,\n",
" 'sequence': 'steve jobs is the president of apple.',\n",
" 'token': 2343,\n",
" 'token_str': 'president'},\n",
" {'score': 0.01692850887775421,\n",
" 'sequence': 'steve jobs is the creator of apple.',\n",
" 'token': 8543,\n",
" 'token_str': 'creator'},\n",
" {'score': 0.008718223311007023,\n",
" 'sequence': 'steve jobs is the chairman of apple.',\n",
" 'token': 3472,\n",
" 'token_str': 'chairman'}]"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"prediction['model_response']['predictions']"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d1dd161a-8244-4f78-a852-3f25b5435b32",
"metadata": {},
"outputs": [],
"source": [
"input_text = 'elon jobs is the [MASK] of Apple.'\n",
"prediction = client.predict(model_version_id=response.id, model_input=input_text)\n",
"print(prediction)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5c31e48e-9dbc-407b-b2ac-3008a91be93d",
"metadata": {},
"outputs": [],
"source": [
"prediction['model_response']['predictions']"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
2 changes: 1 addition & 1 deletion examples/deploy_custom_models/hugging_face_transformer/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
transformers==4.30.0
torch==2.0.1
slashml==0.1.4
requests
Loading