Unit 9: Model optimization (Model Deployment Considerations) by kiluazen · Pull Request #123 · huggingface/computer-vision-course

kiluazen · 2023-12-16T11:06:15Z

Hi, everyone.

This is content for Unit 9: Model Optimization => Module 3 (Model Deployment).
Here's the outline for reference: https://github.com/johko/computer-vision-course/discussions/92

Please Read through and suggest topics I am missing, redundant parts that need to be excluded etc.

Thank you.

Unit 9: Model optimization (Model Deployment Considerations)

Unit 9 : Module 3 Modle Optimization( Deployment Considerations)

mmhamdy

Looking great! just have one question: is this part of a larger chapter on deployment?

adhiiisetiawan · 2023-12-17T18:02:49Z

+
+
+- **Kubernetes Documentation**: [Kubernetes](https://kubernetes.io/docs/home/) is an open-source container orchestration platform widely used for deploying and managing applications. Understanding Kubernetes can help deploy models in a scalable and reliable manner.
+## Best Practices for Deployment in Production


Hi @kiluazen sorry just notice, i think in this section you can add like "model optimization" part. So, one of best practices before deployment is "optimize the model", and because the deployment is important (reduce latency, etc), we need to optimize the model first (depend on where we will deploy the model (ex: cloud, mobile, edge devices))

what do you think?

merveenoyan

I mostly did nitpicking, thank you!

merveenoyan · 2023-12-18T14:42:32Z

+
+## Different Deployment Platforms
+- **Cloud**: Deploying models on cloud platforms like AWS, Google Cloud, or Azure offers a scalable and robust infrastructure for AI model deployment. These platforms provide managed services for hosting models, ensuring scalability, flexibility, and integration with other cloud services.
+    - Advantages: Scalability, high computing power, managed services, integration with cloud ecosystem.


Maybe you can make a statement here for clarity, e.g. Cloud deployment is easily scalable by means of computing power and memory (and so on)
same goes for the rest of the text

This is waiting to be addressed

merveenoyan · 2023-12-18T14:45:27Z

+
+- **Packaging:** Packaging, on the other hand, involves bundling all the necessary components and dependencies of a machine learning model. It's like putting all the puzzle pieces into a box, along with the instructions on assembling it. Packaging includes everything needed to run the model, such as the serialized model file, pre-processing or post-processing code, and required libraries or dependencies.
+
+    Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.


Suggested change

Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.

Serialization is device-agnostic when packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.

merveenoyan · 2023-12-18T14:47:27Z

+- **Load Testing**: Simulate varying workloads to ensure the model's responsiveness under different conditions.
+
+- **Anomaly Detection**: Implement systems to detect deviations in model behavior and performance.
+- **Real-time Monitoring**: Utilize tools for immediate identification of issues in deployed models.


giving an example of an issue (e.g. distribution shift) would be nice

MKhalusova

LGTM, thanks for writing this!

MKhalusova · 2023-12-18T18:41:30Z

+
+- **Packaging:** Packaging, on the other hand, involves bundling all the necessary components and dependencies of a machine learning model. It's like putting all the puzzle pieces into a box, along with the instructions on assembling it. Packaging includes everything needed to run the model, such as the serialized model file, pre-processing or post-processing code, and required libraries or dependencies.
+
+    Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.


Suggested change

Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.

Serialization is device-agnostic when packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.

MKhalusova · 2023-12-18T18:42:51Z

+
+    Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.
+
+## Model Serving and Inference


This would be a great place to mention Inference Endpoints - https://huggingface.co/inference-endpoints

wow, cool. I just found out that huggingface has this.

merveenoyan · 2023-12-20T12:33:48Z

@kiluazen Can you incorporate @MKhalusova's changes and I'll give another review and we can merge next?

Co-authored-by: Mohammed Hamdy <62081584+mmhamdy@users.noreply.github.com>

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

merveenoyan · 2023-12-21T14:47:04Z

@@ -0,0 +1,52 @@
+# Model Deployment Considerations
+


Can you briefly mention here what this chapter will talk about? e.g. "In this chapter we will go through various deployment practices"

merveenoyan · 2023-12-21T14:47:21Z

+
+## Different Deployment Platforms
+- **Cloud**: Deploying models on cloud platforms like AWS, Google Cloud, or Azure offers a scalable and robust infrastructure for AI model deployment. These platforms provide managed services for hosting models, ensuring scalability, flexibility, and integration with other cloud services.
+    - Advantages: Scalability, high computing power, managed services, integration with cloud ecosystem.


This is waiting to be addressed

Model optimization module3

kiluazen · 2023-12-25T10:10:52Z

Hey @merveenoyan @MKhalusova, I have made the changes.
Please comment and suggest for further improvement .
Sorry for the late response.

merveenoyan

Looks good to me! waiting for a community contributor to approve and we can merge

kiluazen and others added 9 commits December 8, 2023 18:19

draft1 of module 3

779d566

serving, inference done

ce76cfd

deployment platforms done

4527c0a

finished draft1

ecdf8b2

file name change

5b5cefd

Merge pull request #2 from adhiiisetiawan/model_optimization_module3

d4571bb

Unit 9: Model optimization (Model Deployment Considerations)

Merge branch 'johko:main' into main

fd551e3

grammer changes

8e310bc

Merge pull request #3 from adhiiisetiawan/model_optimization_module3

865a979

Unit 9 : Module 3 Modle Optimization( Deployment Considerations)

kiluazen requested review from MKhalusova and merveenoyan as code owners December 16, 2023 11:06

mmhamdy reviewed Dec 17, 2023

View reviewed changes

Comment thread chapters/en/Unit 9 - Model Optimization/model_deployment.mdx Outdated

Comment thread chapters/en/Unit 9 - Model Optimization/model_deployment.mdx Outdated

adhiiisetiawan reviewed Dec 17, 2023

View reviewed changes

merveenoyan reviewed Dec 18, 2023

View reviewed changes

MKhalusova reviewed Dec 18, 2023

View reviewed changes

Merge branch 'johko:main' into main

361146b

adhiiisetiawan and others added 10 commits December 21, 2023 00:33

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

0fbd3ca

Co-authored-by: Mohammed Hamdy <62081584+mmhamdy@users.noreply.github.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

18ef6c7

Co-authored-by: Mohammed Hamdy <62081584+mmhamdy@users.noreply.github.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

e174512

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

cb1650b

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

50d101e

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

a6f0900

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

8afbe03

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

2895d20

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

e1fce05

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

Update chapters/en/Unit 9 - Model Optimization/model_deployment.mdx

a24a67d

Co-authored-by: Merve Noyan <merveenoyan@gmail.com>

merveenoyan reviewed Dec 21, 2023

View reviewed changes

kiluazen added 2 commits December 24, 2023 14:21

changes to cloud deploy

677079c

distrubution shift eg added

14e9e37

kiluazen and others added 5 commits December 24, 2023 15:11

added a intro

0a9b0cd

added hf inference sols

a531332

Merge branch 'main' into model_optimization_module3

1f50098

Merge pull request #4 from adhiiisetiawan/model_optimization_module3

7c49277

Model optimization module3

Merge branch 'johko:main' into main

8fb8bc2

merveenoyan approved these changes Dec 25, 2023

View reviewed changes

mmhamdy approved these changes Dec 25, 2023

View reviewed changes

merveenoyan merged commit d185c35 into huggingface:main Dec 25, 2023

adhiiisetiawan approved these changes Dec 25, 2023

View reviewed changes



		- Kubernetes Documentation: [Kubernetes](https://kubernetes.io/docs/home/) is an open-source container orchestration platform widely used for deploying and managing applications. Understanding Kubernetes can help deploy models in a scalable and reliable manner.
		## Best Practices for Deployment in Production


		- Packaging: Packaging, on the other hand, involves bundling all the necessary components and dependencies of a machine learning model. It's like putting all the puzzle pieces into a box, along with the instructions on assembling it. Packaging includes everything needed to run the model, such as the serialized model file, pre-processing or post-processing code, and required libraries or dependencies.

		Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.


		Serialization is device-agnostic When packaging for cloud deployment. Serialized models are often packaged into containers (e.g., Docker) or deployed as web services (e.g., Flask or FastAPI). Cloud deployments also involve auto-scaling, load balancing, and integration with other cloud services.

		## Model Serving and Inference

Uh oh!

Conversation

kiluazen commented Dec 16, 2023

Uh oh!

mmhamdy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

merveenoyan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MKhalusova left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

merveenoyan commented Dec 20, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kiluazen commented Dec 25, 2023

Uh oh!

merveenoyan left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants