[C4GT] Deployment Optimizations

[singhalkarun]

Project Details

AI Toolchain is a collection of tools for quickly building and deploying machine learning models for various use cases. Currently, the toolchain includes a text translation model, and more models may be added in the future. It abstracts the dirty details of how a model works similar to Huggingface and gives a clean API that you can orchestrate at aenter code here BFF level.

Features to be implemented

Abstract the layer of deployment for AI Tools. Anyone should be easily add a new model to the stack without thinking about deployments. We should be able to deploy AI Tools in such a way where each model (every model can be packaged as a container) should be independently scalable. As a user, I should be able to access APIs associated with any model.

Learning Path

Complexity

Medium

Skills Required

Docker, Docker Compose

Name of Mentors:

@singhalkarun

Project size

8 Weeks

Product Set Up

See the setup here

Acceptance Criteria

• Document to add a new model and test it on local
• Document to deploy a new model
• Document to accesss APIs asscoiated with model

Milestone

A flow to add, test and deploy a model in a scalable manner.

Reference

C4GT

This issue is nominated for Code for GovTech (C4GT) 2023 edition.
C4GT is India's first annual coding program to create a community that can build and contribute to global Digital Public Goods. If you want to use Open Source GovTech to create impact, then this is the opportunity for you! More about C4GT here: https://codeforgovtech.in/

[rafey1104]

I'm really looking forward to submitting a proposal for this project.

[fibonacci35813]

I have experience in this field and would love to contribute in this project.

[godofgeeks23]

Looking forward towards contributing in this project

[ayush0x00]

@singhalkarun can we use Kubernetes to manage all the models deployed as containers? Or do we have to stick to the usage of only docker?

[singhalkarun]

We are planning to move to k8s as well in future. Accordingly, the scope of this project might change. Feel free to create a proposal around it and submit on unstop if you are interested to take this up.

[nitishmalang]

hey @singhalkarun I have submitted proposal

[Tejash11]

We are planning to move to k8s as well in future. Accordingly, the scope of this project might change. Feel free to create a proposal around it and submit on unstop if you are interested to take this up.

Hey @singhalkarun I have submitted the proposal for the same. Looking ahead to work on this