Serveless Deployment of haystack

[kodosuke]

I have been wondering if there is any possibility that we could deployment the pipeline serveless in AWS Lambda, GCP, Azure, ..

[tholor]

Hey @kodosuke, we have been deploying our production pipelines via Kubernetes and these days switching to Ray+Kubernetes. We thought about AWS Lambda a while ago, but it wasn't supporting GPUs which was for us a must-have (and will be for most production deployments of QA pipelines). If you Ray+Kubernetes is an option for you, maybe have a look at our new RayPipeline class (https://haystack.deepset.ai/components/pipelines#distributed-pipelines-with-ray). We will extend its functionality over the next weeks quite a bit.

[kodosuke]

Thanks @tholor

[jatinvinkumar]

Hi @tholor, thanks for the insight! Could you please provide a helm chart or any additional information into the architecture setup?

[tholor]

@jatinvinkumar Our own infrastructure has grown quite a lot so sharing those details won't be very useful. Instead, we will work on some simpler templates / guides on how to deploy Haystack on Kubernetes / AWS and share them with the community. It's already on our roadmap for this quarter. (CC @tstadel )

[Jawahar-Kaliraj]

@tholor @tstadel

Hi @tholor
We have followed the instructions given in the below documentation and hosting haystack in AWS EC2 instance (GPU) as REST API endpoint.

But we are looking forward to guides on deploying Haystack in AWS ECS please.

Many thanks
Jawahar

https://www.deepset.ai/blog/building-a-rest-api-for-question-answering-with-haystack