feat: deploy langflow using langchain-serve

[deepankarm]

Goals

Deploy Langflow on Cloud using a single command - langflow --jcloud

1. Exposes all (HTTP & WebSockets) endpoints defined by the FastAPI app with TLS certs.
2. Exposes the frontend mounted at /.
3. Swagger UI, OpenAPI spec loaded at /docs

---

Using Time Travel Guide

Frontend

API

---

Features

1. Automatic TLS certs with the deployment.
2. Demand custom infrastructure resources as per need with simple a YAML change.
3. Horizontal autoscaling of apps according to usage.
4. Built-in logging, and monitoring for the endpoints (advanced dashboards coming soon)
5. No infrastructure headache (no dockerfiles/k8s and beyond) to manage.

Hidden from the user

1. Pushes the required langflow image to Jina AI Cloud during the release for faster deployment.
2. Added lcserve_push and lcserve_deploy make targets for testing deployment during development.

[deepankarm]

Moving to draft PR to complete

(TODO before merging the PR) Pushes the required langflow image to Jina AI Cloud during the release for faster deployment

[ogabrielluiz]

I tried running it here but after deployment, I get the following {"message":"failure to get a peer from the ring-balancer"}.
The building process seems to take a while due to the many libraries LangFlow installs. This message appears because it has not finished building?

[deepankarm]

I tried running it here but after deployment, I get the following {"message":"failure to get a peer from the ring-balancer"}.

There's an issue with the latest lc-serve build right now. I'm testing a fix, and will release it soon.

The building process seems to take a while due to the many libraries LangFlow installs. This message appears because it has not finished building?

Yes, the docker image size is quite large (>5G) because of all the dependencies, which is why I want to separate these 2 things

• Building & pushing the image during release.yml (This action is still pending).
• During langflow jcloud, use the latest version's image for the deployment.

[deepankarm]

@ogabrielluiz can you please give it (langflow jcloud) another try with the latest commit?

Gotchas

• make lcserve_push takes quite some time (>15mins), so as I commented, I'll do this during the release action.
• langflow jcloud expects the docker image to be already pushed for the current $(poetry version --short). So if you're working with a development branch for which the image is not pushed (release is not done), the deployment would fail.
• Right now for testing, I've pushed the image only for langflow==0.0.71, so please install this version in your env and retry the command.

Tomorrow, I'll do a few more runs with a few more fixes for the size of the image.

[ogabrielluiz]

Tried it and it works!

One point of concern is adding long-running steps to the release.yml. We can discuss more about that as we flesh out this feature.

[deepankarm]

One point of concern is adding long-running steps to the release.yml.

Sure, I understand this. There are 2 options here -

1. Have a workflow in langchain-serve repo that builds & pushes this image and it gets triggered by the release.yml workflow using workflow-dispatch.
2. Have another workflow in the current repo that builds & pushes this image and it gets triggered by the release.yml workflow using workflow-dispatch, but doesn't wait for it and goes to pypi release.

I'd prefer starting with 1 as the images are pushed/pulled to/from my Jina AI account ("jinaai+docker://deepankarm/langflow").

[ogabrielluiz]

Option 1 looks interesting.

Another point is we should make langchain-serve an optional dependency. This is something we'll have to work on soon as the dependencies are getting out of hand already, so maybe we start with langchain-serve.

[ogabrielluiz]

I was thinking of creating a folder in the root called deploy and having some of the options available there, including jina. Would this PR still work in that scenario?

[deepankarm]

@ogabrielluiz How to make it available to pypi users if this code is outside the package?

We can definitely make jcloud deployment work with a different directory structure, but should we handle that in another PR (once we have other providers)?

[deepankarm]

@ogabrielluiz The PR is now ready. Awaiting your approval to run the CI. There might be issues when we merge it and run the release workflow while triggering the external event (though I've tried to make sure there are no issues there).

[deepankarm]

Added how to deploy & API usage to the readme 👉 link