Proposal: Add Apache APISIX as an optional gateway for LocalAI

[moonming]

Hi LocalAI maintainers,

I'm Ming Wen, a PMC member of
Apache APISIX.

Would you be open to documenting Apache APISIX as an optional gateway for
LocalAI?

The integration would use LocalAI's existing OpenAI-compatible API without
changing its core code. I can provide a tested example covering proxying,
streaming, forwarded headers, and inference timeouts.

If this is welcome, would you prefer:

1. A short section in the existing reverse proxy guide; or
2. A brief reference to a complete example maintained in the APISIX repository?

I'd be happy to follow your preferred scope. Thanks!

[localai-org-maint-bot]

I’m picking this up. I’ll add a focused Apache APISIX section to the existing reverse-proxy/TLS guide with a tested route example for streaming, forwarded headers, and long inference timeouts, then validate the documentation build.

[mudler]

Hi LocalAI maintainers,

I'm Ming Wen, a PMC member of Apache APISIX.

Would you be open to documenting Apache APISIX as an optional gateway for LocalAI?

Hi @moonming, yes, that would make sense, especially if you already have examples or documentation in APISIX on how to run it with LocalAI, we could cross-link directly

The integration would use LocalAI's existing OpenAI-compatible API without changing its core code. I can provide a tested example covering proxying, streaming, forwarded headers, and inference timeouts.

If this is welcome, would you prefer:

1. A short section in the existing reverse proxy guide; or
2. A brief reference to a complete example maintained in the APISIX repository?

I'd be happy to follow your preferred scope. Thanks!

1. It was partly picked up by our automatic bot in docs: add Apache APISIX reverse proxy example #11294, probably would benefit of also giving a bit more of space in presenting APISIX, and explain how to use it with LocalAI in full (cc @localai-org-maint-bot ) - feel free to comment on the PR or open up yours - I'll be happy to review or either steer the bot
2. Generally we try to cross-link, if you have a section showing how to integrate LocalAI and examples we can reference in our docs directly - this would make much more sense as they would be updated directly by you

[localai-org-maint-bot]

Updated #11294 at c49caac64 following this direction. The APISIX section now explains the gateway use case, container-network reachability, route setup, model discovery, a streaming chat request, TLS/base URL behavior, and LocalAI versus APISIX authentication. The route and chat JSON examples parse and the diff check passes; Hugo is unavailable on this host. @moonming, if the APISIX project publishes a maintained LocalAI integration guide, we can replace the provisional cross-link note with its URL.

[moonming]

Thanks! We've opened the APISIX-side integration guide:
apache/apisix#13770

The PR is currently under review. Once it is merged and published, I'll share the stable documentation URL for the cross-link in #11294. Feedback on the integration scope is welcome.