Distributed GPU serving API petals.ml integration

[sapcode]

Discussed in #4451

^{Originally posted by sapcode May 28, 2023}
Thank you very much for your work and publishing!!
@SigGravitas could you please look into this http://petals.ml and this example https://github.com/bigscience-workshop/petals/blob/main/examples/prompt-tuning-personachat.ipynb
If you can integrate that, it will allow distributed GPU serving community free models without any paid API needs...
Here is a how to spawn own GPU swarm https://github.com/bigscience-workshop/petals/wiki/Launch-your-own-swarm

[isaaclepes]

I came here to suggest this same feature. 👍

[isaaclepes]

Man, I was really hoping to see movement on this. Petals could be a game changer and it already has the new CodeLLaMA 34B model up and running.

[github-actions]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions]

This issue was closed automatically because it has been stale for 10 days with no activity.