Fireworks as an LLM provider?

[aimzieslol]

I did attempt it using podman but it gets upset at my -G definition:

openai://accounts/fireworks/models/llama-v3-8b-hf@api.fireworks.ai/inference/v1

Yes, Fireworks defines accounts/fireworks/models/llama-v3-8b-hf as the entire model name. Yes, you have to provide the /inference/v1 for their chat completions.

Workarounds would be appreciated! I'll use Groq in the meantime.

[evilsocket]

Hi! I had to integrate it, fortunately the API is the same of OpenAI so it was pretty quick! :D

nerve -G "fireworks://llama-v3-70b-instruct" ...

Keep in mind that llama-v3-8b-hf will fail with: model is missing conversation_config, please use /completions API instead because of this.