03 Supported AI Providers and Setup

Supported AI Providers & Setup

WishPilot features a Unified Multi-Provider Streaming Interface that allows candidates to leverage any major LLM provider without changing client code.

---

1. Provider Comparison & Specifications

Provider	Default Model	Typical First Token (TTFT)	Best Used For	Free Tier Available?
Groq Cloud	openai/gpt-oss-120b	~80ms - 140ms	Fastest live spoken answers & Whisper STT	Yes (Generous free tier)
Cerebras	llama-3.3-70b	~100ms - 160ms	Wafer-scale ultra-high token throughput	Yes (Free developer tier)
Together AI	Llama-3.3-70B-Turbo	~180ms - 260ms	High concurrency and cost-effective scaling	Yes ($5 free credits)
Fireworks AI	llama-v3p3-70b-instruct	~150ms - 220ms	Compound inference & fast function calling	Yes ($1 free credits)
NVIDIA NIM	nemotron-3-ultra-550b	~250ms - 400ms	Complex system design reasoning & algorithms	Yes (1,000 free credits)
Hugging Face	Llama-3.3-70B-Instruct	~300ms - 500ms	Open-source community models	Yes (Free serverless tier)
OpenRouter	claude-3.5-sonnet	~400ms - 600ms	Universal aggregator (Claude, DeepSeek R1)	BYOK / Pay per token
OpenAI Direct	gpt-4o	~350ms - 500ms	Industry standard coding & vision reasoning	Pay per token
Google Gemini	gemini-2.0-flash	~200ms - 350ms	High-speed multimodal & large context	Yes (Free tier via AI Studio)

---

2. Step-by-Step Provider Setup Guides

1. Groq Cloud (Recommended Default)

Groq LPUs provide the fastest streaming speeds available today, rendering full 200-word responses in under 500ms.

1. Visit console.groq.com/keys and log in.
2. Click Create API Key, name it WishPilot, and copy the key (gsk_...).
3. In WishPilot: Go to Models & Audio ➔ Select Groq Cloud ➔ Paste the key.
4. Default Model: openai/gpt-oss-120b (Flagship).

2. Cerebras Inference

Cerebras runs on wafer-scale AI chips, delivering extreme throughput for 70B parameter models.

1. Go to cloud.cerebras.ai and sign up.
2. Generate an API Key under API Keys.
3. In WishPilot: Select Cerebras ➔ Paste your key (csk-...).
4. Default Model: llama-3.3-70b.

3. Together AI

1. Visit api.together.ai and create an account.
2. Copy your API key from your profile dashboard.
3. In WishPilot: Select Together AI ➔ Paste key.
4. Default Model: meta-llama/Llama-3.3-70B-Instruct-Turbo.

4. Fireworks AI

1. Visit fireworks.ai and log in.
2. Navigate to API Keys and generate a new key (fw_...).
3. In WishPilot: Select Fireworks AI ➔ Paste key.
4. Default Model: accounts/fireworks/models/llama-v3p3-70b-instruct.

5. NVIDIA NIM (NVIDIA Inference Microservices)

1. Go to build.nvidia.com and sign in with an NVIDIA account.
2. Select any model (e.g. nvidia/nemotron-3-ultra-550b-a55b or meta/llama-3.3-70b-instruct).
3. Click Get API Key (nvapi-...).
4. In WishPilot: Select NVIDIA NIM ➔ Paste key.

6. Hugging Face Inference

1. Go to huggingface.co/settings/tokens.
2. Create an Access Token with read permissions (hf_...).
3. In WishPilot: Select Hugging Face ➔ Paste token.
4. Default Model: meta-llama/Llama-3.3-70B-Instruct.

7. OpenRouter

OpenRouter provides unified access to models like Claude 3.5 Sonnet, DeepSeek R1, and GPT-4o through a single key.

1. Visit openrouter.ai/keys.
2. Create an API key (sk-or-...).
3. In WishPilot: Select OpenRouter ➔ Paste key.
4. Model ID: anthropic/claude-3.5-sonnet (or type any custom model identifier).

8. OpenAI Direct

1. Go to platform.openai.com/api-keys.
2. Generate a Secret Key (sk-...).
3. In WishPilot: Select OpenAI ➔ Paste key.
4. Default Model: gpt-4o (or gpt-4o-mini for faster response times).

9. Google Gemini

1. Visit aistudio.google.com and click Get API key.
2. Copy your key (AIzaSy...).
3. In WishPilot: Select Google Gemini ➔ Paste key.
4. Default Model: gemini-2.0-flash.

---

3. Recommended Setup Strategy

For optimal performance during practice and live technical interviews, use this Dual-Provider Configuration:

1. Speech-to-Text (STT): Use Groq Cloud for Whisper Large v3 Turbo (sub-300ms transcription).
2. Answer Generation:
   • For Ultra-Fast Spoken Answers: Use Groq (openai/gpt-oss-120b) or Cerebras (llama-3.3-70b). Answers start streaming within ~100ms.
   • For Deep System Design & Coding: Use OpenRouter (anthropic/claude-3.5-sonnet) or OpenAI (gpt-4o).