Need for Speed: LLMs Beyond OpenAI with .NET 8 SSE + Channels, Llama3, and Fireworks.ai

This repository demonstrates how to combine:

• .NET 8 Web APIs
• .NET 8 channels
• Microsoft Semantic Kernel
• Server Sent Events (SSE)
• Meta Llama 3 70B on Fireworks.ai

To build a high throughput recipe generator.

To see the full writeup, check out the article: Need for Speed: LLMs Beyond OpenAI with .NET 8 SSE + Channels, Llama3, and Fireworks.ai

Running the Demo

Follow these instructions to run the demo:

First, get an API key from Fireworks.ai and Groq.com and add them to the user secrets.

cd api
dotnet user-secrets init
dotnet user-secrets set "RecipesConfig:FireworksKey" "YOUR_KEY_HERE"
dotnet user-secrets set "RecipesConfig:GroqKey" "YOUR_KEY_HERE"

To run the front-end:

cd web
yarn        # Restore packages
yarn dev    # Run the front-end demo.

In another terminal session:

cd api
dotnet run