Skip to content

Releases: codewithmuh/private-llm-chat-cloudrun-gcp

Release list

v1.0.0: Serverless Private LLM Chat on Google Cloud Run with NVIDIA L4 GPUs

v1.0.0: Serverless Private LLM Chat on Google Cloud Run with NVIDIA L4 GPUs Latest
Latest

Choose a tag to compare

View all tags
@codewithmuh codewithmuh released this 26 Oct 06:49
v1.0.0
34e5ab7

🚀 What's New in v1.0.0

Welcome to v1.0.0 of Private LLM Chat on Google Cloud Run!

🌟 Release Highlights

  • Serverless Cloud Run GPUs: Deploy open-source LLMs (Qwen, Llama 3, Gemma) on Cloud Run with official vLLM containers and NVIDIA L4 GPUs.
  • Scale to Zero with Wake-on-Message: Automatically scales to zero instances when idle (zsh compute costs), waking seamlessly on incoming chat queries.
  • Private Identity-Aware Architecture: Internal services secured via Google Cloud IAM ID tokens without exposing public API endpoints or keys.
  • ChatGPT-Grade Web Application: Built with Next.js 16 and Django 5.2, featuring multi-user accounts, 2FA, file/image OCR extraction, web search (SearXNG), and Whisper speech input.
  • Infrastructure as Code: Turnkey Terraform configs automating Artifact Registry, Cloud Storage FUSE mounts, Cloud SQL PostgreSQL 17, and Cloud Run.
Assets 2
Loading