Skip to content

Test Deployment Build v0.3 Fast Response Time

Pre-release
Pre-release

Choose a tag to compare

@Zelefant Zelefant released this 30 Mar 19:47
· 26 commits to deployment_testing since this release

Changelog:

  • Response time has been massively improved. It now responds in about 6-12 seconds on an NVIDIA GeForce RTX 3060 with 8 GB of VRAM using Qwen3-4B.
  • Hotfix to add user information instead of dummy information.

Instructions:

  • Must be on Debian-based Linux (Ubuntu, Debian, Linux Mint)
  • Download source code (zip) and extract
  • Port forward network on port 8000
  • Run ./setup.sh in the root of the project files
  • When the server is running, connect to https://[your-ip-here]:8000/
  • Default admin username: admin, default password: password1234

Notes:

  • Response time fix only applies to Qwen module currently. This will be fixed soon. Mistral and Llama will still take a long time to respond.
  • First time opening the chat page after launching the program will take a long time, this is intentional. Launching the chat page again will be instant.
  • Known issue: Session creation is buggy. Try to stick to one session for now.
  • Known issue: Submitting a quiz will say "No active quiz" and won't give you the answer key. Simply reopen the session and it'll show you the answer key.
  • Known issue: System prompt exposed when you switch sessions.