LLM Chat Stream

=========================== EvilOllama - Secure LLM Server Setup for Cardputer

🧠 This guide shows how to set up a secure, authenticated Ollama LLM server on a Raspberry Pi 5 (8GB), publicly accessible and ready for M5Stack Cardputer.

---

✅ Prerequisites

• PC running Linux / WSL / Raspberry Pi 5 (8GB) ( or less but consider LLM need a lot of ressources)
• Domain name (e.g., 7h30th3r0n3.com)
• Port 443 open to the internet
• M5Stack Cardputer (ESP32-S3)
• Wi-Fi access for Cardputer

---

🧰 1. Install Ollama

1. Install Ollama on the Pi: curl -fsSL https://ollama.com/install.sh | sh

2. Pull a lightweight model: ollama pull tinyllama

3. Start the Ollama server: ollama serve &

4. Test it locally: curl -X POST http://localhost:11434/api/generate
   -H "Content-Type: application/json"
   -d '{"model": "tinyllama", "prompt": "Hello!", "stream": false}'

---

🔐 2. Install and Configure Apache

1. Install Apache and required modules: sudo apt install apache2 apache2-utils sudo a2enmod ssl proxy proxy_http headers auth_basic rewrite

2. Set up basic HTTP authentication: sudo htpasswd -c /etc/apache2/.htpasswd theotherone

---

🌐 3. Enable SSL with Let's Encrypt

1. Install certbot: sudo apt install certbot python3-certbot-apache

2. Run the setup: sudo certbot --apache

---

🛡️ 4. Secure Reverse Proxy Configuration

1. Edit the main SSL virtual host: sudo nano /etc/apache2/sites-available/000-default-le-ssl.conf

2. Add this inside <VirtualHost *:443>:

   Redirect /evilOllama to Ollama locally

   RedirectMatch 301 ^/evilOllama$ /evilOllama/

   ProxyPass /evilOllama/ http://localhost:11434/ ProxyPassReverse /evilOllama/ http://localhost:11434/

   <Location /evilOllama/> AuthType Basic AuthName "Evil Ollama Access" AuthUserFile /etc/apache2/.htpasswd Require valid-user RequestHeader set Authorization expr=%{HTTP:Authorization}

3. Reload Apache: sudo systemctl reload apache2

---

🧪 5. Test the Secured Endpoint

curl -u userchoosen:YourPasswordHere
-X POST https://yourdomain.com/evilOllama/api/generate
-H "Content-Type: application/json"
-d '{"model": "tinyllama", "prompt": "Hello evil server!", "stream": false}'

✅ You should receive a JSON response from tinyllama.

---

🎉 Ready for Cardputer!