Skip to content

LLM8850 Image Generation LCM

Jump to bottom
JdaieLin edited this page Mar 11, 2026 · 5 revisions

The LLM8850 AI accelerator can run a Latent Consistency Model (LCM) based on Stable Diffusion 1.5 for on-device image generation. LCM is a fast, distilled variant of diffusion models that generates images in very few steps (typically 4–8), making it practical on embedded hardware.

When configured, the chatbot can generate images on demand during a conversation — the user can ask "generate an image of a sunset", and the image will be generated and displayed on the Whisplay HAT screen automatically.

⚠️ Important: The LCM image generation service uses a significant amount of memory (~2 GB+). It is not possible to run Qwen3 / Qwen3-VL LLM alongside the LCM service on the same LLM8850 device at the same time. You must choose an alternative LLM backend:

  • An online LLM service (e.g. OpenAI, Gemini, Grok, Perplexity), or
  • A local Ollama instance running on Pi5, or
  • The special image-tool-direct mode, which bypasses the LLM entirely and sends user text straight to the image generation tool (no conversation ability, image generation only).

Prerequisites

  • LLM8850 (AX650N-based) device
  • SD card ≥ 64 GB — the model repository is over 20 GB
  • git-lfs installed (sudo apt install -y git-lfs && git lfs install)
  • Python 3 with pip

Step 1 — Clone the Model Repository

Clone the official AXERA LCM repository from HuggingFace onto your device:

cd ~
git clone https://huggingface.co/AXERA-TECH/lcm-lora-sdv1-5
cd lcm-lora-sdv1-5

Note: The repository is over 20 GB. Make sure you have enough free space on your SD card (64 GB or larger recommended). The clone may take a while depending on your network speed.

I found that the requirements in the repository are very strict and may cause installation issues. I recommend removing all the version constraints in requirements.txt before installing:

sed -i 's/==[0-9\.]*//g' requirements.txt

Install the Python dependencies:

sudo apt install cmake -y
pip install -r requirements.txt --break-system-packages
pip install https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc2/axengine-0.1.3-py3-none-any.whl --break-system-packages

Try running the model locally to verify the installation:

python3 launcher.py -o "ax650n_txt2img_axe.png" --prompt "Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"

Step 2 — Download serve.py

Download the serve.py file into the lcm-lora-sdv1-5 directory. This wraps the inference pipeline into a persistent HTTP service with a /generate endpoint so the model is loaded only once and can serve multiple requests. Each image generation request takes no more than 3 seconds after the initial load.

curl -o serve.py https://storage.whisplay.ai/lcm-lora-sdv1-5/serve.py

The server provides two endpoints:

  • GET /health — returns {"status": "ok"} (for health checks)
  • POST /generate — generates an image from a text prompt

POST /generate request body (JSON):

Field Type Required Description
prompt string Yes Text description of the image to generate
return_base64 boolean No Default true. Return the image as base64 in the response
init_image_base64 string No Base64-encoded source image for img2img generation
init_image_path string No Local file path to a source image for img2img generation
seed number No Random seed for reproducible generation
isize string No Image size, e.g. "512" or "1024x768"

POST /generate response body (JSON):

Field Type Description
image_base64 string Generated image as base64-encoded PNG (when return_base64 is true)
seed number Seed used for this generation
width number Width of the generated image in pixels
height number Height of the generated image in pixels
save_path string | null Server-side save path, if provided in the request

You can test the service manually:

# start the server (default port 8806)
python3 serve.py --port 8806

In another terminal, run the health check and generation commands:

# Health check
curl http://localhost:8806/health

# Generate an image
curl -X POST http://localhost:8806/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a cat sitting on a windowsill", "return_base64": true}' | python3 -c "
import sys, json, base64
data = json.load(sys.stdin)
with open('test.png', 'wb') as f:
    f.write(base64.b64decode(data['image_base64']))
print(f'Saved test.png ({data[\"width\"]}x{data[\"height\"]}, seed={data[\"seed\"]})')
"

Step 3 — Download startup.sh (Auto-Start on Boot)

Download the startup.sh file into the same directory. This script installs a systemd service so the LCM HTTP server starts automatically on boot.

curl -o startup.sh https://storage.whisplay.ai/lcm-lora-sdv1-5/startup.sh

Make it executable and run it with sudo:

chmod +x startup.sh

To customize the port or other options:

sudo ./startup.sh --port 8806 --isize 512 --python-bin /usr/bin/python3

To uninstall the service later:

sudo ./startup.sh --uninstall

Step 4 — Configure Whisplay AI Chatbot

Set IMAGE_GENERATION_SERVER=llm8850lcm in your .env file:

IMAGE_GENERATION_SERVER=llm8850lcm

# Host and port of the LCM service (default: 127.0.0.1:8806)
LLM8850LCM_HOST=127.0.0.1
LLM8850LCM_PORT=8806

Since the LCM service cannot run alongside Qwen3/Qwen3-VL on the same LLM8850, you need to choose a different LLM backend. Here are the recommended options:

LLM Backend Options

Option A — Online LLM / local LLM with tool calling (e.g. Gemini, OpenAI, Grok, Perplexity, ollama)

Use a cloud LLM that supports tool/function calling. The LLM will automatically trigger the image generation tool when the user asks for an image.

ASR_SERVER=llm8850whisper
LLM_SERVER=gemini
TTS_SERVER=llm8850melotts

IMAGE_GENERATION_SERVER=llm8850lcm

LLM8850_WHISPER_HOST=http://localhost:8801
LLM8850_MELOTTS_HOST=http://localhost:8802
LLM8850LCM_HOST=127.0.0.1
LLM8850LCM_PORT=8806

GEMINI_API_KEY=your_gemini_api_key

Option B — Image-Tool-Direct Mode (No LLM, Image Generation Only)

If you only need image generation without conversation ability, use the image-tool-direct LLM mode. This bypasses the LLM entirely and sends the user's spoken text straight to the image generation tool as the prompt.

ASR_SERVER=llm8850whisper
LLM_SERVER=image-tool-direct
TTS_SERVER=llm8850melotts

IMAGE_GENERATION_SERVER=llm8850lcm

LLM8850_WHISPER_HOST=http://localhost:8801
LLM8850_MELOTTS_HOST=http://localhost:8802
LLM8850LCM_HOST=127.0.0.1
LLM8850LCM_PORT=8806

In this mode, everything you say is treated as an image generation prompt. The chatbot will respond with a fixed reply like "The image has been generated, please check on the screen." after each generation. This is the simplest setup if you only want a voice-controlled image generator.

How It Works

When IMAGE_GENERATION_SERVER=llm8850lcm is set, Whisplay registers two LLM tools automatically:

  1. generateImage — Generates a new image from a text prompt. The LLM calls this tool when the user asks for an image.
  2. showPreviouslyGeneratedImage — Displays the most recently generated image again.

The LCM tool also supports image-to-image (img2img) generation. If the user says something like "make it look like a painting" while an image is already displayed on screen, the tool attaches the current image as the init_image_base64 input to steer the generation.

Generated images are saved in the data/images/ folder on the chatbot device and displayed on the Whisplay HAT screen automatically.

Full .env Reference

Variable Required Default Description
IMAGE_GENERATION_SERVER Yes (none) Set to llm8850lcm to enable LCM image generation
LLM8850LCM_HOST No 127.0.0.1 Hostname or IP of the LCM service
LLM8850LCM_PORT No 8806 Port of the LCM service

Notes

  • Memory constraint: The LCM service uses significant memory. It cannot coexist with the Qwen3 / Qwen3-VL LLM services on the same LLM8850 device. Stop the Qwen3 service before starting the LCM service, or use a cloud LLM instead.
  • SD card size: The model repository (lcm-lora-sdv1-5) is over 20 GB. Use a 64 GB or larger SD card.
  • Generated images are stored in data/images/ and displayed automatically on the Whisplay HAT display.
  • The chatbot tracks the most recently displayed image (whether generated or captured by camera). This image is used as init_image_base64 input when the user asks for image edits.
  • Generation speed depends on the image resolution. At 512×512 on LLM8850, expect a few seconds per image.
  • The LCM service must be started before the chatbot. If the service is unavailable, image generation requests will fail (the chatbot continues to work normally for text conversations).
  • The service initialization takes some time after boot. Wait about 30–60 seconds after system startup before requesting image generation.

Clone this wiki locally