-
Notifications
You must be signed in to change notification settings - Fork 14
Dev Backend API
The geniusai-server exposes a REST API over HTTP (default port 19819). All responses use JSON with the structure { "results": ..., "error": ..., "warning": ... }.
This reference is intended for developers integrating with or extending the backend. Regular users interact through the Lightroom plugin and do not need to call these endpoints directly.
Minimal liveness check. Returns {"status": "ok"}.
Detailed health check. Returns server status, CLIP model cache state, and configured provider availability.
Returns the running backend version string.
Checks whether the backend version is compatible with the plugin version passed in the request body.
Called by the Lightroom plugin on first connect. Accepts catalog/configuration parameters and initializes per-catalog state.
Returns the list of available AI models grouped by provider (gemini, chatgpt, ollama, lmstudio, llamacpp, mlx). Filters out providers that are not configured or not reachable; the two local backends report what is installed on disk, so they are empty when no local model has been downloaded.
Returns recent log lines from the server log.
Streams raw log file content. log_type can be server or plugin.
Gracefully shuts down the backend process.
Restarts the backend process. Note: this endpoint is known to be unreliable on Windows (see Troubleshooting).
Unloads heavy ML models from memory without shutting the server down (useful to reclaim VRAM/RAM between sessions).
Triggers an in-place backend self-update from the latest GitHub release.
Indexes a batch of photos sent as multipart file uploads. Generates embeddings and/or AI metadata depending on options.
Key request fields:
| Field | Type | Description |
|---|---|---|
photo_id |
string | Stable file-based photo identifier |
catalog_id |
string | Lightroom catalog identifier |
provider |
string | LLM provider (gemini, chatgpt, ollama, lmstudio, llamacpp, mlx) |
model |
string | Model name within the provider (for llamacpp the GGUF file name, for mlx the model directory name) |
llm_n_ctx |
int |
llamacpp only: context window override (0/absent = default) |
llm_n_parallel |
int |
llamacpp only: photos decoded concurrently |
llm_gpu_layers |
int |
llamacpp only: layers offloaded to the GPU (0 = CPU only) |
generate_metadata |
bool | Generate keywords/title/caption/alt_text |
create_embeddings |
bool | Create SigLIP2 semantic embeddings |
detect_faces |
bool | Run InsightFace detection on the photo |
regenerate |
bool | Re-process even if data already exists |
extra_context |
object | Optional context hints (folder, date, GPS, keywords) |
Same as /index but accepts image data as base64 strings rather than file uploads.
Indexes photos using server-side file paths instead of uploading image data (for local-backend setups where the server has filesystem access).
Returns stored metadata and embedding status for one or more photo_id values.
Returns a list of all indexed photo_id values, optionally filtered by catalog.
Given a list of photo_id values, returns which ones are not yet indexed or are missing specific data.
Removes all data (embeddings, metadata, face data) for a given photo_id.
Removes only the AI-generated metadata fields for a given photo_id, leaving embeddings intact.
Removes backend records for photos that no longer exist in a given catalog.
Associates an existing backend record with a (potentially new) catalog ID.
Runs a semantic search query against indexed photos.
Key request fields:
| Field | Type | Description |
|---|---|---|
query |
string | Natural language search query |
catalog_id |
string | Restrict to a specific catalog |
scope_ids |
array | Restrict search to specific photo_id values |
max_results |
int | Maximum number of results to return |
strictness |
float | Minimum similarity score threshold |
use_clip |
bool | Include visual embedding similarity |
use_metadata |
bool | Include keyword/caption/title text search |
Response: results array with photo_id and score fields, sorted by relevance descending.
Finds photos similar to a reference photo using perceptual hash (phash) or CLIP embeddings.
Key request fields:
| Field | Type | Description |
|---|---|---|
photo_id |
string | Reference photo |
mode |
string |
phash or clip
|
scope_ids |
array | Optional photo_id scope |
max_results |
int | Maximum results |
strictness |
string |
strict, normal, or loose (phash mode) |
Groups a set of photos into similarity clusters (used internally by the culling workflow).
Runs the full culling pipeline on a set of photos: grouping, scoring, and classification into picks/alternates/rejects.
Generates a Lightroom develop recipe for a photo sent as a file upload.
Key request fields:
| Field | Type | Description |
|---|---|---|
photo_id |
string | Photo identifier |
provider |
string | LLM provider |
model |
string | Model name |
intent |
string | Style preset key (e.g. natural_pro, moody_dramatic) |
style_strength |
float | 0.0–1.0, how aggressively to apply the style |
composition_mode |
string |
none, subtle, or aggressive
|
instruction_override |
string | Optional per-photo free-text instruction |
Response: A develop recipe object with global adjustments and optional mask list.
Same as /edit but accepts image data as base64.
Internal: applies style training context when building the edit prompt.
Detects faces in an uploaded image and stores their embeddings.
Finds photos containing faces similar to those in a reference image.
Re-clusters all stored face embeddings into person groups.
Returns all detected persons with name, photo count, and optional thumbnails.
Returns the representative face thumbnail for a person as JPEG.
Updates the name assigned to a person cluster.
Returns the list of photo_id values associated with a specific person.
Imports existing Lightroom catalog metadata (keywords, title, caption, rating, etc.) into the backend database for a batch of photos.
Synchronous keyword clustering: groups catalog keywords by semantic similarity.
Async version: starts a background clustering job and returns a job_id.
Polls the status and result of an async clustering job.
Applies a list of approved keyword merge pairs to the backend's metadata records.
Saves a photo's Lightroom develop settings as a labeled training example.
Lists all stored training examples.
Returns aggregated statistics about stored training examples (count per label, coverage).
Returns the total number of stored training examples.
Removes the training example for a specific photo.
Clears all training examples.
Returns whether the SigLIP2 embedding model is downloaded and ready.
Triggers a background download of the CLIP model.
Returns the current progress of an ongoing CLIP model download.
The backend hosts two local inference engines: llamacpp (in-process llama.cpp, GGUF models, present only in builds compiled with the llamacpp cargo feature) and mlx (an lrgenius-mlx helper process, Apple silicon only, no cargo feature). Both are exposed through the same routes — the MLX half is nested under an mlx key so that older plugins reading the top-level fields still see the llama.cpp engine.
Lists local models that are installed and those offered for download.
{
"installed": [{ "name": "...", "model_path": "...", "mmproj_path": "...", "source": "downloaded|env|lmstudio" }],
"downloadable": [{ "id": "gemma4-e4b", "label": "...", "approx_bytes": 0, "min_ram_gb": 16, "installed": false }],
"supported": true,
"model_dir": "~/.cache/lrgenius/models/llm",
"mlx": {
"installed": [{ "name": "gemma-4-e4b-it-4bit", "model_dir": "...", "source": "downloaded|env|lmstudio|huggingface" }],
"downloadable": [{ "id": "mlx-gemma4-e4b", "label": "...", "approx_bytes": 0, "min_ram_gb": 16, "installed": false }],
"supported": false,
"reason": "MLX runs only on Apple silicon Macs. …",
"model_dir": "~/.cache/lrgenius/models/mlx"
}
}supported: false always carries a reason for MLX; for llama.cpp it means the binary was built without the llamacpp feature.
Engine state (status, loaded model_name, …) for the llama.cpp engine, with the MLX engine's equivalent nested under mlx.
Starts a background download of a catalog entry. Body: { "id": "gemma4-e4b" }. The id space is shared across both catalogs and the id alone selects the backend, so there is a single download queue: a GGUF pair is fetched as two files, an MLX entry as a repo snapshot staged into a .part directory and renamed on success.
Progress of the current local-model download (same shape as the CLIP download status).
Returns aggregate database statistics:
- total indexed photos
- photos with SigLIP embeddings
- photos with Vertex AI embeddings
- photos with title / caption / keywords
- total detected faces
- total persons
Creates and streams a ZIP backup of the full persistent LanceDB data directory.
One-time migration endpoint: converts legacy Lightroom UUID-based IDs to photo_id values.
Request body: { "mappings": [{ "old_id": "...", "new_id": "..." }] }
All endpoints return:
{
"results": { ... },
"error": null,
"warning": null
}-
results— the payload (varies per endpoint). -
error— a string if an error occurred,nullotherwise. -
warning— a non-fatal warning string,nullotherwise.
On HTTP error responses (4xx/5xx), error is always set.