systems api server

API server

Active contributors: Magnus Hedemark

Purpose

FastAPI HTTP server that implements the SearXNG-compatible search API with graceful degradation. Manages engine lifecycle (startup, warmup, shutdown), concurrent dispatch, and response formatting.

Key abstractions

Type	File	Description
`app`	`slopsearx/server.py`	FastAPI application instance. Title: "SlopSearX", version "0.1.0". Hosts all endpoints and lifecycle handlers.
`_active_engines`	`slopsearx/server.py`	Module-level dict of enabled engine instances. Populated at startup via `discover_engines()`.
`_ranker`	`slopsearx/server.py`	Module-level `PresenceRanker` instance. Used by every search request to merge and rank results.
`_cache`	`slopsearx/server.py`	Module-level `SearchCache` instance. Initialized at startup. Gracefully degrades if Valkey is unavailable.
`_rate_limiter`	`slopsearx/server.py`	Module-level `RateLimiter` instance. Injected into each engine adapter at startup.
`search()`	`slopsearx/server.py`	`GET /search` handler. Accepts all standard SearXNG query parameters. Returns JSON by default or YAML+Markdown with `format=yaml`.
`health()`	`slopsearx/server.py`	`GET /health` handler. Runs per-engine health checks concurrently and returns aggregate status. Returns 200 even if some engines are unhealthy.
`metrics()`	`slopsearx/server.py`	`GET /metrics` handler. Returns OpenMetrics text via `render_metrics()`.
`config()`	`slopsearx/server.py`	`GET /config` handler. Returns the categories-to-engines mapping for runtime discovery, built from instantiated engines.
`_dispatch_engine()`	`slopsearx/server.py`	Dispatches a query to one engine with a 3-second timeout. Returns `AdapterResponse` and never raises.
`startup()`	`slopsearx/server.py`	FastAPI lifespan event handler. Discovers engines, initializes cache and rate limiter, injects rate limiter into engines, and warms up all engines concurrently.
`shutdown()`	`slopsearx/server.py`	FastAPI lifespan event handler. Gracefully shuts down all engines, cache, and rate limiter.

How it works

Full search request flow

1. GET /search?q=python&format=json&categories=general
2. Validate query (return 400 if empty)
3. Determine target engines:
   - If `engines` param is set, filter to those engines
   - Otherwise, use all active engines
   - If `categories` param is set, filter engines by category membership
4. Return 503 if no engines are available
5. Check Valkey cache:
   - HIT: return cached response immediately (~2ms)
   - MISS: continue
6. Dispatch to all target engines concurrently via asyncio.gather()
7. For each engine, _dispatch_engine() calls engine.search() with 3s timeout
8. Collect AdapterResponse objects
9. Record per-engine metrics (query count, latency, status)
10. Pass engine results to PresenceRanker.rank() for dedup and ranking
11. Build metadata (response time, engine status, unresponsive list)
12. Format response (JSON or YAML+Markdown)
13. Cache merged result set (skip if all engines unresponsive)
14. Return response with appropriate status code (200 or 503)

Engine selection logic

The server supports two modes of engine selection:

Explicit engine list: When the engines query parameter is provided, only those engines are queried. Category filters are ignored.
Category-based selection: When no explicit engine list is provided, engines are filtered by requested categories. An engine is included if it declares any of the requested categories.

Timeout mechanism

Each engine dispatch has a 3-second timeout enforced by asyncio.wait_for():

result = await asyncio.wait_for(engine.search(query, params), timeout=timeout_s)

If an engine exceeds the timeout, _dispatch_engine() returns an AdapterResponse with EngineStatus.TIMEOUT and a latency of 3000ms. This prevents a single slow engine from delaying the entire response.

Error responses

Status	Condition
400	Missing or empty `q` parameter. Body: `{"error": "query_required", "message": "..."}`
503	No target engines found or all engines returned non-OK status. Body includes empty results and unresponsive engine list.
429	Client-side rate limiting (optional, not yet implemented).
200	All other cases, including partial failures. Failing engines are reported in `unresponsive_engines`.

The system never returns 500 for a valid request. All engine errors are caught and classified.

Lifecycle

Startup: The startup() event handler runs these steps in order:

Initialize SearchCache (gracefully degrades if Valkey is unavailable)
Initialize RateLimiter with LocalTokenBucket strategy (defaults to dev mode)
Warm up the rate limiter
Load config and discover engines (if not already populated for test fixtures)
Inject the rate limiter into each engine
Concurrently warm up all engines

Shutdown: The shutdown() event handler runs:

Concurrently shut down all engines
Shut down the rate limiter

Endpoints

Endpoint	Method	Description
`GET /search`	Search	Main search endpoint. Parameters: `q`, `format`, `categories`, `engines`, `language`, `pageno`, `time_range`, `safesearch`.
`GET /health`	Health check	Per-engine health status. Returns `{"status": "ok"}` or `{"status": "degraded"}`.
`GET /metrics`	Metrics	OpenMetrics text format for Prometheus scraping.
`GET /config`	Config discovery	Returns `{"categories": {"general": ["brave", ...], ...}}`.

Integration points

Engine adapters: The server imports engines at module level to trigger @register_engine decoration before startup
Config system: load_config() is called at startup to resolve the layered configuration
Cache: SearchCache is checked before dispatch and written after merging
Rate limiter: Injected into every engine adapter instance during startup
Metrics: Recorded on every search request for per-engine observability

Entry points for modification

Adding a new endpoint: add a FastAPI route decorator and handler in server.py
Changing the timeout: modify the timeout_s parameter in _dispatch_engine() signature
Modifying engine selection: change the filter logic in search() handler
Adjusting startup sequence: modify startup() event handler