feat: add cursor pagination to GET /api/threads and thread_list MCP tool

[bertheto]

feat: add cursor pagination to GET /api/threads and thread_list MCP tool (UP-20)

Problem

GET /api/threads and the thread_list MCP tool return all threads with no limit. For bus instances with a large thread history, this causes unnecessary memory usage and latency on every listing request — even when the caller only needs the most recent page.

By comparison, GET /api/threads/{id}/messages and the msg_list MCP tool already support cursor pagination via after_seq + limit. This PR extends the same pattern to thread listing.

---

Solution

Add optional limit and before parameters to GET /api/threads and the thread_list MCP tool. When omitted, the behaviour is identical to the current implementation (all threads returned). When provided, the response is paged using a keyset cursor on created_at.

The response is now an envelope object instead of a flat array:

{
  "threads": [
    {"id": "...", "topic": "...", "status": "...", "system_prompt": "...", "created_at": "..."}
  ],
  "total": 42,
  "has_more": true,
  "next_cursor": "2026-03-02T10:00:00.123456+00:00"
}

Field	Description
threads	Current page of threads, ordered newest-first
total	Total thread count matching the filters (without pagination)
has_more	true when more pages are available
next_cursor	ISO datetime to pass as before for the next page. null when has_more is false

Pagination parameters

Parameter	Type	Default	Description
limit	integer	0	Max threads to return. 0 = all (no limit). Hard cap: 200.
before	string	—	ISO datetime cursor. Returns threads created strictly before this timestamp. Pass next_cursor from a previous response.

Example: walking pages

GET /api/threads?limit=50
→ { threads: [...50], total: 142, has_more: true, next_cursor: "2026-01-15T..." }

GET /api/threads?limit=50&before=2026-01-15T...
→ { threads: [...50], total: 142, has_more: true, next_cursor: "2026-01-10T..." }

GET /api/threads?limit=50&before=2026-01-10T...
→ { threads: [...42], total: 142, has_more: false, next_cursor: null }

---

Implementation details

• src/db/crud.py: thread_list() refactored to a single dynamic query builder (replaces 3 duplicated SQL paths). New thread_count() function for total count without pagination.
• src/db/database.py: Migration adds idx_threads_created_at ON threads(created_at) for efficient keyset queries.
• src/main.py: GET /api/threads updated. + in timezone offset (+00:00) is normalized from URL-decoded space before parsing. Returns 400 on invalid before format.
• src/mcp_server.py: thread_list tool schema updated with limit and before.
• src/tools/dispatch.py: handle_thread_list() forwards new params, returns envelope.
• src/static/js/shared-threads.js: Updated to unpack response.threads from the envelope.

Backward compatibility

• limit=0 (default) preserves the current "return all" behaviour.
• The response shape changes from a flat array to an envelope object. Existing clients must access response.threads instead of the response directly.

Tests

20 tests in tests/test_thread_pagination.py:

• 13 unit tests (in-memory SQLite): limit, before cursor, combined, ordering, status filter, include_archived, hard cap, sequential page walk without overlap, empty cursor result, thread_count semantics.
• 7 integration tests (live server): envelope backward compat, limit, before cursor, pagination walk (2-by-2 across 5 threads), hard cap, status+limit combined, invalid cursor 400.

Full test suite: 201 tests, all passing.