v0.3.0

Highlights

Ready for the open web

v0.3.0 is the release that makes it reasonable to run indice as a public server that strangers can read and approved people can change.

• Replay runs on an origin of its own. Archived pages run their own JavaScript. indice now serves replay from a second listener and hostname, so a captured page's script can't act as the person reading it or reach indice's write API. Each origin sends its own Content-Security-Policy.
• Three roles, from a file you can commit. users.yaml lists who is a curator (adds and describes) and who is an admin (removes, moderates, and imports through the instance's Browsertrix and Archive-It credentials). Anyone else is a reader, signed in or not. indice re-reads the file when it changes, so approving someone needs no restart. A server refuses to start without one.
• Custody and an audit trail. Each crawl records who accessioned it, and a curator can undo their own additions. An append-only log under events/ records who changed what.
• Limits on what anyone can ask of a server. Request limits per client and overall, bounded uploads (uploads.max_mb, uploads.max_total_gb), and crawls added by URL or upload only, never by a path on the server's disk.
• Privacy. Readers' searches and addresses stay out of the logs, a public failure answers with a reference rather than the error, and replay and search results are kept out of search engines.

Read Securing a server and the deploy guide before putting one on the open web.

A new look

The reading room and the workroom now look different on purpose: a manila header and accents when you can change the archive, the archive's own words in a serif, time drawn as a real axis on search and on collections, and readable names for crawls. Colours, faces and spacing are design tokens by role (tokens.css), in a layer meant for theming.

Name your archive

config.yaml gains a site: section: the archive's name (header, homepage, page titles, replay viewer), a tagline, a Markdown banner for every page, and request_access, shown to someone signed in but not on the roster. See Site name, tagline and banner.

Notes, found again

A note's passage is highlighted when you open its page, clicking a note scrolls to it, and links from the notes list and from search results open the page at the note.

Indexing large WACZs

A WACZ is indexed a page at a time, so memory no longer grows with the crawl, and no file inside a WACZ is inflated past index.max_member_mb (default 1 GB), so a small WACZ can't unpack into gigabytes. indice reconcile checks that the index and the manifest agree, and says how to repair them.

Learn Rust through indice

The Rust primer is rewritten as a series on the docs site, teaching Rust through indice's own code.

Upgrading

• No reindex is needed. Existing search indexes open as they are.
• Downgrading needs one. The search engine moved to tantivy 0.27, which records a few fields' settings in a new form the first time it writes to an index. v0.2.0 then refuses that index as having "an older schema"; indice reindex rebuilds it.
• On a workstation, --manage is gone. Bound to loopback, the operator always has the workroom, so drop the flag: indice serve --manage now fails with an unknown argument. Replay listens on the next port up (8081 beside 8080, or --replay-bind), which must be free.
• A server needs more than it did:
  • --site-url, the address readers use. indice no longer infers it from request headers.
  • A second hostname for replay, given as --replay-url (and --replay-bind for its listener), different from the site's host.
  • A users.yaml. An empty users: [] is a read-only archive.
  • The shipped stack has changed shape. compose.yaml adds oauth2-proxy, for sign-in through any OIDC provider, and Caddy serves a second site for replay. If you run the stack, take compose.yaml and Caddyfile from this release and follow the deploy guide.
• The HTTP API no longer accepts a server-side path for a new crawl: add one by http(s) URL or upload. indice index on the command line still takes local files.

The container image is ghcr.io/edsu/indice:v0.3.0 (also :latest), for linux/amd64 and linux/arm64.

Full changelog: v0.2.0...v0.3.0