Skip to content

Adding Features

Jump to bottom
Paige Quarterman edited this page May 3, 2026 · 1 revision

Adding Features

Recipes for the most common changes. These point at the in-tree how-tos for full step-by-step detail.

Add a new search source (journal)

docs/adding-a-search-source.md has the full walkthrough. Two paths:

  • Permanent (everyone gets it): edit src/config/defaults/search-sources.json. Add an object to the sources array with a unique id. Done.
  • Personal (just you): edit data/user_config/search-sources.json (gitignored). The schema-aware merge keys on id, so a {"sources":[{"id":"existing-id", "enabled":true}]} override flips just that one field.

Three source types: arxiv (direct arXiv API), arxiv-jr (arXiv filtered by journal_ref), crossref (Crossref by ISSN). Each requires different fields — see the schema at src/config/schema/search-sources.schema.json.

Add a new config key to an existing domain

docs/adding-a-config-key.md has the long version. Short version:

  1. Edit the schema (src/config/schema/<domain>.schema.json) — type, validation, description.
  2. Edit the defaults (src/config/defaults/<domain>.json) — sensible default value.
  3. Read it: getConfig('<domain>').<field> from JS, load_config('<domain>').data['<field>'] from Python.
  4. Settings v2 picks up the new field automatically (schema-driven UI).

That's it. No JS service registration, no Python loader update — the manifest already covers all 9 domains.

Add a brand-new config domain

Same recipe, plus:

  • Append the domain name to MANIFEST in two places: src/config/loader.js and scq/config/user.py.
  • (Optional) Add a starter data/user_config/<domain>.json.example so users know the shape.
  • Settings v2 reads the schema automatically; opt out by removing the entry from TABS in src/ui/settings/main.js.

Add a new service

Services live in src/services/ and are DOM-free. The rule: take state, return data. No document, no window, no addEventListener. Pattern:

// src/services/foo.js
export function fooBar(input) {
  // pure logic
}

Test it:

// src/tests/services/foo.test.js
import { fooBar } from '../../services/foo.js';
import { describe, it, expect } from 'vitest';
// ...

If the service needs the database, import from core/db.js (high-level helpers like run, query) — not from the legacy db_utils.js IIFE.

Add a new UI module

UI modules live in src/ui/<page>/ and may touch the DOM. Pattern:

// src/ui/database/foo.js
import { fooBar } from '../../services/foo.js';

export function showFooModal() {
  // DOM stuff
}

Then add it to the page bridge in src/ui/database/main.js:

const BRIDGE = {
  // ...existing entries...
  showFooModal,
};

…and to EXPECTED_BRIDGE_KEYS in src/tests/ui/database/bridge.test.js. The frozen-list spec will fail if you skip this, with a clear "Added without updating list: showFooModal" error.

For scraper modules, the bridge is per-module: append globalThis.showFooModal = showFooModal; at the bottom of the new module file, and add the name to EXPECTED_SCRAPER_BRIDGE in src/tests/ui/scraper/bridge.test.js.

Add a new scq subcommand

Two patterns depending on whether the underlying module has its own argparse:

Module has its own main(argv): add to _PASSTHROUGH_COMMANDS in scq/cli.py:

"my-cmd": _passthrough_module("scq.my_module", supports_argv=True),

Also add to _PASSTHROUGH_MODULES (the docstring-help map). The CLI test's drift catcher fails if you forget either.

Native scq config <subcommand> style: add an add_parser block to _build_parser in scq/cli.py with set_defaults(func=_cmd_my_subcmd), and write the handler.

Migrate user customizations from scraper_config.js

If you (or a contributor) have an old scraper_config.js with custom sources/presets/tags, run:

scq migrate-from-legacy --dry-run     # see what would be written
scq migrate-from-legacy                # actually write

Requires Node.js (the legacy file is a JS object literal, not JSON, so we eval it). Writes to data/user_config/search-sources.json and data/user_config/auto-tag-rules.json without overwriting existing files (use --overwrite to replace).

entryTypes and the inline formatBibTeX aren't migrated — those still ship in scraper_config.js itself.

Add a story to the dev harness

The harness at dev.html is for iterating on individual UI modules. Add a story:

  1. Edit (or create) a file under src/dev/stories/.
  2. Each story is {id, title, description, render(stage, setState)}.
  3. The story's setState mirrors current state into the dump pane — useful for watching a form value change.

The harness is hash-routed; reloading after an edit lands you back where you were.

Where the recipes don't quite fit

  • A genuinely new architectural seam. Open a plan file in plans/ (gitignored, so it's a personal working artifact) and follow the patterns from the archived ones at plans/archive/architecture-refactor.md. Don't extend the archived plan — start a new file.
  • A breaking change to the config schema. Bump the schema's $id if you must, but prefer additive changes — old user_config files should still load. Schema validation runs every time the loader fires, so a backward-incompatible change is loud at the seam.
  • A change that touches both halves. Lock the parity with a vector in tests/vectors/<category>/. Both runners pick it up automatically. See Architecture for the testing topology.

Clone this wiki locally