Skip to content

Contributing to the Wiki

Jump to bottom Edit New page
Joel Natividad edited this page May 30, 2026 · 4 revisions

Contributing to the Wiki

Tier: Advanced

This wiki is itself a git repo at https://github.com/dathere/qsv.wiki.git. Anyone with write access to qsv can clone it, edit pages locally, and push to master. There's no PR review (GitHub Wikis don't support PRs) — so we rely on conventions and a verification checklist instead.

Quick start

git clone https://github.com/dathere/qsv.wiki.git ~/code/qsv.wiki
cd ~/code/qsv.wiki

# Edit pages
$EDITOR Recipe-My-New-Recipe.md

# Commit and push
git add Recipe-My-New-Recipe.md _Sidebar.md
git commit -m "wiki: add Recipe-My-New-Recipe"
git push origin master

Pages appear on the live wiki immediately after the push.

File layout

qsv.wiki/
├── Home.md                              ← landing page
├── _Sidebar.md                          ← left-side nav (rendered specially by GitHub)
├── _Footer.md                           ← bottom footer (rendered specially)
├── _Header.md                           ← reserved for future CSS hooks
├── <Page-Name>.md                       ← every other page; one per file
├── files/                               ← binary assets (CSVs, ZIPs, images)
└── images/                              ← screenshots, charts (optional)

All pages live at repo root with no subdirectories (GitHub Wiki convention). Filenames become URL slugs: Recipe-Foo-Bar.md → https://github.com/dathere/qsv/wiki/Recipe-Foo-Bar and renders as Recipe Foo Bar.

Page conventions

Required structure

Every page must:

  1. Start with an H1 matching the slug humanized: # Recipe: Foo Bar
  2. Declare a tier on line 3: **Tier:** [Beginner|Intermediate|Advanced](Tier-Legend)
  3. End with a ## See also section with 3-5 outbound links

Templates

Command-reference category page

# {{Category Title}}

**Tier:** [{{Tier}}](Tier-Legend)
**Commands covered:** `cmd1`, `cmd2`, … `cmdN`

> [!NOTE]
> Per-command flag reference lives in [/docs/help/](https://github.com/dathere/qsv/tree/master/docs/help).
> This page is the *workflow* layer — when to reach for each command and how they compose.

## Quick decision table
| If you want to… | Use | Notes |
|---|---|---|

## `cmd1`
{{1-paragraph intro}}

**Example: {{scenario with anchor dataset}}**

```bash
qsv {{cmd1}} {{args}} {{anchor_file}}

{{1-paragraph commentary.}}

See also: /docs/help/cmd1.md, Recipe-X

See also

  • …

#### Cookbook recipe page

```markdown
# Recipe: {{Title}}

**Tier:** [{{Tier}}](Tier-Legend)
**Commands used:** `cmdA`, `cmdB`, `cmdC`
**Anchor dataset:** {{name}}

## Problem
## Data
## Solution
## Variations
## Performance notes
## See also

When to update

Trigger Update
A new command is added to qsv Add 1 row to Command-Reference + 1 H2 section in the matching category page in Phase B
A flag is renamed Grep qsv.wiki for the old flag name; update each reference
A new feature in an existing command Add a new example to the category page; add to the related recipe(s) if relevant
A new performance number is published Don't hardcode the number in the wiki. Update docs/BENCHMARKS.md instead — the wiki links to it
A new env var Add to Environment-Variables under the right category section. Source of truth is docs/ENVIRONMENT_VARIABLES.md
A new cookbook recipe Create a Recipe-Foo-Bar.md from the template above, add to _Sidebar.md, and add a row to Cookbook

Verification checklist

Before pushing, run through this list:

  • git clone and render locally with gh repo view, glow, or your editor's Markdown preview.
  • Click every link on the changed page — no 404s, no relative-vs-absolute confusion.
  • Run every code block against its declared anchor dataset. If the block is mocked or illustrative, mark with <!-- illustrative --> comment.
  • Verify fenced language is correct: bash, sql, lua, python, json, toml, jinja.
  • Confirm the new page is in _Sidebar.md in the right tier section.
  • Confirm the "See also" footer has 3-5 outbound links.
  • Confirm the tier badge is at the top of the page, linking to Tier-Legend.
  • No duplicated content from /docs/help/<cmd>.md (paraphrase + link instead).
  • Performance numbers cite their source (docs/BENCHMARKS.md or qsv.dathere.com/benchmarks).
  • Commit message follows the convention (next section).

Commit message convention

Use a wiki: prefix:

  • wiki: add Recipe-Foo-Bar page
  • wiki: update Performance-Tuning to link new BENCHMARKS section
  • wiki: fix dead link in Command-Reference
  • wiki: expand Cookbook with Recipe-Larger-than-RAM

For multi-file changes, summarize in the subject and itemize in the body:

wiki: Phase B batch 2 — Aggregation + Joins category pages

Aggregation-and-Statistics covers stats, moarstats, frequency, …
Joins-and-Set-Ops covers join, joinp, exclude, partition, split …

Don't duplicate /docs/

The wiki's job is workflow + discoverability. The /docs/ tree is the canonical reference. Avoid duplication:

The only exceptions are the Why-qsv page (where some standalone numbers are quoted directly for the pitch) and any legacy content preserved on Cookbook.

Anchor dataset conventions

When you need a dataset for an example, prefer (in order):

  1. wcp.csv — World Cities Population, in files/wcp.zip
  2. NYC 311 — bundled at resources/test/NYC_311_SR_2010-2020-sample-1M.csv
  3. Boston 311 — bundled at resources/test/boston311-*.csv
  4. Allegheny property sales — bundled at resources/test/allegheny_property_sales*.csv
  5. NOAA GHCN-Daily — fetched live via qsv fetch
  6. GitHub stargazers — fetched live via qsv fetch
  7. country_continent.csv — for join examples

The same dataset appearing in multiple pages helps users build mental models. See the anchor table on Home and Cookbook.

Linting

The wiki ships a small bash lint at scripts/lint.sh. It does two checks:

  1. Command-reference check. Scans every page for qsv <command> patterns inside code contexts (fenced blocks + inline backticks) and verifies each <command> is in qsv --list.
  2. Help-link check (optional). When QSV_REPO_PATH is set, scans every page for /docs/help/<X>.md references and verifies each X.md exists in the named qsv checkout.

Run it locally

# Just the command-reference check
./scripts/lint.sh

# Both checks — point at a local qsv source checkout
QSV_REPO_PATH=~/code/qsv ./scripts/lint.sh

Exit codes:

  • 0 — clean
  • 1 — stale or missing references
  • 2 — environment problem (qsv missing, qsv --list unparseable, QSV_REPO_PATH invalid)

Run it in CI

GitHub Actions doesn't run on *.wiki repos directly, so the workflow file lives in the main qsv repo. Copy scripts/wiki-lint-workflow.yml from the wiki to the main repo at .github/workflows/wiki-lint.yml and push.

The workflow runs:

  • Nightly at 04:17 UTC
  • On every PR that touches src/cmd/, src/main.rs, or docs/help/
  • On manual dispatch (workflow_dispatch)

It builds qsv with -F all_features from the PR's source, clones the wiki, and runs the lint. Scheduled failures open a sticky wiki-lint issue (or comment on the existing one) so stale references can't quietly persist.

Suppressing false positives

If a literal qsv <token> appears in a doc that isn't a real command invocation (template placeholder, prose-as-comment in a bash block, etc.), add the token to scripts/lint.allowlist (one per line; # for comments). Don't push false positives through; future contributors will assume they're real.

For /docs/help/<X>.md placeholders (e.g. cmd1.md in template snippets), the script's HELP_ALLOWLIST already includes the common ones (cmd1, cmd2, cmdA-C, X).

Slug rules

GitHub Wiki renders Use-Hyphens-In-Title-Case.md as the page title Use Hyphens In Title Case. So:

  • Recipe-Clean-and-Normalize.md → "Recipe Clean and Normalize"
  • Recipe-CKAN-Integration.md → "Recipe CKAN Integration"
  • For non-ASCII or special characters, URL-encode in cross-page links: [xsv vs qsv stats](xsv-0.13.0-%60stats%60-compared-with-qsv-1.0.0-%60stats%60)

Always use hyphens, never spaces or underscores, in filenames.

Assets

  • Images / screenshots — store in images/ at the repo root. Reference relatively: ![Sidebar](images/sidebar.png).
  • Sample CSVs — store in files/. Reference relatively: [wcp.zip](files/wcp.zip).
  • Benchmark SVGs — DO NOT copy from /docs/ into the wiki. Link to raw URL instead: https://raw.githubusercontent.com/dathere/qsv/master/docs/qsv-benchmarks.svg.

Asciinema casts (future)

For future demos, host on asciinema.org and embed via the standard <script> tag. Test that GitHub Wiki renders the script — if it gets stripped, fall back to a GIF in images/.

Plan reference

The full Phase A/B/C/D/E wiki-expansion plan lives at the contributor's local plan file (or in .claude/plans/the-wiki-is-a-typed-dragon.md if you cloned that with the wiki). This page is the maintenance layer; the plan was the creation layer.

See also

qsv — GitHub · Releases · Discussions · qsv pro · Try it online · Benchmarks · datHere · DeepWiki · Dual-licensed MIT / Unlicense

Edit this page: Contributing to the Wiki

Home · Why qsv? · Tier legend

Get Started

Command Reference

Cookbook

Tuning & Internals

Ecosystem

Reference

Legacy

Clone this wiki locally