Label and catalog names don't have formalized syntax

[octogonz]

We shouldn't allow --label "--label" or --catalog AbCdEF. Also, in some places the CLI is leaking the label ID my-catalog:my-label where a label name is supposed to go.

[octogonz]

Spec for more rigorous syntax

• Revised to clarify that crawled file paths are allowed to contain arbitrary characters, and we'll escape them using percent-encoding.
• Clarified "locator" vs "report" contexts that determine when escaping is needed

# Monodex Identifier & Reference Syntax Proposal

## 1. Scope

This document defines the syntax of identifiers, locators, and reports used by Monodex at the CLI and in storage. It covers:

- **Catalogs**: names Monodex assigns to data sources it indexes.
- **Labels**: names Monodex assigns to versions/snapshots within a catalog.
- **Locators**: parseable strings that identify content within a grammar (breadcrumbs and references).
- **Reports**: human-facing output that describes content without parseable structure.

It does **not** define what paths are allowed inside a catalog. Paths are bytes from external systems (Git trees, working directories, issue systems). Monodex does not own them and must not reject or silently drop them. See §8 for how paths are handled.

---

## 2. Terminology

- **Catalog** — a Monodex-assigned name for a data source. Few and stable. Chosen by the user.
- **Label** — a Monodex-assigned name for a version or snapshot of a catalog (branch, commit, tag, working-directory state, time snapshot). Many and diverse.
- **Path** — a location within a label. The identity of a path is determined by the underlying data source. Monodex does not assign or constrain path syntax.
- **Locator** — a structured string that identifies content within a grammar. Two locator grammars exist: **breadcrumbs** (catalog-relative, of the form `package:file:symbol`) and **references** (globally qualified, of the form `@catalog:label:path` and its sub-forms). Locators are parseable; their structural characters must be encoded when they appear inside path or identifier data.
- **Report** — human-facing output (CLI stdout, error messages, log lines). Reports are not locators. They use decoded paths and separate fields by visual devices that cannot collide with identifier characters.

Catalogs and labels are **identifiers Monodex owns**. Paths are **external data Monodex indexes**. This distinction is load-bearing; see §8.

---

## 3. Motivating Examples

### 3.1 Typical single-repo usage

```bash
--catalog my-repo
--label main
--path main:src/index.ts
```

### 3.2 Cross-catalog references

```bash
--path @frontend:main:src/app.ts
--path @backend:main:src/server.ts
```

### 3.3 Typed labels disambiguate user-chosen names

A user creates a branch literally named `commit`:

```bash
--path @my-repo:branch=commit:src/index.ts
--path @my-repo:commit=abc123:src/index.ts
```

The typed form makes these unambiguous. Other typed kinds include `tag=v1.2.3`, `local=working-dir`, `snapshot=2026-04-16T12-00`.

### 3.4 Non-Git data sources

```bash
--path @github-issues:snapshot=2026-04-16:issue/123
```

### 3.5 Paths with reserved characters

A user's repo contains a file literally named `weird:file.ts`:

```bash
--path @my-repo:main:src/weird%3Afile.ts
```

See §8 for the encoding rules.

---

## 4. Syntax Overview

### Base forms

```
path
label
catalog
```

### Composite reference forms

```
label:path
kind=payload:path
@catalog:path
@catalog:label
@catalog:kind=payload
@catalog:kind=payload:path
```

---

## 5. Typed Label Form

### Structure

```
<kind>=<payload>
```

Examples: `branch=main`, `commit=abc123`, `tag=v1.2.3`, `local=working-dir`, `snapshot=2026-04-16T12-00`.

`kind` is a reserved identifier; `payload` is opaque and may contain `/`. The typed form eliminates ambiguity with user-created branch names that coincide with reserved kinds.

### Status of `=` Today

The typed form is **reserved grammar**. In the current implementation, `=` is a permitted character in label identifiers but is not parsed or interpreted: `--label branch=main` today yields a label literally named `branch=main`. Users may adopt `kind=payload` as a naming convention in their own automation, and such names will remain valid when the typed form is parsed natively in the future.

---

## 6. Parsing Rules (Planned)

These rules describe the future parser. Today only the bare forms of `--catalog` and `--label` are parsed; composite forms are reserved.

1. If the string starts with `@`, parse catalog first:

   ```
   @catalog:...
   ```

2. Split the remaining string on `:` (left-to-right, respecting path encoding per §8):
   - 1 segment → label or path (based on context)
   - 2 segments → `label:path` or `kind=payload:path`
   - 3 segments → `@catalog:label:path` or `@catalog:kind=payload:path`

3. Within a label segment: if `=` present, parse as `kind=payload`; otherwise treat as opaque.

4. `/` is never parsed structurally; it is always part of a label or a decoded path.

5. Path segments are percent-decoded per §8 before use.

---

## 7. CLI Semantics

### `--catalog`

Accepts only the bare form:

```
catalog
```

### `--label`

Accepts only the bare form today:

```
label
```

Composite forms (`kind=payload`, `@catalog:label`, `@catalog:kind=payload`) are **not accepted** at the CLI yet. `=` is a valid character inside a bare label but is not interpreted.

### `--path`

Accepts only the bare path form today. Planned composite forms:

```
path
label:path
kind=payload:path
@catalog:path
@catalog:label:path
@catalog:kind=payload:path
```

---

## 8. Paths

### 8.1 Principle

Paths are facts about external systems. Monodex does not assign path syntax and must not refuse to index a file because its path contains a character that collides with Monodex's locator grammars.

This rules out two failure modes:

- **Rejection** — refusing to crawl a file because its path contains `:`, `@`, or `=`. Monodex does not control what filenames appear in a Git tree.
- **Silent omission** — skipping such files with a warning. The user gets a crawl reported as "successful" but search results are missing content they expect. Worse than rejection because the failure is invisible.

Both are forbidden.

### 8.2 Storage

Paths are stored verbatim. No normalization, rewriting, or character substitution. The path field round-trips bit-for-bit with what the data source reported.

### 8.3 Encoding at Locator Boundaries

When a path appears inside a locator — a breadcrumb or a reference, any context where it is concatenated with grammar characters — it is **percent-encoded per RFC 3986**.

Characters that **must** be encoded in a path within a locator:

- Grammar-reserved: `:`, `@`, `=`, `+`, `#`
- The escape character itself: `%`
- Whitespace and control characters

`/` is **not** encoded. It is a legitimate path separator and does not collide with any locator-grammar character.

Decoding is the inverse: percent-sequences in the path segment of a locator are decoded before lookup. Storage still holds the decoded form.

Percent-encoding was chosen over backslash or quote-based escaping because it survives shells, JSON, and YAML without re-escaping, and it keeps ordinary paths mostly readable.

### 8.4 Examples

| Stored path         | In a breadcrumb                           | In a global reference                |
| ------------------- | ----------------------------------------- | ------------------------------------ |
| `src/index.ts`      | `node-core-library:index.ts:foo`          | `@my-repo:main:src/index.ts`         |
| `src/weird:file.ts` | `node-core-library:weird%3Afile.ts:foo`   | `@my-repo:main:src/weird%3Afile.ts`  |
| `50%off/notes.md`   | `node-core-library:50%25off/notes.md:foo` | `@my-repo:main:50%25off/notes.md`    |

### 8.5 Breadcrumbs (Catalog-Relative Locators)

Breadcrumbs have the form `package:file:symbol`. The `:` is a structural separator within the breadcrumb grammar. Path components embedded in a breadcrumb are percent-encoded per §8.3. Example: a file named `weird:file.ts` in package `node-core-library` renders as `node-core-library:weird%3Afile.ts:JsonFile.load`.

Breadcrumbs are catalog-relative: they do not begin with `@catalog`. A reader who needs the fully-qualified identity of a breadcrumb must consult the surrounding context (the search result's `catalog` field, the crawl's scope, etc.).

### 8.6 Human Reports

Report lines in CLI output (e.g. `Source:` lines, error messages, progress output) are not locators and must not look like them. Paths in reports use the stored (decoded) form. Fields within a report line are separated by a visual device that cannot appear in a catalog name — parentheses, ` / `, or a newline — never by `:` or `@`.

Correct:

```
Source: (my-repo) src/weird:file.ts
Source: my-repo / src/weird:file.ts
```

Incorrect (looks like a locator, isn't one):

```
Source: my-repo:src/weird:file.ts
Source: @my-repo:src/weird:file.ts
```

---

## 9. Identifier Rules

### 9.1 Forbidden Characters

Forbidden in bare catalog and label identifiers:

```
:  @  +  #  whitespace  control characters
```

These are reserved for current or future locator grammar. `+` and `#` are not used by the grammar today but are reserved to keep future extensions non-breaking.

`=` is additionally forbidden in catalogs. `=` is permitted in labels but is not interpreted today (§5); a label containing `=` is an opaque identifier.

### 9.2 Catalog (kebab-case)

```
^[a-z0-9]+(?:-[a-z0-9]+)*$
```

- Length 1–64 characters.
- Lowercase ASCII alphanumeric words separated by single `-`.
- No leading, trailing, or consecutive `-`.

Examples: `my-repo`, `frontend`, `backend-api`.

### 9.3 Label (Git-like)

```
^[a-z0-9]+(?:[./=-][a-z0-9]+)*$
```

- Length 1–128 characters.
- Lowercase ASCII alphanumeric words separated by single `.`, `/`, `=`, or `-`.
- No leading, trailing, or consecutive separators.
- `=` is a permitted separator character but is not interpreted as a typed-form delimiter today.

Examples: `main`, `feature/x`, `release/v1.2.3`, `working-dir`, `branch=main`, `repo/sub/feature`.

### 9.4 Kind (typed prefix, planned)

```
^[a-z0-9]+$
```

Applies when the typed form `kind=payload` is parsed natively in a future release. Not enforced today.

Examples: `branch`, `commit`, `tag`, `local`, `snapshot`.

---

## 10. Key Principles

> Catalogs and labels are identifiers Monodex owns and may constrain. Paths are external data Monodex must represent faithfully.
>
> Labels remain human- and Git-like. Machine semantics are layered via `kind=payload` only when needed.
>
> Locators and reports are different things. Locators are parseable and encode at boundaries; reports are for humans and use decoded paths with visually distinct separators.

[octogonz]

Dev plan:

# Monodex Identifier Terminology & Validation Spec

> **Companion document:** the identifier & reference syntax proposal at https://github.com/microsoft/monodex/issues/25 (the "syntax spec") defines the grammar for catalog, label, and path references, including forms not yet implemented. That document is authoritative for grammar. This document is authoritative for what lands in the current implementation.

## 1. Problem

The codebase uses two forms for referring to a label and is inconsistent about which goes where:

- **`label_name`** — bare label identifier like `main` or `feature/login-flow`
- **`label_id`** — fully qualified storage key like `rushstack:main`

The distinction is sound but the boundaries have drifted:

- **CLI commands disagree on input shape.** `use` takes `--catalog` and `--label` as two bare flags. `search`, `view`, and `crawl` document `--label` as the colon-joined form `<catalog>:<label>`. `crawl`'s doc comments contradict themselves and its handler requires the colon form — so the bare-name usage the docstring implies actually fails validation.
- **The qualified form leaks into user-facing surfaces** (CLI flags, `Label ID:` printouts, error messages) where it doesn't belong. The code currently parses and prints IDs where it should parse and print names, which is precisely the confusion this spec exists to eliminate.
- **No identifier has defined character rules.** Any string is accepted today, including values that collide with characters the syntax spec reserves for future grammar.
- **The word `label_name` is itself a source of drift.** Every field named `label_name` is an opportunity for a future reader or author to conflate the name and the id, as has already happened.

## 2. Terminology

Apply consistently across every file. No mixing, no drift.

- **`catalog`** — bare catalog identifier, e.g. `my-repo`. User-facing.
- **`label`** — the semantic component stored inside a catalog, e.g. `main`, `feature/login-flow`. User-facing. This is the rename target for every current `label_name` usage.
- **`label_id`** — fully qualified storage key `<catalog>:<label>`. **Internal only.** Appears in the Qdrant `active_label_ids` payload, as a UUID seed for label-metadata points, and in internal log/debug output. A user inspecting a Qdrant collection sees the qualified form in raw data; that is the intended boundary.

Strict rule: a field, variable, parameter, log line, CLI flag, or doc section that refers to the user-typed short form is named `label` (or `catalog`). Anything named with an `_id` suffix must actually contain the qualified `<catalog>:<label>` form. Mismatches are bugs.

## 3. Scope: What Lands Now vs. What's Reserved

Today's commands (`use`, `crawl`, `search`, `view`) only need:

- A `--catalog <catalog>` flag accepting a bare catalog.
- A `--label <label>` flag accepting a bare label.
- Composition of `label_id = <catalog>:<label>` at the storage boundary.

The syntax spec describes a richer grammar — `@catalog:label` references, typed labels (`kind=payload`), path references (`label:path`, `@catalog:label:path`), and reserved characters for further extensions (`+`, `#`). **None of that grammar is parsed or accepted as input today.** What this spec implements from the syntax spec is just:

- The character rules for bare `catalog` and bare `label` identifiers, including allowing `=` inside labels (see §4 invariant 4 for the exact treatment).
- The reserved-character bans for `@`, `+`, `#`, `:`, whitespace, and control characters, enforced today so future grammar can land without breaking existing data.

The `@` prefix is reserved as the lead-in for the planned cross-catalog reference form (`@catalog:label`) and is forbidden in any bare catalog or label today. Reserving it now means the future grammar can be added as a pure extension.

## 4. Invariants

1. **Validation happens at boundaries.** Any catalog or label entering the system through a CLI flag, a config file, a stored context file, or a Qdrant payload read is validated before use. Composition and internal passing of already-validated identifiers does not re-validate.

2. **Users never type or see the qualified form directly.** The CLI accepts catalog and label only as two separate flags. The legacy bare colon-joined `--label catalog:label` form is rejected. User-facing output prints `Catalog:` and `Label:` as separate fields and never uses `Label ID:` or similar phrasing.

3. **CLI flags fall back to stored default context independently.** Missing `--catalog` → default catalog; missing `--label` → default label.

4. **Reserved characters are forbidden in bare identifiers.** These are forbidden in both `catalog` and `label` values today: `:`, `@`, `+`, `#`, whitespace, and control characters. Rationale: the syntax spec plans richer grammar using these characters, and reserving them now prevents breaking changes later.

   **Special case for `=`:** `=` is **allowed inside labels but reserved for future grammar and not interpreted today.** A user who types `--label branch=main` today gets a label literally named `branch=main`, which composes into `label_id = my-repo:branch=main`. Monodex does not parse, split, or assign meaning to the `=`. This permits users to adopt a `kind=payload` naming convention in their own automation (e.g. a cronjob crawling `branch=main`) and ensures the wire format will be compatible when Monodex later grows native typed-label support. `=` is forbidden in `catalog` identifiers.

5. **Catalog and label character sets and regexes:**

   - **`catalog`**: strict kebab-case.
     ```
     ^[a-z0-9]+(?:-[a-z0-9]+)*$
     ```
     Length 1–64. Lowercase ASCII alphanumeric words separated by single `-`. No leading, trailing, or consecutive `-`.

   - **`label`**: Git-like, with `=` permitted internally per invariant 4.
     ```
     ^[a-z0-9]+(?:[./=-][a-z0-9]+)*$
     ```
     Length 1–128. Lowercase ASCII alphanumeric words separated by single `.`, `/`, `-`, or `=`. No leading, trailing, or consecutive separators.

   These regexes are authoritative and must be implemented exactly as written. Error messages should still be produced by runtime code (see §5) rather than by schema pattern mismatch.

6. **Relative paths cannot contain reserved grammar characters.** A `relative_path` captured at crawl time must not contain `:`, `@`, or `=`. `:` would break breadcrumb parsing; `@` and `=` would break future reference round-tripping.

7. **All `label_id` construction goes through a single composition function.** No ad-hoc `format!("{}:{}", catalog, label)` anywhere in the codebase. The composition function is the only place that knows the separator is `:`, so that if the storage encoding ever changes it changes in exactly one place.

8. **No backward compatibility.** Existing Qdrant collections, context files, or configs containing identifiers that fail the new validation are broken by this change and will not load. A clear error pointing at the offending value is the migration path.

## 5. Error Reporting

Follow Rush Stack's conventions:

- **Runtime validation produces the useful errors; JSON schemas stay loose.** Schemas constrain to `type: string, minLength: 1, maxLength: N` and at most a trivial shape hint. Nontrivial rules are enforced in code because a schema pattern mismatch produces `does not match pattern` — unhelpful — whereas runtime code can produce `the catalog name "foo--bar" contains consecutive '-' characters`.
- **Every error names the field, quotes the full offending value, names the specific problem, and suggests a fix where one exists.** Target messages of this quality:
  - `the label "foo:bar" contains ':', which is reserved. If you meant to specify a catalog, use --catalog.`
  - `the catalog name "My-Repo" must be lowercase. Try "my-repo" instead.`
  - `the catalog name "foo--bar" contains consecutive '-' characters.`
  - `the label "feature_login" contains '_', which is not allowed. Use '-' or '/' as separators.`
  - `the catalog name is empty.`
  - `the label "..." exceeds the maximum length of 128 characters.`

The error API is the agent's call. Whatever form it takes (enum, struct, thiserror, plain strings), it must produce messages of this quality.

## 6. Where Validation Must Happen

- CLI argument parsing for `--catalog` and `--label` on every command that accepts them.
- Default context file load (`load_default_context` in `src/main.rs`).
- Catalog config validation (`CatalogConfig::validate` in `src/main.rs`).
- Qdrant label-metadata deserialization (`LabelMetadata` in `src/engine/uploader.rs`).
- Crawl-time path assertion (reject any relative path containing `:`, `@`, or `=`).
- JSON schemas — loose constraints only; authoritative validation is in code. Add a comment in each schema referencing the Rust validator.

## 7. Tasks

1. **Centralize identifier validation and composition.** The current code has `compute_label_id` in `util.rs` and ad-hoc colon-splitting in `main.rs`. Move validation and composition into one module. Every `label_id` produced in the codebase flows through the single composition function defined here. Internal module layout, function signatures, and error types are the agent's call, subject to the invariants above.

2. **Rewrite CLI flag handling in `src/main.rs`.** `--catalog` and `--label` each take a bare validated identifier. The legacy `--label catalog:label` form is rejected with a clear error. The existing `resolve_label_id` function should go away or be rewritten; its current contract (takes a colon-joined string) is wrong. Defaults fill in missing components.

3. **Purge the qualified form from user-facing output.** The core confusion is that the tool today parses and prints IDs where it should parse and print names. Every `println!`, `eprintln!`, error string, and `format!` that emits `Label ID:` or `{catalog}:{label}` in user-facing output must be replaced with separate `Catalog:` / `Label:` fields. Known sites in `src/main.rs`: `run_use` around lines 682 and 711–712, `run_crawl_label` around line 1289, similar block around line 1671. Grep for more.

4. **Rename `label_name` to `label` everywhere in the codebase.** Source files, struct fields, function parameters, log strings, doc comments, Qdrant payload field references. The only remaining uses of the phrase `label_name` after this change should be in migration notes in CHANGELOG.md. The word `label` must mean the name; the `_id` suffix must mean the qualified form; nothing blurs the two. Note: the Qdrant payload field `LabelMetadata.label_name` is itself renamed to `label` — no compatibility layer. This is a deliberate consequence of the no-backward-compatibility stance.

5. **Apply validation at every boundary listed in §6.**

6. **Update schemas.** Constrain catalog and label fields to `{ "type": "string", "minLength": 1, "maxLength": 128 }` with at most a trivial shape hint (e.g. `"pattern": "^[^\\s:@+#]+$"` to catch the most common confusions — note `=` is deliberately permitted here since it is allowed in labels). Add a comment in each schema pointing to the Rust validator.

7. **Update DESIGN.md.** Revise the "Label-Based Indexing" section to use the strict terminology; `label_id` moves into a "Storage Format" subsection explaining it as internal. Every "Query Interface" example uses two separate flags. Add a concise "Identifier Syntax" subsection that:
   - Gives the regexes and length bounds for `catalog` and `label` as implemented today, matching §4 invariant 5 exactly.
   - Shows brief examples of valid and invalid values, including at least one example using `=` in a label to illustrate the reserved-but-not-interpreted treatment.
   - Documents the full envisioned syntax (typed labels `kind=payload`, `@catalog:label` references, `@catalog:label:path` path references, reserved `+` and `#`) as planned grammar, with a clear note that only bare catalog and label are parsed today.
   - References https://github.com/microsoft/monodex/issues/25 for the full spec.
   Justifies why `@`, `+`, `#` are forbidden in bare identifiers today, and why `=` is permitted but not interpreted: because they are reserved for future grammar described in the linked issue.

8. **Update README.md.** Every example uses the two-flag form. No `Label ID` anywhere.

9. **Update CHANGELOG.md.** One entry documenting the breaking change and noting that existing collections with non-conforming identifiers must be recreated, and that the `label_name` payload field has been renamed to `label`.

## 8. Acceptance Criteria

Review gate, not an implementation step.

- The string `label_name` appears nowhere in source code — it is fully renamed to `label`. Only CHANGELOG.md retains a reference, documenting the rename.
- `label_id`, `Label ID`, `catalog:label`, `<catalog>:<label>`, and `qualified` appear only in: the validation module, Qdrant storage code actually serializing to or deserializing from `active_label_ids`, or comments explicitly marking an internal-storage usage.
- No user-facing CLI help text, console output, error message, or public doc references the qualified form.
- All `label_id` construction in the codebase goes through the single composition function. No ad-hoc `format!` concatenations.
- `@`, `+`, `#`, `:`, whitespace, and control characters are rejected in both `--catalog` and `--label` input, with messages naming the specific character and, where applicable, pointing to the fix. `=` is rejected in `--catalog` but accepted in `--label` without any special interpretation.
- Every validation path in §6 fails cleanly on malformed input with a message of the quality shown in §5.
- DESIGN.md describes the full envisioned grammar from the linked issue but is clear about what is and isn't implemented today.