Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/tinyglobby-globs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@stainless-code/codemap": patch
---

Replace `fast-glob` with `tinyglobby` for Node include globs. Smaller dependency footprint; `expandDirectories: false` keeps matching aligned with the previous behavior.
6 changes: 0 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,34 +19,28 @@
### Patch Changes

- [#8](https://github.com/stainless-code/codemap/pull/8) [`889ed5b`](https://github.com/stainless-code/codemap/commit/889ed5b695823e9a57f133c9643af9dbb3e89236) Thanks [@SutuSebastian](https://github.com/SutuSebastian)! - **Query CLI**

- **`codemap query --json`**: print a JSON array of result rows to stdout (and **`{"error":"…"}`** on SQL errors) for agents and automation. Document that the query subcommand does **not** cap rows — use SQL **`LIMIT`** for bounded results. Update bundled agent rule and skill with **`--json`** preference, verbatim structural answers, and generic SQL recipes (fan-out + sample targets).

- **`codemap query --recipe <id>`** for bundled read-only SQL so agents can run common structural queries without embedding SQL on the command line. **`--json`** works with recipes the same way as ad-hoc SQL. Bundled ids include dependency **`fan-out`** / **`fan-out-sample`** / **`fan-out-sample-json`** (JSON1 **`json_group_array`**) / **`fan-in`**, index **`index-summary`**, **`files-largest`**, React **`components-by-hooks`** (comma-based hook count, no JSON1), and **`markers-by-kind`**. The benchmark suite uses the **`fan-out`** recipe SQL for an indexed-path scenario; docs clarify that recipes add no extra query cost vs pasting the same SQL.

- **Recipe discovery (no index / DB):** **`codemap query --recipes-json`** prints all bundled recipes (**`id`**, **`description`**, **`sql`**) as JSON. **`codemap query --print-sql <id>`** prints one recipe’s SQL. **`listQueryRecipeCatalog()`** in **`src/cli/query-recipes.ts`** is the single derived view of **`QUERY_RECIPES`** for the JSON output.

**Golden tests**

- **`bun run test:golden`**: index **`fixtures/minimal`**, run scenarios from **`fixtures/golden/scenarios.json`**, and compare query JSON to **`fixtures/golden/minimal/`**. Use **`bun scripts/query-golden.ts --update`** after intentional fixture or schema changes. Documented in **benchmark.md** and **CONTRIBUTING**.

**Query robustness**

- With **`--json`**, **`{"error":"…"}`** is printed for invalid SQL, database open failures, and **`codemap query`** bootstrap failures (config / resolver setup), not only bad SQL. The CLI sets **`process.exitCode`** instead of **`process.exit`** so piped stdout is not cut off mid-stream.

**Benchmark & `CODEMAP_BENCHMARK_CONFIG`**

- Each **`indexedSql`** in custom scenario JSON is validated as a single read-only **`SELECT`** (or **`WITH` … `SELECT`**) — DDL/DML and **`RETURNING`** are rejected before execution.
- Config file paths are resolved from **`process.cwd()`** (see **benchmark.md**). **`traditional.regex`** strings are developer-controlled (local JSON); **`files`** mode compiles the regex once per scenario.
- Overlapping **globs** in the traditional path are **deduplicated** so **Files read** / **Bytes read** count each path once.
- The default **components in `shop/`** scenario uses a **`LIKE`** filter aligned with the traditional globs under **`components/shop/`** (**\*.tsx** and **\*.jsx**, matching **`components`** rows from the parser) and avoids unrelated paths such as **`workshop`**.

**Recipes (determinism)**

- Bundled recipe SQL adds stable secondary **`ORDER BY`** columns (and orders inner **`LIMIT`** samples) so **`--recipe`** / **`--json`** output does not vary on aggregate ties.

**External QA**

- **`bun run qa:external`**: **`--max-files`** and **`--max-symbols`** must be positive integers (invalid values throw before indexing).

## 0.1.3
Expand Down
2 changes: 1 addition & 1 deletion bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru

**`src/sqlite-db.ts`:** Node uses **`better-sqlite3`**; Bun uses **`bun:sqlite`**. Same schema everywhere. **`better-sqlite3`** allows **one SQL statement per `prepare()`**; **`bun:sqlite`** accepts **multiple statements** in one `run()`. On Node, **`runSql()`** splits multi-statement strings on **`;`** and runs each fragment. Do **not** put **`;`** inside **`--` line comments** in **`db.ts`** DDL strings (naive split would break). Details: [packaging.md § Node vs Bun](./packaging.md#node-vs-bun).

**`src/worker-pool.ts`:** Bun `Worker` or Node `worker_threads`. **`src/glob-sync.ts`:** Bun **`Glob`** or **`fast-glob`** for include patterns. **`src/config.ts`:** loads **`codemap.config.json`** / **`codemap.config.ts`** (JSON read path: **`Bun.file`** on Bun, **`readFile` + `JSON.parse`** on Node — [packaging.md § Node vs Bun](./packaging.md#node-vs-bun)), then validates with **Zod** (`codemapUserConfigSchema`). Details: [User config](#user-config).
**`src/worker-pool.ts`:** Bun `Worker` or Node `worker_threads`. **`src/glob-sync.ts`:** Bun **`Glob`** or **`tinyglobby`** for include patterns. **`src/config.ts`:** loads **`codemap.config.json`** / **`codemap.config.ts`** (JSON read path: **`Bun.file`** on Bun, **`readFile` + `JSON.parse`** on Node — [packaging.md § Node vs Bun](./packaging.md#node-vs-bun)), then validates with **Zod** (`codemapUserConfigSchema`). Details: [User config](#user-config).

**Shipped artifact:** **`dist/`** — `package.json` **`bin`** and **`exports`** both point at **`dist/index.mjs`** ([packaging.md](./packaging.md)); tsdown also emits **lazy CLI chunks** (`cmd-index`, `cmd-query`, `cmd-agents`, …) loaded via **`import()`** from **`src/cli/main.ts`**.

Expand Down Expand Up @@ -104,7 +104,7 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru
| `css-parser.ts` | CSS extraction via `lightningcss` — custom properties, classes, keyframes, `@theme` blocks |
| `resolver.ts` | Import path resolution via `oxc-resolver` — respects `tsconfig` aliases, builds dependency graph |
| `constants.ts` | Shared constants — e.g. `LANG_MAP` |
| `glob-sync.ts` | Include globs — Bun `Glob` vs `fast-glob` on Node ([packaging § Node vs Bun](./packaging.md#node-vs-bun)) |
| `glob-sync.ts` | Include globs — Bun `Glob` vs `tinyglobby` on Node ([packaging § Node vs Bun](./packaging.md#node-vs-bun)) |
| `markers.ts` | Shared marker extraction (`TODO`/`FIXME`/`HACK`/`NOTE`) — used by all parsers |
| `parse-worker.ts` | Worker thread entry point — reads, parses, and extracts file data in parallel |
| `adapters/` | `LanguageAdapter` types and built-in TS/CSS/text implementations |
Expand Down
4 changes: 2 additions & 2 deletions docs/packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ How **@stainless-code/codemap** is built and published. **Doc index:** [README.m

Published tarballs match **`package.json` `files`**: **`CHANGELOG.md`**, **`dist/`**, **`templates/`** (no `src/`). **`bun run pack`**, then point the consumer at **`file:…/stainless-code-codemap-*.tgz`**, or use **`file:/path/to/repo`** after build, or **`bun link`**. If **`better-sqlite3`** fails in the consumer, **`npm rebuild better-sqlite3`** (native addon must match that Node).

**Engines** (`package.json`): **Node** `^20.19.0 || >=22.12.0` (matches **`oxc-parser`**; **`better-sqlite3`** is prebuilt for current Node majors only). **Bun** `>=1.0.0`. **Native bindings:** `better-sqlite3`, `lightningcss`, `oxc-parser`, `oxc-resolver` (NAPI); **`fast-glob`** and **`zod`** are JS-only. **`zod`** validates `codemap.config.*` at runtime (**`codemapUserConfigSchema`** in **`src/config.ts`**); see [architecture.md § User config](./architecture.md#user-config).
**Engines** (`package.json`): **Node** `^20.19.0 || >=22.12.0` (matches **`oxc-parser`**; **`better-sqlite3`** is prebuilt for current Node majors only). **Bun** `>=1.0.0`. **Native bindings:** `better-sqlite3`, `lightningcss`, `oxc-parser`, `oxc-resolver` (NAPI); **`tinyglobby`** and **`zod`** are JS-only. **`zod`** validates `codemap.config.*` at runtime (**`codemapUserConfigSchema`** in **`src/config.ts`**); see [architecture.md § User config](./architecture.md#user-config).

## Node vs Bun

Expand All @@ -21,7 +21,7 @@ Same schema and CLI; implementation differs by runtime. Details: [architecture.m
| ------------- | --------------------------------- | --------------------------------------------- |
| SQLite | **`bun:sqlite`** (`sqlite-db.ts`) | **`better-sqlite3`** |
| Workers | **`Worker`** → `parse-worker.ts` | **`worker_threads`** → `parse-worker-node.ts` |
| Include globs | **`Glob`** (`glob-sync.ts`) | **`fast-glob`** |
| Include globs | **`Glob`** (`glob-sync.ts`) | **`tinyglobby`** |
| JSON config | **`Bun.file(…).json()`** | **`readFile` + `JSON.parse`** (`config.ts`) |

**`db.ts`** does not import **`bun:sqlite`**. Upstream API: [Bun SQLite](https://bun.com/docs/api/sqlite). No **`bun build --compile`** shipping — see [Bun executables](https://bun.sh/docs/bundler/executables).
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,10 @@
"dependencies": {
"@clack/prompts": "^1.2.0",
"better-sqlite3": "^12.8.0",
"fast-glob": "^3.3.3",
"lightningcss": "^1.32.0",
"oxc-parser": "^0.124.0",
"oxc-resolver": "^11.19.1",
"tinyglobby": "^0.2.15",
"zod": "^4.3.6"
},
"devDependencies": {
Expand Down
11 changes: 8 additions & 3 deletions src/glob-sync.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ import { expect, test } from "bun:test";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";

import fg from "fast-glob";
import { globSync as tinyglobbySync } from "tinyglobby";

import { globSync } from "./glob-sync";

test("globSync matches fast-glob (Bun parity)", () => {
test("globSync matches tinyglobby (Bun parity)", () => {
if (typeof Bun === "undefined") return;

const cwd = join(
Expand All @@ -15,6 +15,11 @@ test("globSync matches fast-glob (Bun parity)", () => {
);
const pattern = "**/*.{ts,tsx}";
const a = globSync(pattern, cwd).sort();
const b = fg.sync(pattern, { cwd, dot: true, absolute: false }).sort();
const b = tinyglobbySync(pattern, {
cwd,
dot: true,
absolute: false,
expandDirectories: false,
}).sort();
expect(a).toEqual(b);
});
11 changes: 8 additions & 3 deletions src/glob-sync.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import fg from "fast-glob";
import { globSync as tinyglobbySync } from "tinyglobby";

/**
* Glob files relative to `cwd` (dotfiles included). On Bun uses `Glob` from `bun`;
* on Node uses `fast-glob` for identical published behavior.
* on Node uses `tinyglobby` for identical published behavior.
*/
export function globSync(pattern: string, cwd: string): string[] {
if (typeof Bun !== "undefined") {
Expand All @@ -11,5 +11,10 @@ export function globSync(pattern: string, cwd: string): string[] {
const glob = new Glob(pattern);
return Array.from(glob.scanSync({ cwd, dot: true }));
}
return fg.sync(pattern, { cwd, dot: true, absolute: false });
return tinyglobbySync(pattern, {
cwd,
dot: true,
absolute: false,
expandDirectories: false,
});
}
2 changes: 1 addition & 1 deletion tsdown.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ export default defineConfig({
neverBundle: [
"bun",
"better-sqlite3",
"fast-glob",
"tinyglobby",
"lightningcss",
"oxc-parser",
"oxc-resolver",
Expand Down
Loading