chore: merge development into main for v1.3.0 release#354
Merged
Conversation
* chore: add development branch workflow - Add merge-gate.yml: enforces only 'development' can merge into main - Update CI/CodeQL/Docker workflows to run on both main and development - Update dependabot.yml: target-branch set to development for all ecosystems - Update copilot-instructions.md: document branch workflow convention - Rulesets configured: Main (requires merge-gate + squash-only), Development (requires CI status checks + PR) - Default branch set to development - All open PRs retargeted to development Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: skip Vercel preview deployments on non-main branches Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: trigger check refresh --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: comprehensive audit fixes — security, performance, resilience, API hardening Addresses findings from issue #314: - SSRF protection for webhook URLs (CRITICAL) - Scrub secrets from exports - Stored XSS prevention on document URL - O(n²) and N+1 fixes in bulk operations - Rate limit cache eviction improvement - SSE backpressure handling - Replace raw Error() with typed errors - Fetch timeouts on all network calls - Input validation on API parameters - Search query length limit - Silent catch block logging - DNS rebinding check fix - N+1 in Slack user resolution - Pagination on webhook/search list endpoints Closes #314 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review comments: fix SSE listener leak, preserve caller signals, add SSRF validation to webhook test, chunk SQL params, use dynamic import in test - SSE backpressure: create single disconnect promise, race against drain (no listener accumulation) - http-utils.ts/onenote.ts: use AbortSignal.any() to combine caller signal with timeout - Webhook test endpoint: validate URL with validateWebhookUrlSsrf before fetch - bulk.ts: chunk IN clause to 999 params max (SQLite limit) - webhooks.test.ts: dynamic import after vi.mock() for deterministic DNS mocking Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: consolidate and fix CI/CD workflows - Merge lint + typecheck into single job (saves one npm ci) - Add concurrency groups to ci, docker, codeql (cancel stale runs) - Add dependency-review-action on PRs (block vulnerable deps) - Add workflow_call trigger to ci.yml for reusability - Remove duplicate npm publish from release.yml (release-please owns it) - Fix SDK paths: sdk-go/ → sdk/go/, sdk-python/ → sdk/python/ - Fix Dependabot paths to match actual SDK directories - Add github-actions ecosystem to Dependabot (keep actions up to date) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb48187. * feat: add --from option to pack create for folder/URL sources Adds createPackFromSource() that builds packs directly from local folders, files, or URLs without requiring database interaction. CLI: libscope pack create --name my-pack --from ~/docs/ [--extensions .md,.html] [--exclude pattern] [--no-recursive] Features: - Walks directories recursively using registered parsers - Fetches URLs via fetchAndConvert - Supports extension filtering, exclude patterns, progress callback - Multiple --from sources supported - Output format identical to DB export (pack install works unchanged) Closes #328 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * style: fix prettier formatting Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add gzip support for pack files (.json.gz) Pack files can now be compressed with gzip for smaller distribution: - writePackFile/readPackFile auto-detect gzip by extension or magic bytes - installPack accepts both .json and .json.gz files - createPackFromSource defaults to .json.gz output (source packs can be large) - createPack (DB export) still defaults to .json - Auto-detects gzip by magic bytes even if extension is .json 5 new tests covering gzip write, install, magic byte detection, and round-trip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add progress logging and fix dedup handling in pack install - Log each document as it's indexed so large installs show progress - Change pack install to use dedup: 'skip' for graceful duplicate handling - Make title+content_length dedup check respect the dedup mode setting (previously it always threw ValidationError regardless of dedup mode) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: auto-generate tags during pack creation and apply on install - Export tokenize() and add suggestTagsFromText() in tags.ts for DB-free tag generation - createPackFromSource() now auto-generates tags per document via TF-IDF - installPack() applies doc.tags via addTagsToDocument() after indexing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb48187. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bumps [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) from 9.1.2 to 10.1.8. - [Release notes](https://github.com/prettier/eslint-config-prettier/releases) - [Changelog](https://github.com/prettier/eslint-config-prettier/blob/main/CHANGELOG.md) - [Commits](https://github.com/prettier/eslint-config-prettier/commits/v10.1.8) --- updated-dependencies: - dependency-name: eslint-config-prettier dependency-version: 10.1.8 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the minor-and-patch group with 1 update: [lint-staged](https://github.com/lint-staged/lint-staged). Updates `lint-staged` from 16.3.1 to 16.3.2 - [Release notes](https://github.com/lint-staged/lint-staged/releases) - [Changelog](https://github.com/lint-staged/lint-staged/blob/main/CHANGELOG.md) - [Commits](lint-staged/lint-staged@v16.3.1...v16.3.2) --- updated-dependencies: - dependency-name: lint-staged dependency-version: 16.3.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the actions group with 5 updates: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `4` | `6` | | [actions/setup-node](https://github.com/actions/setup-node) | `4` | `6` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4` | `7` | | [actions/setup-python](https://github.com/actions/setup-python) | `5` | `6` | | [actions/setup-go](https://github.com/actions/setup-go) | `5` | `6` | Updates `actions/checkout` from 4 to 6 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v6) Updates `actions/setup-node` from 4 to 6 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v6) Updates `actions/upload-artifact` from 4 to 7 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v4...v7) Updates `actions/setup-python` from 5 to 6 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) Updates `actions/setup-go` from 5 to 6 - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 22.19.13 to 25.3.3. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 25.3.3 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [better-sqlite3](https://github.com/WiseLibs/better-sqlite3) from 11.10.0 to 12.6.2. - [Release notes](https://github.com/WiseLibs/better-sqlite3/releases) - [Commits](WiseLibs/better-sqlite3@v11.10.0...v12.6.2) --- updated-dependencies: - dependency-name: better-sqlite3 dependency-version: 12.6.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [eslint](https://github.com/eslint/eslint) from 9.39.3 to 10.0.2. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v9.39.3...v10.0.2) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Robert DeRienzo <rderienzo@voloridge.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add passthrough LLM mode for ask-question tool
Adds llm.provider = "passthrough" so the ask-question MCP tool returns
retrieved context chunks directly to the calling LLM instead of requiring
a separate OpenAI/Ollama provider. This is the natural design for MCP tools
where the client already has an LLM (e.g. Claude Code).
- config.ts: add "passthrough" to llm.provider union type and env var handling
- rag.ts: add isPassthroughMode() helper and getContextForQuestion() which
retrieves and formats context without an LLM call
- mcp/server.ts: ask-question checks passthrough first and returns context
directly; falls through to existing LLM path otherwise
Enable via config: { "llm": { "provider": "passthrough" } }
Enable via env var: LIBSCOPE_LLM_PROVIDER=passthrough
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: format config.ts and include passthrough in provider override
- Reformat long if-condition to satisfy prettier (printWidth: 100)
- Fix logic bug: passthrough provider was checked in outer condition but
not spread into overrides.llm.provider
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address 9 audit findings from issue #332 Security - middleware: use timingSafeEqual for API key comparison (#2) - url-fetcher, http-utils: replace process-wide NODE_TLS_REJECT_UNAUTHORIZED mutation with per-request undici Agent to eliminate TLS race condition (#1) Bugs - indexing: re-throw unexpected embedding errors so transaction rolls back instead of silently committing chunks with no vector (#3) - search: replace correlated minRating subquery with avg_r.avg_rating from the pre-joined aggregate in FTS and LIKE search paths (#4) Performance - bulk: replace O(n²) docs.find() loops with pre-built Map; replace per-document getDocumentTags() calls with a single getDocumentTagsBatch() query (#5) - config: add 30-second TTL cache to loadConfig() so disk reads are not repeated on every request (#6) Code quality - routes: check res.write() return value to handle SSE backpressure (#7) - reindex: delegate to schema.createVectorTable() instead of duplicating the vec0 DDL inline (#8) - obsidian: replace hand-rolled parseSimpleYaml() with js-yaml, normalise Date objects back to ISO-8601 strings (#9) Docs - agents.md: expand architecture tree to include src/api/ and src/connectors/; add Security Patterns section with correct undici examples - CONTRIBUTING.md: fix check-suite command (npm test → npm run test:coverage) and correct coverage threshold (80% → actual 75%/74%) Tests - bulk.test: add dateFrom/dateTo filter coverage - config.test: add cache-hit test; call invalidateConfigCache() before env-var tests so TTL cache doesn't return stale results Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused warnIfTlsBypassMissing function Dead code after conflict resolution chose the per-request undici Agent approach (which doesn't need a warning about NODE_TLS_REJECT_UNAUTHORIZED). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update tests for config cache and retry semantics - Add invalidateConfigCache() before loadConfig() in 4 env-override tests that were failing because the 30s TTL cache introduced in the config module was returning stale results from the previous test's cache entry - Update http-utils retry assertion: maxRetries=2 means 1 initial + 2 retries = 3 total calls (loop is attempt <= maxRetries) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
… (#336) - Add `src/cli/reporter.ts`: PrettyReporter (ANSI colors + \r progress bar), SilentReporter (no-op for verbose/JSON mode), `isVerbose()` and `createReporter()` factory. `LIBSCOPE_VERBOSE=1` env var alternative. - Update `setupLogging` to default to "silent" in CLI mode (pretty reporter handles user-facing output). Verbose/`--log-level` flags still route to structured JSON pino logs. Fix duplicate `initLogger` calls in onenote connect/disconnect commands to use `setupLogging` consistently. - Update `installPack` in `packs.ts` to support batch embedding and progress reporting: - New `InstallOptions` interface with `batchSize`, `resumeFrom`, `onProgress` fields - Batch documents: chunk all → single `provider.embedBatch` call per batch → single SQLite transaction per batch (avoids N embedding calls) - `resumeFrom` skips the first N documents (enables partial install resume after failure) - `InstallResult` now includes `errors` count - Add `--batch-size` and `--resume-from` CLI options to `pack install` - Tests: `tests/unit/reporter.test.ts` (17 tests covering PrettyReporter, SilentReporter, isVerbose, env var detection); extended `tests/unit/packs.test.ts` with 7 new tests for progress callbacks, batch efficiency, resumeFrom, embedBatch failure handling. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…dates (#337) Bumps the npm_and_yarn group with 2 updates in the / directory: [@hono/node-server](https://github.com/honojs/node-server) and [hono](https://github.com/honojs/hono). Updates `@hono/node-server` from 1.19.9 to 1.19.10 - [Release notes](https://github.com/honojs/node-server/releases) - [Commits](honojs/node-server@v1.19.9...v1.19.10) Updates `hono` from 4.12.3 to 4.12.5 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](honojs/hono@v4.12.3...v4.12.5) --- updated-dependencies: - dependency-name: "@hono/node-server" dependency-version: 1.19.10 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.5 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (closes #340) **SSRF (CWE-918 — CodeQL alert #28)** Replace the two-step validate-then-fetch approach in url-fetcher.ts with IP-pinned requests using node:http / node:https directly. validateUrl() resolves DNS and checks for private IPs, then the validated IP is passed straight to the TCP connection (hostname: pinnedIp, servername: original hostname for TLS SNI). There is now zero TOCTOU window between validation and the actual network request. The redundant post-fetch DNS rebinding check and the env-var-based TLS bypass wrapper are removed; rejectUnauthorized is now passed directly to the request options. An internal _setRequestImpl hook is exported for unit test injection so tests can stub responses without touching node:http / node:https. Tests are updated accordingly. **ReDoS (CWE-1333 — CodeQL alert #24)** Five regexes in confluence.ts used the pattern [^>]*ac:name="X"[^>]* — two [^>]* quantifiers around a fixed literal. For input that contains a large attribute blob without the target ac:name value, the engine must try all O(n²) splits before concluding no match (catastrophic backtracking). Fix: rewrite the leading [^>]* as (?:(?!ac:name="X")[^>])* — a negative lookahead prevents the quantifier from overlapping with the literal, making backtracking structurally impossible. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: replace remaining polynomial regexes in confluence.ts with indexOf helpers The conflict resolution left the original `/<ac:structured-macro [^>]*>([\s\S]*?) <\/ac:structured-macro>/gi` patterns in place for code blocks and info/tip panels (those were not part of the original security fix diff). These have the same O(n²) backtracking problem: with k opening tags and no closing tags, [\s\S]*? scans O(n - pos) chars per attempt, totalling O(n²). Replace the entire convertConfluenceStorage function with the indexOf-based approach (replaceStructuredMacros / replaceTagPairs / extractTagContent helpers) that eliminates all regex-based tag parsing. Also add removeSelfClosingMacros to handle the self-closing TOC case without regex, since the previous self-closing fix still used a [^>]*ac:name="toc"[^>]* pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…) (#339) * feat: concurrent pack installation and -v verbose shorthand (issue #330) Add concurrent batch embedding to installPack for significant performance improvement on large packs, plus CLI ergonomics improvements. Key changes: - `InstallOptions.concurrency` (default: 4): controls how many embedBatch calls run simultaneously; embedding is I/O-bound so parallelism directly reduces wall-clock installation time - Refactor installPack to pre-chunk all documents upfront, then use a semaphore-based scheduler to run up to `concurrency` embedBatch calls concurrently while inserting completed batches in-order (SQLite requires serialised writes); progress callbacks fire after each batch as before - `pack install --concurrency <n>` CLI flag exposes the new option - `-v` shorthand for `--verbose` on the global program options - Fix transaction install-count tracking: count committed docs accurately without relying on subtract-on-failure arithmetic - Add 6 new tests covering concurrency=1 sequential, concurrency=4 parallel, multiple embedBatch calls per install, concurrency limit enforcement, incremental progress reporting, and partial-failure error counting https://claude.ai/code/session_019hzhbEgV1ysnGmFVXBWzkZ * fix: address all 4 Copilot review comments on PR #339 - Validate batchSize, concurrency, resumeFrom at the start of installPack and throw ValidationError for invalid values (comments 3 & 4). Concurrency <= 0 would silently hang the semaphore indefinitely. - Add CLI lower-bound guard: --concurrency < 1 exits with a user-facing error before ever calling installPack (comment 3). - Lazy chunking: pre-chunking all documents upfront held chunks for the entire pack in memory simultaneously. Batches now store only the raw documents; resolveBatch() chunks on demand right before embedBatch is called, so only one batch's worth of chunks is in memory at a time (comment 2). - Wrap provider.embedBatch() in try/catch so synchronous throws are converted to rejected Promises rather than escaping scheduleNext() and leaving the outer Promise permanently pending (comment 1). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
- Guard JSON.parse in rowToWebhook with try/catch, default to [] - Guard JSON.parse in rowToSavedSearch with try/catch, default to null - Guard JSON.parse in loadDbConnectorConfig with try/catch, throw ConfigError - Push dateFrom/dateTo filters into SQL WHERE in listDocuments (before LIMIT) - Validate negative limit in resolveSelector, throw ValidationError - Replace manual substring extension parsing with path.extname() in packs.ts - Verified reporter.ts is already tracked on development (no action needed) - Added tests for all fixes (corrupted JSON, SQL-level date filtering, negative limit) Closes #342 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…page limits (#315) (#343) * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) Adds opt-in spidering to URL indexing. A single seed URL can now crawl and index an entire documentation site or wiki section in one call. New files: - src/core/link-extractor.ts: indexOf-based <a href> extraction, relative URL resolution, fragment stripping, dedup, scheme filtering. No regex. - src/core/spider.ts: BFS crawl engine with sameDomain, pathPrefix, excludePatterns (glob), maxPages (hard cap 200), maxDepth (hard cap 5), 10-min total timeout, robots.txt (User-agent: * and libscope), and 1s inter-request delay. Yields SpiderResult per page; returns SpiderStats. - tests/unit/link-extractor.test.ts: 25 tests covering relative resolution, dedup, fragment stripping, scheme filtering, attribute order, edge cases. - tests/unit/spider.test.ts: 20 tests covering BFS order, depth/page limits, domain + path + pattern filtering, cycle detection, robots.txt, partial failure recovery, stats, and abortReason reporting. Modified: - src/core/url-fetcher.ts: adds fetchRaw() export returning raw body + contentType + finalUrl before HTML-to-markdown conversion, so the spider can extract links from HTML before conversion. - src/api/routes.ts: POST /api/v1/documents/url now accepts spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns. Returns { documents, pagesIndexed, pagesCrawled, pagesSkipped, errors, abortReason }. - src/mcp/server.ts: submit-document tool gains spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns parameters. Safety: all fetched URLs pass through the existing SSRF validation in fetchRaw() (DNS resolution, private IP blocking, scheme allowlist). Hard limits (200 pages, depth 5, 10min) cannot be overridden by callers. robots.txt is fetched once per origin and Disallow rules are honoured. Individual page failures do not abort the crawl. Closes #315 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: resolve CI lint errors in spider implementation - Remove unnecessary type assertions (routes.ts, mcp/server.ts) — TypeScript already narrows SpiderResult/SpiderStats from the generator - Add explicit return type annotation on mock fetchRaw to satisfy no-unsafe-return rule in spider.test.ts - Replace .resolves.not.toThrow() with a direct assertion — vitest .resolves requires a Promise, not an async function Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CodeQL security findings in spider/link-extractor link-extractor.ts (CodeQL #30 — incomplete URL scheme check): Replace the enumerated scheme blocklist (javascript:, vbscript:, etc.) with a strict http/https allowlist check on the resolved URL protocol. An allowlist is exhaustive by definition; a blocklist will always miss obscure schemes like vbscript:, blob:, or future additions. spider.ts (CodeQL #31 — incomplete multi-character sanitization): Replace the regex-based tag stripper /<[^>]+>/g in extractTitle() with an indexOf-based stripTags() function. The regex stops at the first > which can be inside a quoted attribute value (e.g. <img alt="a>b">), potentially leaving partial tag content in the extracted title. The new implementation walks quoted attribute values explicitly so no tag content leaks through regardless of its internal structure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address all Copilot review comments on spider PR (#343) - link-extractor: add word-boundary check in extractHref to prevent matching data-href, aria-href (false positives on non-href attributes) - spider: rename pagesIndexed → pagesFetched throughout (SpiderStats interface already used pagesFetched; sync implementation + tests) - spider: per-origin robots.txt cache (Map<origin, Set>) fetched lazily as new origins are encountered during crawl (was seed-only before) - spider: normalize to raw.finalUrl after redirects — visited set, yielded URL, and link-extraction base all use the canonical URL - routes: validate maxPages/maxDepth are finite positive integers - routes: change conditional spread &&-patterns to ternaries - routes: remove inner try/catch for spider fetch errors; add FetchError to top-level handler (consistent with single-URL mode → 502) - mcp/server: replace conditional spreads with explicit if assignments - mcp/server: validate spider=true requires url (throws ValidationError) - openapi: document spider request fields in IndexFromUrlRequest schema, add SpiderResponse schema, update 201 response to oneOf Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix prettier formatting in spider files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- README: fix license (BUSL-1.1, not MIT), expand MCP tools table to all 26 tools, expand REST API table with all endpoints (webhooks, links, analytics, connectors status, suggest-tags, bulk ops), add webhooks section with HMAC signing example, add missing CLI commands (bulk ops, saved searches, document links, docs update) - getting-started: fix Node.js requirement (20, not 18), add sections for web dashboard, organize/annotate features, REST API - mcp-setup: expand available tools section to list all 26 tools grouped by category instead of just 4 - mcp-tools reference: add 5 missing tools — update-document, suggest-tags, link-documents, get-document-links, delete-link - rest-api reference: add all missing endpoints, reorganize by category, add examples for update, bulk retag, webhooks, links, saved searches - configuration guide: document passthrough LLM provider - configuration reference: add passthrough LLM, llm.ollamaUrl key, expand config set examples to cover all settable keys - cli reference: expand config set supported keys list Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: comprehensive documentation update for v1.3.0 - README: fix license (BUSL-1.1, not MIT), expand MCP tools table to all 26 tools, expand REST API table with all endpoints (webhooks, links, analytics, connectors status, suggest-tags, bulk ops), add webhooks section with HMAC signing example, add missing CLI commands (bulk ops, saved searches, document links, docs update) - getting-started: fix Node.js requirement (20, not 18), add sections for web dashboard, organize/annotate features, REST API - mcp-setup: expand available tools section to list all 26 tools grouped by category instead of just 4 - mcp-tools reference: add 5 missing tools — update-document, suggest-tags, link-documents, get-document-links, delete-link - rest-api reference: add all missing endpoints, reorganize by category, add examples for update, bulk retag, webhooks, links, saved searches - configuration guide: document passthrough LLM provider - configuration reference: add passthrough LLM, llm.ollamaUrl key, expand config set examples to cover all settable keys - cli reference: expand config set supported keys list Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: allow release-please PRs to pass merge gate and trigger CI Two issues prevented PR #238 from getting CI runs: 1. merge-gate blocked release-please PRs — the gate only allowed 'development' as the source branch, but release-please uses 'release-please--branches--main--components--libscope'. Updated to allow any branch matching 'release-please--*'. 2. CI never ran on the PR — GitHub does not trigger workflows when GITHUB_TOKEN creates a PR (intentional security restriction to prevent infinite loops). Fixed by passing a PAT via secrets.GH_TOKEN to the release-please action so its PR creation triggers CI. Note: requires a 'GH_TOKEN' secret in repo settings — a classic PAT with repo and workflow scopes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: add development branch workflow (#327) * chore: add development branch workflow - Add merge-gate.yml: enforces only 'development' can merge into main - Update CI/CodeQL/Docker workflows to run on both main and development - Update dependabot.yml: target-branch set to development for all ecosystems - Update copilot-instructions.md: document branch workflow convention - Rulesets configured: Main (requires merge-gate + squash-only), Development (requires CI status checks + PR) - Default branch set to development - All open PRs retargeted to development * fix: skip Vercel preview deployments on non-main branches * chore: trigger check refresh --------- * feat: create-pack from local folder or URL sources (#329) * fix: comprehensive audit fixes — security, performance, resilience, API hardening Addresses findings from issue #314: - SSRF protection for webhook URLs (CRITICAL) - Scrub secrets from exports - Stored XSS prevention on document URL - O(n²) and N+1 fixes in bulk operations - Rate limit cache eviction improvement - SSE backpressure handling - Replace raw Error() with typed errors - Fetch timeouts on all network calls - Input validation on API parameters - Search query length limit - Silent catch block logging - DNS rebinding check fix - N+1 in Slack user resolution - Pagination on webhook/search list endpoints Closes #314 * Address PR review comments: fix SSE listener leak, preserve caller signals, add SSRF validation to webhook test, chunk SQL params, use dynamic import in test - SSE backpressure: create single disconnect promise, race against drain (no listener accumulation) - http-utils.ts/onenote.ts: use AbortSignal.any() to combine caller signal with timeout - Webhook test endpoint: validate URL with validateWebhookUrlSsrf before fetch - bulk.ts: chunk IN clause to 999 params max (SQLite limit) - webhooks.test.ts: dynamic import after vi.mock() for deterministic DNS mocking * ci: consolidate and fix CI/CD workflows - Merge lint + typecheck into single job (saves one npm ci) - Add concurrency groups to ci, docker, codeql (cancel stale runs) - Add dependency-review-action on PRs (block vulnerable deps) - Add workflow_call trigger to ci.yml for reusability - Remove duplicate npm publish from release.yml (release-please owns it) - Fix SDK paths: sdk-go/ → sdk/go/, sdk-python/ → sdk/python/ - Fix Dependabot paths to match actual SDK directories - Add github-actions ecosystem to Dependabot (keep actions up to date) * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb48187. * feat: add --from option to pack create for folder/URL sources Adds createPackFromSource() that builds packs directly from local folders, files, or URLs without requiring database interaction. CLI: libscope pack create --name my-pack --from ~/docs/ [--extensions .md,.html] [--exclude pattern] [--no-recursive] Features: - Walks directories recursively using registered parsers - Fetches URLs via fetchAndConvert - Supports extension filtering, exclude patterns, progress callback - Multiple --from sources supported - Output format identical to DB export (pack install works unchanged) Closes #328 * style: fix prettier formatting * feat: add gzip support for pack files (.json.gz) Pack files can now be compressed with gzip for smaller distribution: - writePackFile/readPackFile auto-detect gzip by extension or magic bytes - installPack accepts both .json and .json.gz files - createPackFromSource defaults to .json.gz output (source packs can be large) - createPack (DB export) still defaults to .json - Auto-detects gzip by magic bytes even if extension is .json 5 new tests covering gzip write, install, magic byte detection, and round-trip. * feat: add progress logging and fix dedup handling in pack install - Log each document as it's indexed so large installs show progress - Change pack install to use dedup: 'skip' for graceful duplicate handling - Make title+content_length dedup check respect the dedup mode setting (previously it always threw ValidationError regardless of dedup mode) * feat: auto-generate tags during pack creation and apply on install - Export tokenize() and add suggestTagsFromText() in tags.ts for DB-free tag generation - createPackFromSource() now auto-generates tags per document via TF-IDF - installPack() applies doc.tags via addTagsToDocument() after indexing --------- * feat: add HTML file parser for .html/.htm document indexing (#318) * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb48187. --------- * build(deps-dev): Bump eslint-config-prettier from 9.1.2 to 10.1.8 (#325) Bumps [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) from 9.1.2 to 10.1.8. - [Release notes](https://github.com/prettier/eslint-config-prettier/releases) - [Changelog](https://github.com/prettier/eslint-config-prettier/blob/main/CHANGELOG.md) - [Commits](https://github.com/prettier/eslint-config-prettier/commits/v10.1.8) --- updated-dependencies: - dependency-name: eslint-config-prettier dependency-version: 10.1.8 dependency-type: direct:development update-type: version-update:semver-major ... * build(deps-dev): Bump lint-staged from 16.3.1 to 16.3.2 Bumps the minor-and-patch group with 1 update: [lint-staged](https://github.com/lint-staged/lint-staged). Updates `lint-staged` from 16.3.1 to 16.3.2 - [Release notes](https://github.com/lint-staged/lint-staged/releases) - [Changelog](https://github.com/lint-staged/lint-staged/blob/main/CHANGELOG.md) - [Commits](lint-staged/lint-staged@v16.3.1...v16.3.2) --- updated-dependencies: - dependency-name: lint-staged dependency-version: 16.3.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch ... * build(deps): Bump the actions group with 5 updates Bumps the actions group with 5 updates: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `4` | `6` | | [actions/setup-node](https://github.com/actions/setup-node) | `4` | `6` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4` | `7` | | [actions/setup-python](https://github.com/actions/setup-python) | `5` | `6` | | [actions/setup-go](https://github.com/actions/setup-go) | `5` | `6` | Updates `actions/checkout` from 4 to 6 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v6) Updates `actions/setup-node` from 4 to 6 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v6) Updates `actions/upload-artifact` from 4 to 7 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v4...v7) Updates `actions/setup-python` from 5 to 6 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) Updates `actions/setup-go` from 5 to 6 - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... * build(deps-dev): Bump @types/node from 22.19.13 to 25.3.3 Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 22.19.13 to 25.3.3. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 25.3.3 dependency-type: direct:development update-type: version-update:semver-major ... * build(deps): Bump better-sqlite3 from 11.10.0 to 12.6.2 Bumps [better-sqlite3](https://github.com/WiseLibs/better-sqlite3) from 11.10.0 to 12.6.2. - [Release notes](https://github.com/WiseLibs/better-sqlite3/releases) - [Commits](WiseLibs/better-sqlite3@v11.10.0...v12.6.2) --- updated-dependencies: - dependency-name: better-sqlite3 dependency-version: 12.6.2 dependency-type: direct:production update-type: version-update:semver-major ... * build(deps-dev): Bump eslint from 9.39.3 to 10.0.2 Bumps [eslint](https://github.com/eslint/eslint) from 9.39.3 to 10.0.2. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v9.39.3...v10.0.2) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.2 dependency-type: direct:development update-type: version-update:semver-major ... * feat: add passthrough LLM mode for ask-question tool (#335) * feat: add passthrough LLM mode for ask-question tool Adds llm.provider = "passthrough" so the ask-question MCP tool returns retrieved context chunks directly to the calling LLM instead of requiring a separate OpenAI/Ollama provider. This is the natural design for MCP tools where the client already has an LLM (e.g. Claude Code). - config.ts: add "passthrough" to llm.provider union type and env var handling - rag.ts: add isPassthroughMode() helper and getContextForQuestion() which retrieves and formats context without an LLM call - mcp/server.ts: ask-question checks passthrough first and returns context directly; falls through to existing LLM path otherwise Enable via config: { "llm": { "provider": "passthrough" } } Enable via env var: LIBSCOPE_LLM_PROVIDER=passthrough * fix: format config.ts and include passthrough in provider override - Reformat long if-condition to satisfy prettier (printWidth: 100) - Fix logic bug: passthrough provider was checked in outer condition but not spread into overrides.llm.provider --------- * fix: address 9 audit findings from issue #332 (#333) * fix: address 9 audit findings from issue #332 Security - middleware: use timingSafeEqual for API key comparison (#2) - url-fetcher, http-utils: replace process-wide NODE_TLS_REJECT_UNAUTHORIZED mutation with per-request undici Agent to eliminate TLS race condition (#1) Bugs - indexing: re-throw unexpected embedding errors so transaction rolls back instead of silently committing chunks with no vector (#3) - search: replace correlated minRating subquery with avg_r.avg_rating from the pre-joined aggregate in FTS and LIKE search paths (#4) Performance - bulk: replace O(n²) docs.find() loops with pre-built Map; replace per-document getDocumentTags() calls with a single getDocumentTagsBatch() query (#5) - config: add 30-second TTL cache to loadConfig() so disk reads are not repeated on every request (#6) Code quality - routes: check res.write() return value to handle SSE backpressure (#7) - reindex: delegate to schema.createVectorTable() instead of duplicating the vec0 DDL inline (#8) - obsidian: replace hand-rolled parseSimpleYaml() with js-yaml, normalise Date objects back to ISO-8601 strings (#9) Docs - agents.md: expand architecture tree to include src/api/ and src/connectors/; add Security Patterns section with correct undici examples - CONTRIBUTING.md: fix check-suite command (npm test → npm run test:coverage) and correct coverage threshold (80% → actual 75%/74%) Tests - bulk.test: add dateFrom/dateTo filter coverage - config.test: add cache-hit test; call invalidateConfigCache() before env-var tests so TTL cache doesn't return stale results * fix: remove unused warnIfTlsBypassMissing function Dead code after conflict resolution chose the per-request undici Agent approach (which doesn't need a warning about NODE_TLS_REJECT_UNAUTHORIZED). * fix: update tests for config cache and retry semantics - Add invalidateConfigCache() before loadConfig() in 4 env-override tests that were failing because the 30s TTL cache introduced in the config module was returning stale results from the previous test's cache entry - Update http-utils retry assertion: maxRetries=2 means 1 initial + 2 retries = 3 total calls (loop is attempt <= maxRetries) --------- * feat: CLI logging improvements and pack installation performance (#330) (#336) - Add `src/cli/reporter.ts`: PrettyReporter (ANSI colors + \r progress bar), SilentReporter (no-op for verbose/JSON mode), `isVerbose()` and `createReporter()` factory. `LIBSCOPE_VERBOSE=1` env var alternative. - Update `setupLogging` to default to "silent" in CLI mode (pretty reporter handles user-facing output). Verbose/`--log-level` flags still route to structured JSON pino logs. Fix duplicate `initLogger` calls in onenote connect/disconnect commands to use `setupLogging` consistently. - Update `installPack` in `packs.ts` to support batch embedding and progress reporting: - New `InstallOptions` interface with `batchSize`, `resumeFrom`, `onProgress` fields - Batch documents: chunk all → single `provider.embedBatch` call per batch → single SQLite transaction per batch (avoids N embedding calls) - `resumeFrom` skips the first N documents (enables partial install resume after failure) - `InstallResult` now includes `errors` count - Add `--batch-size` and `--resume-from` CLI options to `pack install` - Tests: `tests/unit/reporter.test.ts` (17 tests covering PrettyReporter, SilentReporter, isVerbose, env var detection); extended `tests/unit/packs.test.ts` with 7 new tests for progress callbacks, batch efficiency, resumeFrom, embedBatch failure handling. * Claude/fix issue 331 s1qzu (#338) * build(deps): Bump the npm_and_yarn group across 1 directory with 2 updates (#337) Bumps the npm_and_yarn group with 2 updates in the / directory: [@hono/node-server](https://github.com/honojs/node-server) and [hono](https://github.com/honojs/hono). Updates `@hono/node-server` from 1.19.9 to 1.19.10 - [Release notes](https://github.com/honojs/node-server/releases) - [Commits](honojs/node-server@v1.19.9...v1.19.10) Updates `hono` from 4.12.3 to 4.12.5 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](honojs/hono@v4.12.3...v4.12.5) --- updated-dependencies: - dependency-name: "@hono/node-server" dependency-version: 1.19.10 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.5 dependency-type: indirect dependency-group: npm_and_yarn ... * fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (#341) * fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (closes #340) **SSRF (CWE-918 — CodeQL alert #28)** Replace the two-step validate-then-fetch approach in url-fetcher.ts with IP-pinned requests using node:http / node:https directly. validateUrl() resolves DNS and checks for private IPs, then the validated IP is passed straight to the TCP connection (hostname: pinnedIp, servername: original hostname for TLS SNI). There is now zero TOCTOU window between validation and the actual network request. The redundant post-fetch DNS rebinding check and the env-var-based TLS bypass wrapper are removed; rejectUnauthorized is now passed directly to the request options. An internal _setRequestImpl hook is exported for unit test injection so tests can stub responses without touching node:http / node:https. Tests are updated accordingly. **ReDoS (CWE-1333 — CodeQL alert #24)** Five regexes in confluence.ts used the pattern [^>]*ac:name="X"[^>]* — two [^>]* quantifiers around a fixed literal. For input that contains a large attribute blob without the target ac:name value, the engine must try all O(n²) splits before concluding no match (catastrophic backtracking). Fix: rewrite the leading [^>]* as (?:(?!ac:name="X")[^>])* — a negative lookahead prevents the quantifier from overlapping with the literal, making backtracking structurally impossible. * fix: replace remaining polynomial regexes in confluence.ts with indexOf helpers The conflict resolution left the original `/<ac:structured-macro [^>]*>([\s\S]*?) <\/ac:structured-macro>/gi` patterns in place for code blocks and info/tip panels (those were not part of the original security fix diff). These have the same O(n²) backtracking problem: with k opening tags and no closing tags, [\s\S]*? scans O(n - pos) chars per attempt, totalling O(n²). Replace the entire convertConfluenceStorage function with the indexOf-based approach (replaceStructuredMacros / replaceTagPairs / extractTagContent helpers) that eliminates all regex-based tag parsing. Also add removeSelfClosingMacros to handle the self-closing TOC case without regex, since the previous self-closing fix still used a [^>]*ac:name="toc"[^>]* pattern. --------- * feat: concurrent pack installation and -v verbose shorthand (issue #330) (#339) * feat: concurrent pack installation and -v verbose shorthand (issue #330) Add concurrent batch embedding to installPack for significant performance improvement on large packs, plus CLI ergonomics improvements. Key changes: - `InstallOptions.concurrency` (default: 4): controls how many embedBatch calls run simultaneously; embedding is I/O-bound so parallelism directly reduces wall-clock installation time - Refactor installPack to pre-chunk all documents upfront, then use a semaphore-based scheduler to run up to `concurrency` embedBatch calls concurrently while inserting completed batches in-order (SQLite requires serialised writes); progress callbacks fire after each batch as before - `pack install --concurrency <n>` CLI flag exposes the new option - `-v` shorthand for `--verbose` on the global program options - Fix transaction install-count tracking: count committed docs accurately without relying on subtract-on-failure arithmetic - Add 6 new tests covering concurrency=1 sequential, concurrency=4 parallel, multiple embedBatch calls per install, concurrency limit enforcement, incremental progress reporting, and partial-failure error counting https://claude.ai/code/session_019hzhbEgV1ysnGmFVXBWzkZ * fix: address all 4 Copilot review comments on PR #339 - Validate batchSize, concurrency, resumeFrom at the start of installPack and throw ValidationError for invalid values (comments 3 & 4). Concurrency <= 0 would silently hang the semaphore indefinitely. - Add CLI lower-bound guard: --concurrency < 1 exits with a user-facing error before ever calling installPack (comment 3). - Lazy chunking: pre-chunking all documents upfront held chunks for the entire pack in memory simultaneously. Batches now store only the raw documents; resolveBatch() chunks on demand right before embedBatch is called, so only one batch's worth of chunks is in memory at a time (comment 2). - Wrap provider.embedBatch() in try/catch so synchronous throws are converted to rejected Promises rather than escaping scheduleNext() and leaving the outer Promise permanently pending (comment 1). --------- * fix: address 7 pre-release bugs from audit (#342) (#344) - Guard JSON.parse in rowToWebhook with try/catch, default to [] - Guard JSON.parse in rowToSavedSearch with try/catch, default to null - Guard JSON.parse in loadDbConnectorConfig with try/catch, throw ConfigError - Push dateFrom/dateTo filters into SQL WHERE in listDocuments (before LIMIT) - Validate negative limit in resolveSelector, throw ValidationError - Replace manual substring extension parsing with path.extname() in packs.ts - Verified reporter.ts is already tracked on development (no action needed) - Added tests for all fixes (corrupted JSON, SQL-level date filtering, negative limit) Closes #342 * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) (#343) * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) Adds opt-in spidering to URL indexing. A single seed URL can now crawl and index an entire documentation site or wiki section in one call. New files: - src/core/link-extractor.ts: indexOf-based <a href> extraction, relative URL resolution, fragment stripping, dedup, scheme filtering. No regex. - src/core/spider.ts: BFS crawl engine with sameDomain, pathPrefix, excludePatterns (glob), maxPages (hard cap 200), maxDepth (hard cap 5), 10-min total timeout, robots.txt (User-agent: * and libscope), and 1s inter-request delay. Yields SpiderResult per page; returns SpiderStats. - tests/unit/link-extractor.test.ts: 25 tests covering relative resolution, dedup, fragment stripping, scheme filtering, attribute order, edge cases. - tests/unit/spider.test.ts: 20 tests covering BFS order, depth/page limits, domain + path + pattern filtering, cycle detection, robots.txt, partial failure recovery, stats, and abortReason reporting. Modified: - src/core/url-fetcher.ts: adds fetchRaw() export returning raw body + contentType + finalUrl before HTML-to-markdown conversion, so the spider can extract links from HTML before conversion. - src/api/routes.ts: POST /api/v1/documents/url now accepts spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns. Returns { documents, pagesIndexed, pagesCrawled, pagesSkipped, errors, abortReason }. - src/mcp/server.ts: submit-document tool gains spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns parameters. Safety: all fetched URLs pass through the existing SSRF validation in fetchRaw() (DNS resolution, private IP blocking, scheme allowlist). Hard limits (200 pages, depth 5, 10min) cannot be overridden by callers. robots.txt is fetched once per origin and Disallow rules are honoured. Individual page failures do not abort the crawl. Closes #315 * fix: resolve CI lint errors in spider implementation - Remove unnecessary type assertions (routes.ts, mcp/server.ts) — TypeScript already narrows SpiderResult/SpiderStats from the generator - Add explicit return type annotation on mock fetchRaw to satisfy no-unsafe-return rule in spider.test.ts - Replace .resolves.not.toThrow() with a direct assertion — vitest .resolves requires a Promise, not an async function * fix: address CodeQL security findings in spider/link-extractor link-extractor.ts (CodeQL #30 — incomplete URL scheme check): Replace the enumerated scheme blocklist (javascript:, vbscript:, etc.) with a strict http/https allowlist check on the resolved URL protocol. An allowlist is exhaustive by definition; a blocklist will always miss obscure schemes like vbscript:, blob:, or future additions. spider.ts (CodeQL #31 — incomplete multi-character sanitization): Replace the regex-based tag stripper /<[^>]+>/g in extractTitle() with an indexOf-based stripTags() function. The regex stops at the first > which can be inside a quoted attribute value (e.g. <img alt="a>b">), potentially leaving partial tag content in the extracted title. The new implementation walks quoted attribute values explicitly so no tag content leaks through regardless of its internal structure. * fix: address all Copilot review comments on spider PR (#343) - link-extractor: add word-boundary check in extractHref to prevent matching data-href, aria-href (false positives on non-href attributes) - spider: rename pagesIndexed → pagesFetched throughout (SpiderStats interface already used pagesFetched; sync implementation + tests) - spider: per-origin robots.txt cache (Map<origin, Set>) fetched lazily as new origins are encountered during crawl (was seed-only before) - spider: normalize to raw.finalUrl after redirects — visited set, yielded URL, and link-extraction base all use the canonical URL - routes: validate maxPages/maxDepth are finite positive integers - routes: change conditional spread &&-patterns to ternaries - routes: remove inner try/catch for spider fetch errors; add FetchError to top-level handler (consistent with single-URL mode → 502) - mcp/server: replace conditional spreads with explicit if assignments - mcp/server: validate spider=true requires url (throws ValidationError) - openapi: document spider request fields in IndexFromUrlRequest schema, add SpiderResponse schema, update 201 response to oneOf * style: fix prettier formatting in spider files --------- --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR is a “development → main” release merge for v1.3.0, bringing in new crawling/spidering capabilities, pack creation/installation enhancements (including gzip + concurrency), improved error-handling for MCP tools, and a broad set of test/doc updates.
Changes:
- Add URL spider/crawler support (core spider engine + REST/MCP integration) and link extraction utilities.
- Expand knowledge pack workflows (create from sources, gzip read/write, concurrent install, progress reporting).
- Improve robustness (config caching, safer TLS handling via per-request undici Agent, corrupted-JSON handling, logging tweaks) with extensive new/updated unit tests and docs.
Reviewed changes
Copilot reviewed 65 out of 66 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/webhooks.test.ts | Stabilize async assertions; add corrupted JSON coverage for webhook rows. |
| tests/unit/update-document.test.ts | Switch timestamp test to fake timers. |
| tests/unit/spider.test.ts | New comprehensive tests for spiderUrl crawling behavior. |
| tests/unit/saved-searches.test.ts | Add corrupted JSON coverage for saved search filters. |
| tests/unit/reporter.test.ts | New tests for CLI reporter output behaviors. |
| tests/unit/parsers.test.ts | Add HTML parser coverage and extension support assertions. |
| tests/unit/packs.test.ts | Large expansion of pack creation/install tests incl gzip + concurrency. |
| tests/unit/mcp-server.test.ts | New tests for MCP error helpers + core tool business logic. |
| tests/unit/link-extractor.test.ts | New tests for HTML link extraction and normalization. |
| tests/unit/http-utils.test.ts | Update retry call-count expectation. |
| tests/unit/connectors-config.test.ts | Add corrupted JSON test for connector DB config. |
| tests/unit/config.test.ts | Add config cache invalidation + caching test. |
| tests/unit/bulk.test.ts | Add date filtering and limit validation behavior tests. |
| src/providers/local.ts | Add minimal typed interface for transformers pipeline output. |
| src/mcp/server.ts | Factor MCP error helpers; add spider submit mode; passthrough ask behavior. |
| src/mcp/errors.ts | New shared MCP errorResponse/withErrorHandling utilities. |
| src/core/webhooks.ts | Handle corrupted events JSON; tighten SELECT columns. |
| src/core/versioning.ts | Log and tolerate corrupted version metadata JSON. |
| src/core/url-fetcher.ts | Use undici Agent dispatcher for self-signed TLS; add fetchRaw API. |
| src/core/tags.ts | Add batch tag fetch; export tokenize; add suggestTagsFromText helper. |
| src/core/spider.ts | New BFS spider engine with robots, caps, filtering, timeout, link extraction. |
| src/core/search.ts | Optimize minRating filter to use joined avg rating subquery. |
| src/core/saved-searches.ts | Handle corrupted saved search filters JSON with warning + null fallback. |
| src/core/reindex.ts | Delegate vector table creation to schema helper. |
| src/core/rag.ts | Add passthrough LLM provider mode and context-only retrieval function. |
| src/core/parsers/index.ts | Register new HtmlParser. |
| src/core/parsers/html.ts | New HTML-to-markdown parser with tag ignores. |
| src/core/packs.ts | Add gzip pack I/O; createPackFromSource; concurrent/batched install logic. |
| src/core/link-extractor.ts | New non-regex link extractor for HTML anchors. |
| src/core/indexing.ts | Add dedup modes for title+length; make embedding insert failures rollback. |
| src/core/export.ts | Improve logging when optional webhooks table missing. |
| src/core/documents.ts | Add dateFrom/dateTo list filters; use JS time for updated_at (testable). |
| src/core/bulk.ts | Push date filters into SQL; use batch tag fetch helper. |
| src/core/analytics.ts | Debug log when analytics table missing instead of silent ignore. |
| src/connectors/slack.ts | Improve debug logging for optional table cleanup failures. |
| src/connectors/obsidian.ts | Use js-yaml for frontmatter; normalize YAML dates. |
| src/connectors/index.ts | Throw typed ConfigError on corrupted JSON configs; improve cleanup logs. |
| src/connectors/http-utils.ts | Retry loop refactor; per-request undici Agent for TLS bypass. |
| src/connectors/confluence.ts | Replace regex for self-closing macro stripping with indexOf-based scan. |
| src/config.ts | Add llm passthrough provider; add 30s config cache + invalidation. |
| src/cli/reporter.ts | New pretty/silent CLI reporter abstraction. |
| src/cli/index.ts | Add verbose flag alias; integrate reporter into pack install; add source pack creation. |
| src/api/routes.ts | Add spider mode to URL indexing endpoint; map FetchError to 502. |
| src/api/openapi.ts | Document spider mode request/response schemas and examples. |
| package.json | Update deps (better-sqlite3, eslint/prettier config) + add undici. |
| package-lock.json | Lockfile updates for dependency bumps/additions. |
| docs/reference/rest-api.md | Expand and restructure REST API documentation/examples. |
| docs/reference/mcp-tools.md | Document additional MCP tools and parameters. |
| docs/reference/configuration.md | Update config docs for passthrough provider and new keys. |
| docs/reference/cli.md | Update supported config keys list. |
| docs/guide/mcp-setup.md | Update MCP tools overview list. |
| docs/guide/getting-started.md | Update Node requirement; add dashboard/API guidance. |
| docs/guide/configuration.md | Document passthrough provider mode. |
| agents.md | Expand architecture + security patterns (TLS dispatcher guidance, etc.). |
| README.md | Update MCP tool list, REST endpoints, CLI docs; license blurb update. |
| .github/workflows/sdk-python.yml | Bump action versions used by SDK Python CI. |
| .github/workflows/sdk-go.yml | Bump action versions used by SDK Go CI. |
| .github/workflows/release.yml | Bump checkout action version. |
| .github/workflows/release-python.yml | Bump checkout/setup-python action versions. |
| .github/workflows/release-please.yml | Add PAT token usage; bump actions versions. |
| .github/workflows/merge-gate.yml | New workflow enforcing merge-to-main source branch policy. |
| .github/workflows/docker.yml | Run on development too; bump checkout action version. |
| .github/workflows/codeql.yml | Run on development too; bump checkout action version. |
| .github/workflows/ci.yml | Run on development too; bump actions versions; bump upload-artifact. |
| .github/dependabot.yml | Target Dependabot PRs to development branch. |
| .github/copilot-instructions.md | Document required branch workflow guidance. |
Comment on lines
+37
to
43
| for (let attempt = 0; attempt <= maxRetries; attempt++) { | ||
| const fetchOptions = { | ||
| ...(options ?? {}), | ||
| ...(dispatcher ? { dispatcher: dispatcher as unknown } : {}), | ||
| } as RequestInit; | ||
| const response = await fetch(url, fetchOptions); | ||
|
|
There was a problem hiding this comment.
fetchWithRetry no longer enforces a timeout or composes the caller-provided options.signal with a default timeout signal. This can cause connector syncs to hang indefinitely on stalled connections. Consider restoring the previous AbortSignal timeout behavior (and combining signals) while still passing the per-request dispatcher when allowSelfSignedCerts is enabled.
Comment on lines
+345
to
+347
| // Apply filters (except for seed URL at depth 0 — always fetch it) | ||
| if (depth > 0) { | ||
| if (sameDomain && !isSameDomain(url, seedHostname)) { |
There was a problem hiding this comment.
The module header says robots.txt Disallow rules are honoured, but the implementation skips all filtering (including robots.txt) for the seed URL at depth 0 (if (depth > 0) { ... }). If the seed URL path is disallowed, the spider will still fetch it. Either apply robots checks to the seed as well, or adjust the header/docs to reflect the exception.
Comment on lines
+79
to
+81
| Supported providers: `openai`, `ollama`, `passthrough`. | ||
|
|
||
| The `passthrough` provider is for advanced integrations where you supply your own LLM responses externally. When set, the `ask` command emits an event stream that your application handles rather than calling an LLM directly. |
There was a problem hiding this comment.
The passthrough provider description says the ask command “emits an event stream … rather than calling an LLM directly”, but the current implementation (e.g. MCP ask-question) appears to return a single synthesized context prompt + sources. Please update this doc to match the actual passthrough behavior (or implement the described streaming behavior).
Comment on lines
+454
to
+462
| try { | ||
| const vecBuffer = Buffer.from(new Float32Array(embedding).buffer); | ||
| insertEmbedding.run(chunkId, vecBuffer); | ||
| } catch (err) { | ||
| const message = err instanceof Error ? err.message : String(err); | ||
| if (!message.includes("no such table")) { | ||
| log.warn({ chunkId, err }, "Failed to insert vector embedding"); | ||
| } | ||
| } |
There was a problem hiding this comment.
In installPack, failures inserting embeddings (e.g. dimension mismatch, vec0 constraint errors, etc.) are caught and the transaction continues, which can commit chunks without embeddings and make those chunks invisible to semantic search. indexDocument intentionally rethrows non-"no such table" errors to roll back; this path should match that behavior (either rethrow to rollback, or skip inserting the entire document/batch and count an error).
RobertLD
added a commit
that referenced
this pull request
Mar 6, 2026
) * Prepare for release (#345) * chore: add development branch workflow (#327) * chore: add development branch workflow - Add merge-gate.yml: enforces only 'development' can merge into main - Update CI/CodeQL/Docker workflows to run on both main and development - Update dependabot.yml: target-branch set to development for all ecosystems - Update copilot-instructions.md: document branch workflow convention - Rulesets configured: Main (requires merge-gate + squash-only), Development (requires CI status checks + PR) - Default branch set to development - All open PRs retargeted to development Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: skip Vercel preview deployments on non-main branches Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: trigger check refresh --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: create-pack from local folder or URL sources (#329) * fix: comprehensive audit fixes — security, performance, resilience, API hardening Addresses findings from issue #314: - SSRF protection for webhook URLs (CRITICAL) - Scrub secrets from exports - Stored XSS prevention on document URL - O(n²) and N+1 fixes in bulk operations - Rate limit cache eviction improvement - SSE backpressure handling - Replace raw Error() with typed errors - Fetch timeouts on all network calls - Input validation on API parameters - Search query length limit - Silent catch block logging - DNS rebinding check fix - N+1 in Slack user resolution - Pagination on webhook/search list endpoints Closes #314 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review comments: fix SSE listener leak, preserve caller signals, add SSRF validation to webhook test, chunk SQL params, use dynamic import in test - SSE backpressure: create single disconnect promise, race against drain (no listener accumulation) - http-utils.ts/onenote.ts: use AbortSignal.any() to combine caller signal with timeout - Webhook test endpoint: validate URL with validateWebhookUrlSsrf before fetch - bulk.ts: chunk IN clause to 999 params max (SQLite limit) - webhooks.test.ts: dynamic import after vi.mock() for deterministic DNS mocking Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: consolidate and fix CI/CD workflows - Merge lint + typecheck into single job (saves one npm ci) - Add concurrency groups to ci, docker, codeql (cancel stale runs) - Add dependency-review-action on PRs (block vulnerable deps) - Add workflow_call trigger to ci.yml for reusability - Remove duplicate npm publish from release.yml (release-please owns it) - Fix SDK paths: sdk-go/ → sdk/go/, sdk-python/ → sdk/python/ - Fix Dependabot paths to match actual SDK directories - Add github-actions ecosystem to Dependabot (keep actions up to date) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb481870c883f77278291b72245b1ca0b890a78c. * feat: add --from option to pack create for folder/URL sources Adds createPackFromSource() that builds packs directly from local folders, files, or URLs without requiring database interaction. CLI: libscope pack create --name my-pack --from ~/docs/ [--extensions .md,.html] [--exclude pattern] [--no-recursive] Features: - Walks directories recursively using registered parsers - Fetches URLs via fetchAndConvert - Supports extension filtering, exclude patterns, progress callback - Multiple --from sources supported - Output format identical to DB export (pack install works unchanged) Closes #328 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * style: fix prettier formatting Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add gzip support for pack files (.json.gz) Pack files can now be compressed with gzip for smaller distribution: - writePackFile/readPackFile auto-detect gzip by extension or magic bytes - installPack accepts both .json and .json.gz files - createPackFromSource defaults to .json.gz output (source packs can be large) - createPack (DB export) still defaults to .json - Auto-detects gzip by magic bytes even if extension is .json 5 new tests covering gzip write, install, magic byte detection, and round-trip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add progress logging and fix dedup handling in pack install - Log each document as it's indexed so large installs show progress - Change pack install to use dedup: 'skip' for graceful duplicate handling - Make title+content_length dedup check respect the dedup mode setting (previously it always threw ValidationError regardless of dedup mode) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: auto-generate tags during pack creation and apply on install - Export tokenize() and add suggestTagsFromText() in tags.ts for DB-free tag generation - createPackFromSource() now auto-generates tags per document via TF-IDF - installPack() applies doc.tags via addTagsToDocument() after indexing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add HTML file parser for .html/.htm document indexing (#318) * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb481870c883f77278291b72245b1ca0b890a78c. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * build(deps-dev): Bump eslint-config-prettier from 9.1.2 to 10.1.8 (#325) Bumps [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) from 9.1.2 to 10.1.8. - [Release notes](https://github.com/prettier/eslint-config-prettier/releases) - [Changelog](https://github.com/prettier/eslint-config-prettier/blob/main/CHANGELOG.md) - [Commits](https://github.com/prettier/eslint-config-prettier/commits/v10.1.8) --- updated-dependencies: - dependency-name: eslint-config-prettier dependency-version: 10.1.8 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): Bump lint-staged from 16.3.1 to 16.3.2 Bumps the minor-and-patch group with 1 update: [lint-staged](https://github.com/lint-staged/lint-staged). Updates `lint-staged` from 16.3.1 to 16.3.2 - [Release notes](https://github.com/lint-staged/lint-staged/releases) - [Changelog](https://github.com/lint-staged/lint-staged/blob/main/CHANGELOG.md) - [Commits](https://github.com/lint-staged/lint-staged/compare/v16.3.1...v16.3.2) --- updated-dependencies: - dependency-name: lint-staged dependency-version: 16.3.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): Bump the actions group with 5 updates Bumps the actions group with 5 updates: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `4` | `6` | | [actions/setup-node](https://github.com/actions/setup-node) | `4` | `6` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4` | `7` | | [actions/setup-python](https://github.com/actions/setup-python) | `5` | `6` | | [actions/setup-go](https://github.com/actions/setup-go) | `5` | `6` | Updates `actions/checkout` from 4 to 6 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v4...v6) Updates `actions/setup-node` from 4 to 6 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v4...v6) Updates `actions/upload-artifact` from 4 to 7 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v4...v7) Updates `actions/setup-python` from 5 to 6 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v5...v6) Updates `actions/setup-go` from 5 to 6 - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](https://github.com/actions/setup-go/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): Bump @types/node from 22.19.13 to 25.3.3 Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 22.19.13 to 25.3.3. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 25.3.3 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): Bump better-sqlite3 from 11.10.0 to 12.6.2 Bumps [better-sqlite3](https://github.com/WiseLibs/better-sqlite3) from 11.10.0 to 12.6.2. - [Release notes](https://github.com/WiseLibs/better-sqlite3/releases) - [Commits](https://github.com/WiseLibs/better-sqlite3/compare/v11.10.0...v12.6.2) --- updated-dependencies: - dependency-name: better-sqlite3 dependency-version: 12.6.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): Bump eslint from 9.39.3 to 10.0.2 Bumps [eslint](https://github.com/eslint/eslint) from 9.39.3 to 10.0.2. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](https://github.com/eslint/eslint/compare/v9.39.3...v10.0.2) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Robert DeRienzo <rderienzo@voloridge.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add passthrough LLM mode for ask-question tool (#335) * feat: add passthrough LLM mode for ask-question tool Adds llm.provider = "passthrough" so the ask-question MCP tool returns retrieved context chunks directly to the calling LLM instead of requiring a separate OpenAI/Ollama provider. This is the natural design for MCP tools where the client already has an LLM (e.g. Claude Code). - config.ts: add "passthrough" to llm.provider union type and env var handling - rag.ts: add isPassthroughMode() helper and getContextForQuestion() which retrieves and formats context without an LLM call - mcp/server.ts: ask-question checks passthrough first and returns context directly; falls through to existing LLM path otherwise Enable via config: { "llm": { "provider": "passthrough" } } Enable via env var: LIBSCOPE_LLM_PROVIDER=passthrough Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: format config.ts and include passthrough in provider override - Reformat long if-condition to satisfy prettier (printWidth: 100) - Fix logic bug: passthrough provider was checked in outer condition but not spread into overrides.llm.provider Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address 9 audit findings from issue #332 (#333) * fix: address 9 audit findings from issue #332 Security - middleware: use timingSafeEqual for API key comparison (#2) - url-fetcher, http-utils: replace process-wide NODE_TLS_REJECT_UNAUTHORIZED mutation with per-request undici Agent to eliminate TLS race condition (#1) Bugs - indexing: re-throw unexpected embedding errors so transaction rolls back instead of silently committing chunks with no vector (#3) - search: replace correlated minRating subquery with avg_r.avg_rating from the pre-joined aggregate in FTS and LIKE search paths (#4) Performance - bulk: replace O(n²) docs.find() loops with pre-built Map; replace per-document getDocumentTags() calls with a single getDocumentTagsBatch() query (#5) - config: add 30-second TTL cache to loadConfig() so disk reads are not repeated on every request (#6) Code quality - routes: check res.write() return value to handle SSE backpressure (#7) - reindex: delegate to schema.createVectorTable() instead of duplicating the vec0 DDL inline (#8) - obsidian: replace hand-rolled parseSimpleYaml() with js-yaml, normalise Date objects back to ISO-8601 strings (#9) Docs - agents.md: expand architecture tree to include src/api/ and src/connectors/; add Security Patterns section with correct undici examples - CONTRIBUTING.md: fix check-suite command (npm test → npm run test:coverage) and correct coverage threshold (80% → actual 75%/74%) Tests - bulk.test: add dateFrom/dateTo filter coverage - config.test: add cache-hit test; call invalidateConfigCache() before env-var tests so TTL cache doesn't return stale results Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused warnIfTlsBypassMissing function Dead code after conflict resolution chose the per-request undici Agent approach (which doesn't need a warning about NODE_TLS_REJECT_UNAUTHORIZED). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update tests for config cache and retry semantics - Add invalidateConfigCache() before loadConfig() in 4 env-override tests that were failing because the 30s TTL cache introduced in the config module was returning stale results from the previous test's cache entry - Update http-utils retry assertion: maxRetries=2 means 1 initial + 2 retries = 3 total calls (loop is attempt <= maxRetries) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: CLI logging improvements and pack installation performance (#330) (#336) - Add `src/cli/reporter.ts`: PrettyReporter (ANSI colors + \r progress bar), SilentReporter (no-op for verbose/JSON mode), `isVerbose()` and `createReporter()` factory. `LIBSCOPE_VERBOSE=1` env var alternative. - Update `setupLogging` to default to "silent" in CLI mode (pretty reporter handles user-facing output). Verbose/`--log-level` flags still route to structured JSON pino logs. Fix duplicate `initLogger` calls in onenote connect/disconnect commands to use `setupLogging` consistently. - Update `installPack` in `packs.ts` to support batch embedding and progress reporting: - New `InstallOptions` interface with `batchSize`, `resumeFrom`, `onProgress` fields - Batch documents: chunk all → single `provider.embedBatch` call per batch → single SQLite transaction per batch (avoids N embedding calls) - `resumeFrom` skips the first N documents (enables partial install resume after failure) - `InstallResult` now includes `errors` count - Add `--batch-size` and `--resume-from` CLI options to `pack install` - Tests: `tests/unit/reporter.test.ts` (17 tests covering PrettyReporter, SilentReporter, isVerbose, env var detection); extended `tests/unit/packs.test.ts` with 7 new tests for progress callbacks, batch efficiency, resumeFrom, embedBatch failure handling. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Claude/fix issue 331 s1qzu (#338) * build(deps): Bump the npm_and_yarn group across 1 directory with 2 updates (#337) Bumps the npm_and_yarn group with 2 updates in the / directory: [@hono/node-server](https://github.com/honojs/node-server) and [hono](https://github.com/honojs/hono). Updates `@hono/node-server` from 1.19.9 to 1.19.10 - [Release notes](https://github.com/honojs/node-server/releases) - [Commits](https://github.com/honojs/node-server/compare/v1.19.9...v1.19.10) Updates `hono` from 4.12.3 to 4.12.5 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](https://github.com/honojs/hono/compare/v4.12.3...v4.12.5) --- updated-dependencies: - dependency-name: "@hono/node-server" dependency-version: 1.19.10 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.5 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (#341) * fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (closes #340) **SSRF (CWE-918 — CodeQL alert #28)** Replace the two-step validate-then-fetch approach in url-fetcher.ts with IP-pinned requests using node:http / node:https directly. validateUrl() resolves DNS and checks for private IPs, then the validated IP is passed straight to the TCP connection (hostname: pinnedIp, servername: original hostname for TLS SNI). There is now zero TOCTOU window between validation and the actual network request. The redundant post-fetch DNS rebinding check and the env-var-based TLS bypass wrapper are removed; rejectUnauthorized is now passed directly to the request options. An internal _setRequestImpl hook is exported for unit test injection so tests can stub responses without touching node:http / node:https. Tests are updated accordingly. **ReDoS (CWE-1333 — CodeQL alert #24)** Five regexes in confluence.ts used the pattern [^>]*ac:name="X"[^>]* — two [^>]* quantifiers around a fixed literal. For input that contains a large attribute blob without the target ac:name value, the engine must try all O(n²) splits before concluding no match (catastrophic backtracking). Fix: rewrite the leading [^>]* as (?:(?!ac:name="X")[^>])* — a negative lookahead prevents the quantifier from overlapping with the literal, making backtracking structurally impossible. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: replace remaining polynomial regexes in confluence.ts with indexOf helpers The conflict resolution left the original `/<ac:structured-macro [^>]*>([\s\S]*?) <\/ac:structured-macro>/gi` patterns in place for code blocks and info/tip panels (those were not part of the original security fix diff). These have the same O(n²) backtracking problem: with k opening tags and no closing tags, [\s\S]*? scans O(n - pos) chars per attempt, totalling O(n²). Replace the entire convertConfluenceStorage function with the indexOf-based approach (replaceStructuredMacros / replaceTagPairs / extractTagContent helpers) that eliminates all regex-based tag parsing. Also add removeSelfClosingMacros to handle the self-closing TOC case without regex, since the previous self-closing fix still used a [^>]*ac:name="toc"[^>]* pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: concurrent pack installation and -v verbose shorthand (issue #330) (#339) * feat: concurrent pack installation and -v verbose shorthand (issue #330) Add concurrent batch embedding to installPack for significant performance improvement on large packs, plus CLI ergonomics improvements. Key changes: - `InstallOptions.concurrency` (default: 4): controls how many embedBatch calls run simultaneously; embedding is I/O-bound so parallelism directly reduces wall-clock installation time - Refactor installPack to pre-chunk all documents upfront, then use a semaphore-based scheduler to run up to `concurrency` embedBatch calls concurrently while inserting completed batches in-order (SQLite requires serialised writes); progress callbacks fire after each batch as before - `pack install --concurrency <n>` CLI flag exposes the new option - `-v` shorthand for `--verbose` on the global program options - Fix transaction install-count tracking: count committed docs accurately without relying on subtract-on-failure arithmetic - Add 6 new tests covering concurrency=1 sequential, concurrency=4 parallel, multiple embedBatch calls per install, concurrency limit enforcement, incremental progress reporting, and partial-failure error counting https://claude.ai/code/session_019hzhbEgV1ysnGmFVXBWzkZ * fix: address all 4 Copilot review comments on PR #339 - Validate batchSize, concurrency, resumeFrom at the start of installPack and throw ValidationError for invalid values (comments 3 & 4). Concurrency <= 0 would silently hang the semaphore indefinitely. - Add CLI lower-bound guard: --concurrency < 1 exits with a user-facing error before ever calling installPack (comment 3). - Lazy chunking: pre-chunking all documents upfront held chunks for the entire pack in memory simultaneously. Batches now store only the raw documents; resolveBatch() chunks on demand right before embedBatch is called, so only one batch's worth of chunks is in memory at a time (comment 2). - Wrap provider.embedBatch() in try/catch so synchronous throws are converted to rejected Promises rather than escaping scheduleNext() and leaving the outer Promise permanently pending (comment 1). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> * fix: address 7 pre-release bugs from audit (#342) (#344) - Guard JSON.parse in rowToWebhook with try/catch, default to [] - Guard JSON.parse in rowToSavedSearch with try/catch, default to null - Guard JSON.parse in loadDbConnectorConfig with try/catch, throw ConfigError - Push dateFrom/dateTo filters into SQL WHERE in listDocuments (before LIMIT) - Validate negative limit in resolveSelector, throw ValidationError - Replace manual substring extension parsing with path.extname() in packs.ts - Verified reporter.ts is already tracked on development (no action needed) - Added tests for all fixes (corrupted JSON, SQL-level date filtering, negative limit) Closes #342 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) (#343) * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) Adds opt-in spidering to URL indexing. A single seed URL can now crawl and index an entire documentation site or wiki section in one call. New files: - src/core/link-extractor.ts: indexOf-based <a href> extraction, relative URL resolution, fragment stripping, dedup, scheme filtering. No regex. - src/core/spider.ts: BFS crawl engine with sameDomain, pathPrefix, excludePatterns (glob), maxPages (hard cap 200), maxDepth (hard cap 5), 10-min total timeout, robots.txt (User-agent: * and libscope), and 1s inter-request delay. Yields SpiderResult per page; returns SpiderStats. - tests/unit/link-extractor.test.ts: 25 tests covering relative resolution, dedup, fragment stripping, scheme filtering, attribute order, edge cases. - tests/unit/spider.test.ts: 20 tests covering BFS order, depth/page limits, domain + path + pattern filtering, cycle detection, robots.txt, partial failure recovery, stats, and abortReason reporting. Modified: - src/core/url-fetcher.ts: adds fetchRaw() export returning raw body + contentType + finalUrl before HTML-to-markdown conversion, so the spider can extract links from HTML before conversion. - src/api/routes.ts: POST /api/v1/documents/url now accepts spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns. Returns { documents, pagesIndexed, pagesCrawled, pagesSkipped, errors, abortReason }. - src/mcp/server.ts: submit-document tool gains spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns parameters. Safety: all fetched URLs pass through the existing SSRF validation in fetchRaw() (DNS resolution, private IP blocking, scheme allowlist). Hard limits (200 pages, depth 5, 10min) cannot be overridden by callers. robots.txt is fetched once per origin and Disallow rules are honoured. Individual page failures do not abort the crawl. Closes #315 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: resolve CI lint errors in spider implementation - Remove unnecessary type assertions (routes.ts, mcp/server.ts) — TypeScript already narrows SpiderResult/SpiderStats from the generator - Add explicit return type annotation on mock fetchRaw to satisfy no-unsafe-return rule in spider.test.ts - Replace .resolves.not.toThrow() with a direct assertion — vitest .resolves requires a Promise, not an async function Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CodeQL security findings in spider/link-extractor link-extractor.ts (CodeQL #30 — incomplete URL scheme check): Replace the enumerated scheme blocklist (javascript:, vbscript:, etc.) with a strict http/https allowlist check on the resolved URL protocol. An allowlist is exhaustive by definition; a blocklist will always miss obscure schemes like vbscript:, blob:, or future additions. spider.ts (CodeQL #31 — incomplete multi-character sanitization): Replace the regex-based tag stripper /<[^>]+>/g in extractTitle() with an indexOf-based stripTags() function. The regex stops at the first > which can be inside a quoted attribute value (e.g. <img alt="a>b">), potentially leaving partial tag content in the extracted title. The new implementation walks quoted attribute values explicitly so no tag content leaks through regardless of its internal structure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address all Copilot review comments on spider PR (#343) - link-extractor: add word-boundary check in extractHref to prevent matching data-href, aria-href (false positives on non-href attributes) - spider: rename pagesIndexed → pagesFetched throughout (SpiderStats interface already used pagesFetched; sync implementation + tests) - spider: per-origin robots.txt cache (Map<origin, Set>) fetched lazily as new origins are encountered during crawl (was seed-only before) - spider: normalize to raw.finalUrl after redirects — visited set, yielded URL, and link-extraction base all use the canonical URL - routes: validate maxPages/maxDepth are finite positive integers - routes: change conditional spread &&-patterns to ternaries - routes: remove inner try/catch for spider fetch errors; add FetchError to top-level handler (consistent with single-URL mode → 502) - mcp/server: replace conditional spreads with explicit if assignments - mcp/server: validate spider=true requires url (throws ValidationError) - openapi: document spider request fields in IndexFromUrlRequest schema, add SpiderResponse schema, update 201 response to oneOf Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix prettier formatting in spider files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: bring main up to date with development for v1.3.0 (#353) * chore: add development branch workflow (#327) * chore: add development branch workflow - Add merge-gate.yml: enforces only 'development' can merge into main - Update CI/CodeQL/Docker workflows to run on both main and development - Update dependabot.yml: target-branch set to development for all ecosystems - Update copilot-instructions.md: document branch workflow convention - Rulesets configured: Main (requires merge-gate + squash-only), Development (requires CI status checks + PR) - Default branch set to development - All open PRs retargeted to development Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: skip Vercel preview deployments on non-main branches Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: trigger check refresh --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: create-pack from local folder or URL sources (#329) * fix: comprehensive audit fixes — security, performance, resilience, API hardening Addresses findings from issue #314: - SSRF protection for webhook URLs (CRITICAL) - Scrub secrets from exports - Stored XSS prevention on document URL - O(n²) and N+1 fixes in bulk operations - Rate limit cache eviction improvement - SSE backpressure handling - Replace raw Error() with typed errors - Fetch timeouts on all network calls - Input validation on API parameters - Search query length limit - Silent catch block logging - DNS rebinding check fix - N+1 in Slack user resolution - Pagination on webhook/search list endpoints Closes #314 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review comments: fix SSE listener leak, preserve caller signals, add SSRF validation to webhook test, chunk SQL params, use dynamic import in test - SSE backpressure: create single disconnect promise, race against drain (no listener accumulation) - http-utils.ts/onenote.ts: use AbortSignal.any() to combine caller signal with timeout - Webhook test endpoint: validate URL with validateWebhookUrlSsrf before fetch - bulk.ts: chunk IN clause to 999 params max (SQLite limit) - webhooks.test.ts: dynamic import after vi.mock() for deterministic DNS mocking Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: consolidate and fix CI/CD workflows - Merge lint + typecheck into single job (saves one npm ci) - Add concurrency groups to ci, docker, codeql (cancel stale runs) - Add dependency-review-action on PRs (block vulnerable deps) - Add workflow_call trigger to ci.yml for reusability - Remove duplicate npm publish from release.yml (release-please owns it) - Fix SDK paths: sdk-go/ → sdk/go/, sdk-python/ → sdk/python/ - Fix Dependabot paths to match actual SDK directories - Add github-actions ecosystem to Dependabot (keep actions up to date) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb481870c883f77278291b72245b1ca0b890a78c. * feat: add --from option to pack create for folder/URL sources Adds createPackFromSource() that builds packs directly from local folders, files, or URLs without requiring database interaction. CLI: libscope pack create --name my-pack --from ~/docs/ [--extensions .md,.html] [--exclude pattern] [--no-recursive] Features: - Walks directories recursively using registered parsers - Fetches URLs via fetchAndConvert - Supports extension filtering, exclude patterns, progress callback - Multiple --from sources supported - Output format identical to DB export (pack install works unchanged) Closes #328 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * style: fix prettier formatting Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add gzip support for pack files (.json.gz) Pack files can now be compressed with gzip for smaller distribution: - writePackFile/readPackFile auto-detect gzip by extension or magic bytes - installPack accepts both .json and .json.gz files - createPackFromSource defaults to .json.gz output (source packs can be large) - createPack (DB export) still defaults to .json - Auto-detects gzip by magic bytes even if extension is .json 5 new tests covering gzip write, install, magic byte detection, and round-trip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add progress logging and fix dedup handling in pack install - Log each document as it's indexed so large installs show progress - Change pack install to use dedup: 'skip' for graceful duplicate handling - Make title+content_length dedup check respect the dedup mode setting (previously it always threw ValidationError regardless of dedup mode) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: auto-generate tags during pack creation and apply on install - Export tokenize() and add suggestTagsFromText() in tags.ts for DB-free tag generation - createPackFromSource() now auto-generates tags per document via TF-IDF - installPack() applies doc.tags via addTagsToDocument() after indexing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add HTML file parser for .html/.htm document indexing (#318) * feat: add HTML file parser for .html/.htm document indexing Adds HtmlParser using the existing node-html-markdown dependency. Strips <script>, <style>, and <nav> tags before conversion. Registered for .html and .htm extensions. Includes 12 tests covering conversion, tag stripping, edge cases. Closes #317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address CodeQL and review comments on HTML parser - Replace regex-based tag stripping with node-html-markdown's native ignore option — eliminates all 3 CodeQL alerts (incomplete sanitization, bad HTML filtering regexp) - Wrap translate() in try/catch, throw ValidationError (consistent with other parsers) - Use trimEnd() instead of trim() to preserve leading indentation - Reuse single NHM instance for efficiency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert "fix: skip Vercel preview deployments on non-main branches" This reverts commit eb481870c883f77278291b72245b1ca0b890a78c. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * build(deps-dev): Bump eslint-config-prettier from 9.1.2 to 10.1.8 (#325) Bumps [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) from 9.1.2 to 10.1.8. - [Release notes](https://github.com/prettier/eslint-config-prettier/releases) - [Changelog](https://github.com/prettier/eslint-config-prettier/blob/main/CHANGELOG.md) - [Commits](https://github.com/prettier/eslint-config-prettier/commits/v10.1.8) --- updated-dependencies: - dependency-name: eslint-config-prettier dependency-version: 10.1.8 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): Bump lint-staged from 16.3.1 to 16.3.2 Bumps the minor-and-patch group with 1 update: [lint-staged](https://github.com/lint-staged/lint-staged). Updates `lint-staged` from 16.3.1 to 16.3.2 - [Release notes](https://github.com/lint-staged/lint-staged/releases) - [Changelog](https://github.com/lint-staged/lint-staged/blob/main/CHANGELOG.md) - [Commits](https://github.com/lint-staged/lint-staged/compare/v16.3.1...v16.3.2) --- updated-dependencies: - dependency-name: lint-staged dependency-version: 16.3.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): Bump the actions group with 5 updates Bumps the actions group with 5 updates: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `4` | `6` | | [actions/setup-node](https://github.com/actions/setup-node) | `4` | `6` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4` | `7` | | [actions/setup-python](https://github.com/actions/setup-python) | `5` | `6` | | [actions/setup-go](https://github.com/actions/setup-go) | `5` | `6` | Updates `actions/checkout` from 4 to 6 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v4...v6) Updates `actions/setup-node` from 4 to 6 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v4...v6) Updates `actions/upload-artifact` from 4 to 7 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v4...v7) Updates `actions/setup-python` from 5 to 6 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v5...v6) Updates `actions/setup-go` from 5 to 6 - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](https://github.com/actions/setup-go/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-node dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): Bump @types/node from 22.19.13 to 25.3.3 Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 22.19.13 to 25.3.3. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 25.3.3 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): Bump better-sqlite3 from 11.10.0 to 12.6.2 Bumps [better-sqlite3](https://github.com/WiseLibs/better-sqlite3) from 11.10.0 to 12.6.2. - [Release notes](https://github.com/WiseLibs/better-sqlite3/releases) - [Commits](https://github.com/WiseLibs/better-sqlite3/compare/v11.10.0...v12.6.2) --- updated-dependencies: - dependency-name: better-sqlite3 dependency-version: 12.6.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): Bump eslint from 9.39.3 to 10.0.2 Bumps [eslint](https://github.com/eslint/eslint) from 9.39.3 to 10.0.2. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](https://github.com/eslint/eslint/compare/v9.39.3...v10.0.2) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.2 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Robert DeRienzo <rderienzo@voloridge.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add passthrough LLM mode for ask-question tool (#335) * feat: add passthrough LLM mode for ask-question tool Adds llm.provider = "passthrough" so the ask-question MCP tool returns retrieved context chunks directly to the calling LLM instead of requiring a separate OpenAI/Ollama provider. This is the natural design for MCP tools where the client already has an LLM (e.g. Claude Code). - config.ts: add "passthrough" to llm.provider union type and env var handling - rag.ts: add isPassthroughMode() helper and getContextForQuestion() which retrieves and formats context without an LLM call - mcp/server.ts: ask-question checks passthrough first and returns context directly; falls through to existing LLM path otherwise Enable via config: { "llm": { "provider": "passthrough" } } Enable via env var: LIBSCOPE_LLM_PROVIDER=passthrough Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: format config.ts and include passthrough in provider override - Reformat long if-condition to satisfy prettier (printWidth: 100) - Fix logic bug: passthrough provider was checked in outer condition but not spread into overrides.llm.provider Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address 9 audit findings from issue #332 (#333) * fix: address 9 audit findings from issue #332 Security - middleware: use timingSafeEqual for API key comparison (#2) - url-fetcher, http-utils: replace process-wide NODE_TLS_REJECT_UNAUTHORIZED mutation with per-request undici Agent to eliminate TLS race condition (#1) Bugs - indexing: re-throw unexpected embedding errors so transaction rolls back instead of silently committing chunks with no vector (#3) - search: replace correlated minRating subquery with avg_r.avg_rating from the pre-joined aggregate in FTS and LIKE search paths (#4) Performance - bulk: replace O(n²) docs.find() loops with pre-built Map; replace per-document getDocumentTags() calls with a single getDocumentTagsBatch() query (#5) - config: add 30-second TTL cache to loadConfig() so disk reads are not repeated on every request (#6) Code quality - routes: check res.write() return value to handle SSE backpressure (#7) - reindex: delegate to schema.createVectorTable() instead of duplicating the vec0 DDL inline (#8) - obsidian: replace hand-rolled parseSimpleYaml() with js-yaml, normalise Date objects back to ISO-8601 strings (#9) Docs - agents.md: expand architecture tree to include src/api/ and src/connectors/; add Security Patterns section with correct undici examples - CONTRIBUTING.md: fix check-suite command (npm test → npm run test:coverage) and correct coverage threshold (80% → actual 75%/74%) Tests - bulk.test: add dateFrom/dateTo filter coverage - config.test: add cache-hit test; call invalidateConfigCache() before env-var tests so TTL cache doesn't return stale results Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused warnIfTlsBypassMissing function Dead code after conflict resolution chose the per-request undici Agent approach (which doesn't need a warning about NODE_TLS_REJECT_UNAUTHORIZED). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update tests for config cache and retry semantics - Add invalidateConfigCache() before loadConfig() in 4 env-override tests that were failing because the 30s TTL cache introduced in the config module was returning stale results from the previous test's cache entry - Update http-utils retry assertion: maxRetries=2 means 1 initial + 2 retries = 3 total calls (loop is attempt <= maxRetries) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: CLI logging improvements and pack installation performance (#330) (#336) - Add `src/cli/reporter.ts`: PrettyReporter (ANSI colors + \r progress bar), SilentReporter (no-op for verbose/JSON mode), `isVerbose()` and `createReporter()` factory. `LIBSCOPE_VERBOSE=1` env var alternative. - Update `setupLogging` to default to "silent" in CLI mode (pretty reporter handles user-facing output). Verbose/`--log-level` flags still route to structured JSON pino logs. Fix duplicate `initLogger` calls in onenote connect/disconnect commands to use `setupLogging` consistently. - Update `installPack` in `packs.ts` to support batch embedding and progress reporting: - New `InstallOptions` interface with `batchSize`, `resumeFrom`, `onProgress` fields - Batch documents: chunk all → single `provider.embedBatch` call per batch → single SQLite transaction per batch (avoids N embedding calls) - `resumeFrom` skips the first N documents (enables partial install resume after failure) - `InstallResult` now includes `errors` count - Add `--batch-size` and `--resume-from` CLI options to `pack install` - Tests: `tests/unit/reporter.test.ts` (17 tests covering PrettyReporter, SilentReporter, isVerbose, env var detection); extended `tests/unit/packs.test.ts` with 7 new tests for progress callbacks, batch efficiency, resumeFrom, embedBatch failure handling. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Claude/fix issue 331 s1qzu (#338) * build(deps): Bump the npm_and_yarn group across 1 directory with 2 updates (#337) Bumps the npm_and_yarn group with 2 updates in the / directory: [@hono/node-server](https://github.com/honojs/node-server) and [hono](https://github.com/honojs/hono). Updates `@hono/node-server` from 1.19.9 to 1.19.10 - [Release notes](https://github.com/honojs/node-server/releases) - [Commits](https://github.com/honojs/node-server/compare/v1.19.9...v1.19.10) Updates `hono` from 4.12.3 to 4.12.5 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](https://github.com/honojs/hono/compare/v4.12.3...v4.12.5) --- updated-dependencies: - dependency-name: "@hono/node-server" dependency-version: 1.19.10 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.5 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (#341) * fix: eliminate SSRF TOCTOU and ReDoS vulnerabilities (closes #340) **SSRF (CWE-918 — CodeQL alert #28)** Replace the two-step validate-then-fetch approach in url-fetcher.ts with IP-pinned requests using node:http / node:https directly. validateUrl() resolves DNS and checks for private IPs, then the validated IP is passed straight to the TCP connection (hostname: pinnedIp, servername: original hostname for TLS SNI). There is now zero TOCTOU window between validation and the actual network request. The redundant post-fetch DNS rebinding check and the env-var-based TLS bypass wrapper are removed; rejectUnauthorized is now passed directly to the request options. An internal _setRequestImpl hook is exported for unit test injection so tests can stub responses without touching node:http / node:https. Tests are updated accordingly. **ReDoS (CWE-1333 — CodeQL alert #24)** Five regexes in confluence.ts used the pattern [^>]*ac:name="X"[^>]* — two [^>]* quantifiers around a fixed literal. For input that contains a large attribute blob without the target ac:name value, the engine must try all O(n²) splits before concluding no match (catastrophic backtracking). Fix: rewrite the leading [^>]* as (?:(?!ac:name="X")[^>])* — a negative lookahead prevents the quantifier from overlapping with the literal, making backtracking structurally impossible. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: replace remaining polynomial regexes in confluence.ts with indexOf helpers The conflict resolution left the original `/<ac:structured-macro [^>]*>([\s\S]*?) <\/ac:structured-macro>/gi` patterns in place for code blocks and info/tip panels (those were not part of the original security fix diff). These have the same O(n²) backtracking problem: with k opening tags and no closing tags, [\s\S]*? scans O(n - pos) chars per attempt, totalling O(n²). Replace the entire convertConfluenceStorage function with the indexOf-based approach (replaceStructuredMacros / replaceTagPairs / extractTagContent helpers) that eliminates all regex-based tag parsing. Also add removeSelfClosingMacros to handle the self-closing TOC case without regex, since the previous self-closing fix still used a [^>]*ac:name="toc"[^>]* pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: concurrent pack installation and -v verbose shorthand (issue #330) (#339) * feat: concurrent pack installation and -v verbose shorthand (issue #330) Add concurrent batch embedding to installPack for significant performance improvement on large packs, plus CLI ergonomics improvements. Key changes: - `InstallOptions.concurrency` (default: 4): controls how many embedBatch calls run simultaneously; embedding is I/O-bound so parallelism directly reduces wall-clock installation time - Refactor installPack to pre-chunk all documents upfront, then use a semaphore-based scheduler to run up to `concurrency` embedBatch calls concurrently while inserting completed batches in-order (SQLite requires serialised writes); progress callbacks fire after each batch as before - `pack install --concurrency <n>` CLI flag exposes the new option - `-v` shorthand for `--verbose` on the global program options - Fix transaction install-count tracking: count committed docs accurately without relying on subtract-on-failure arithmetic - Add 6 new tests covering concurrency=1 sequential, concurrency=4 parallel, multiple embedBatch calls per install, concurrency limit enforcement, incremental progress reporting, and partial-failure error counting https://claude.ai/code/session_019hzhbEgV1ysnGmFVXBWzkZ * fix: address all 4 Copilot review comments on PR #339 - Validate batchSize, concurrency, resumeFrom at the start of installPack and throw ValidationError for invalid values (comments 3 & 4). Concurrency <= 0 would silently hang the semaphore indefinitely. - Add CLI lower-bound guard: --concurrency < 1 exits with a user-facing error before ever calling installPack (comment 3). - Lazy chunking: pre-chunking all documents upfront held chunks for the entire pack in memory simultaneously. Batches now store only the raw documents; resolveBatch() chunks on demand right before embedBatch is called, so only one batch's worth of chunks is in memory at a time (comment 2). - Wrap provider.embedBatch() in try/catch so synchronous throws are converted to rejected Promises rather than escaping scheduleNext() and leaving the outer Promise permanently pending (comment 1). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> * fix: address 7 pre-release bugs from audit (#342) (#344) - Guard JSON.parse in rowToWebhook with try/catch, default to [] - Guard JSON.parse in rowToSavedSearch with try/catch, default to null - Guard JSON.parse in loadDbConnectorConfig with try/catch, throw ConfigError - Push dateFrom/dateTo filters into SQL WHERE in listDocuments (before LIMIT) - Validate negative limit in resolveSelector, throw ValidationError - Replace manual substring extension parsing with path.extname() in packs.ts - Verified reporter.ts is already tracked on development (no action needed) - Added tests for all fixes (corrupted JSON, SQL-level date filtering, negative limit) Closes #342 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) (#343) * feat: URL spidering — crawl linked pages with configurable depth and page limits (#315) Adds opt-in spidering to URL indexing. A single seed URL can now crawl and index an entire documentation site or wiki section in one call. New files: - src/core/link-extractor.ts: indexOf-based <a href> extraction, relative URL resolution, fragment stripping, dedup, scheme filtering. No regex. - src/core/spider.ts: BFS crawl engine with sameDomain, pathPrefix, excludePatterns (glob), maxPages (hard cap 200), maxDepth (hard cap 5), 10-min total timeout, robots.txt (User-agent: * and libscope), and 1s inter-request delay. Yields SpiderResult per page; returns SpiderStats. - tests/unit/link-extractor.test.ts: 25 tests covering relative resolution, dedup, fragment stripping, scheme filtering, attribute order, edge cases. - tests/unit/spider.test.ts: 20 tests covering BFS order, depth/page limits, domain + path + pattern filtering, cycle detection, robots.txt, partial failure recovery, stats, and abortReason reporting. Modified: - src/core/url-fetcher.ts: adds fetchRaw() export returning raw body + contentType + finalUrl before HTML-to-markdown conversion, so the spider can extract links from HTML before conversion. - src/api/routes.ts: POST /api/v1/documents/url now accepts spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns. Returns { documents, pagesIndexed, pagesCrawled, pagesSkipped, errors, abortReason }. - src/mcp/server.ts: submit-document tool gains spider, maxPages, maxDepth, sameDomain, pathPrefix, excludePatterns parameters. Safety: all fetched URLs pass through the existing SSRF validation in fetchRaw() (DNS resolution, private IP blocking, scheme allowlist). Hard limits (200 pages, depth 5, 10min) cannot be overridden by callers. robots.txt is fetched once per origin and Disallow rules are honoured. Individual page failures do not abort the crawl. Closes #315 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: resolve CI lint errors in spider implementation - Remove unnecessary type assertions (routes.ts, mcp/server.ts) — TypeScript already narrows SpiderResult/SpiderStats from the generator - Add explicit return type annotation on mock fetchRaw to satisfy no-unsafe-return rule in spider.test.ts - Replace .resolves.not.toThrow() with a direct assertion — vitest .resolves requires a Promise, not an async function Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CodeQL security findings in spider/link-extractor link-extractor.ts (CodeQL #30 — incomplete URL scheme check): Replace the enumerated scheme blocklist (javascript:, vbscript:, etc.) with a strict http/https allowlist check on the resolved URL protocol. An allowlist is exhaustive by definition; a blocklist will always miss obscure schemes like vbscript:, blob:, or future additions. spider.ts (CodeQL #31 — incomplete multi-character sanitization): Replace the regex-based tag stripper /<[^>]+>/g in extractTitle() with an indexOf-based stripTags() function. The regex stops at the first > which can be inside a quoted attribute value (e.g. <img alt="a>b">), potentially leaving partial tag content in the extracted title. The new implementation walks quoted attribute values explicitly so no tag content leaks through regardless of its internal structure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address all Copilot review comments on spider PR (#343) - link-extractor: add word-boundary check in extractHref to prevent matching data-href, aria-href (false positives on non-href attributes) - spider: rename pagesIndexed → pagesFetched throughout (SpiderStats interface already used pagesFetched; sync implementation + tests) - spider: per-origin robots.txt cache (Map<origin, Set>) fetched lazily as new origins are encountered during crawl (was seed-only before) - spider: normalize to raw.finalUrl after redirects — visited set, yielded URL, and link-extraction base all use the canonical URL - routes: validate maxPages/maxDepth are finite positive integers - routes: change conditional spread &&-patterns to ternaries - routes: remove inner try/catch for spider fetch errors; add FetchError to top-level handler (consistent with single-URL mode → 502) - mcp/server: replace conditional spreads with explicit if assignments - mcp/server: validate spider=true requires url (throws ValidationError) - openapi: document spider request fields in IndexFromUrlRequest schema, add SpiderResponse schema, update 201 response to oneOf Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix prettier formatting in spider files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: comprehensive documentation update for v1.3.0 (#347) - README: fix license (BUSL-1.1, not MIT), expand MCP tools table to all 26 tools, expand REST API table with all endpoints (webhooks, links, analytics, connectors status, suggest-tags, bulk ops), add webhooks section with HMAC signing example, add missing CLI commands (bulk ops, saved searches, document links, docs update) - getting-started: fix Node.js requirement (20, not 18), add sections for web dashboard, organize/annotate features, REST API - mcp-setup: expand available tools section to list all 26 tools grouped by category instead of just 4 - mcp-tools reference: add 5 missing tools — update-document, suggest-tags, link-documents, get-document-links, delete-link - rest-api reference: add all missing endpoints, reorganize by category, add examples for update, bulk retag, webhooks, links, saved searches - configuration guide: document passthrough LLM provider - configuration reference: add passthrough LLM, llm.ollamaUrl key, expand config set examples to cover all settable keys - cli reference: expand config set supported keys list Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: allow release-please PRs to pass merge gate and trigger CI (#348) * docs: comprehensive documentation update for v1.3.0 - README: fix license (BUSL-1.1, not MIT), expand MCP tools table to all 26 tools, expand REST API table with all endpoints (webhooks, links, analytics, connectors status, suggest-tags, bulk ops), add webhooks section with HMAC signing example, add missing CLI commands (bulk ops, saved searches, document links, docs update) - getting-started: fix Node.js requirement (20, not 18), add sections for web dashboard, organize/annotate features, REST API - mcp-setup: expand available tools section to list all 26 tools grouped by category instead of just 4 - mcp-tools reference: add 5 missing tools — update-document, suggest-tags, link-documents, get-document-links, delete-link - rest-api reference: add all missing endpoints, reorganize by category, add examples for update, bulk retag, webhooks, links, saved searches - configuration guide: document passthrough LLM provider - configuration reference: add passthrough LLM, llm.ollamaUrl key, expand config set examples to cover all settable keys - cli reference: expand config set supported keys list Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: allow release-please PRs to pass merge gate and trigger CI Two issues prevented PR #238 from getting CI runs: 1. merge-gate blocked release-please PRs — the gate only allowed 'development' as the source branch, but release-please uses 'release-please--branches--main--components--libscope'. Updated to allow any branch matching 'release-please--*'. 2. CI never ran on the PR — GitHub does not trigger workflows when GITHUB_TOKEN creates a PR (intentional security restriction to prevent infinite loops). Fixed by passing a PAT via secrets.GH_TOKEN to the release-please action so its PR creation triggers CI. Note: requires a 'GH_TOKEN' secret in repo settings — a classic PAT with repo and workflow scopes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * ci: trigger checks --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: merge development into main for v1.3.0 release (#354) * chore: add development branch workflow (#327) * chore: add development branch workflow - Add merge-gate.yml: enforces only 'development' can merge into main - Update CI/CodeQL/Docker workflows to run on both main and development - Update dependabot.yml: target-branch set to development for all ecosystems - Update copilot-instructions.md: document branch workflow convention - Rulesets configured: Main (requires merge-gate + squash-only), Development (requires CI status checks + PR) - Default branch set to development - All open PRs retargeted to development Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: skip Vercel preview deployments on non-main branches Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: trigger check refresh --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: create-pack from local folder or URL sources (#329) * fix: comprehensive audit fixes — security, performance, resilience, API hardening Addresses findings from issue #31…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.