feat: create-pack command from local folder or URL

[RobertLD]

Problem

Currently creating a distributable pack requires a two-step workflow:

1. libscope add <folder> — index files into the database
2. libscope pack create --name foo — export from DB to JSON

This is clunky when you just want to package a folder of docs (HTML, Markdown, PDF, etc.) into a .json pack file for distribution without polluting your local database.

Proposed Solution

Add a libscope pack create --from <source> option that builds a pack directly from:

Source types

• Local folder: libscope pack create --name activ-aop --from ~/docs/activ/aop/
• URL: libscope pack create --name my-pack --from https://example.com/docs/page
• Glob pattern: libscope pack create --name my-pack --from "~/docs/**/*.html"

Implementation Plan

Phase 1: Folder source

1. Add --from <path> option to pack create CLI command
2. When --from is provided, skip DB export and instead:
   • Walk the directory recursively, respecting existing parser extensions
   • Parse each file using the registered parser (Markdown, HTML, CSV, JSON, YAML, PDF, Word)
   • Chunk each document using existing chunkText() logic
   • Build the pack JSON structure in memory
   • Write to output file
3. Support --extensions filter (default: all registered parser extensions)
4. Support --recursive flag (default: true)
5. Support --exclude <glob> to skip files (e.g. --exclude "*.min.js")

Phase 2: URL source

1. When --from is a URL, fetch the page using existing fetchAndParse() from url-fetcher
2. If --spider is available (see feat: URL spidering — crawl linked pages with configurable depth and page limits #315), optionally crawl linked pages
3. Build pack from fetched content

Phase 3: Mixed sources

1. Allow multiple --from flags: --from ~/docs/ --from https://example.com/api
2. Deduplicate by content hash

Pack structure (unchanged)

{
  "name": "activ-aop",
  "version": "1.0.0",
  "description": "ACTIV AOP documentation",
  "documents": [
    {
      "title": "Getting Started",
      "content": "# Getting Started\n...",
      "sourceUrl": "file:///home/user/docs/getting-started.html",
      "library": "activ-aop"
    }
  ],
  "metadata": {
    "author": "...",
    "createdAt": "...",
    "documentCount": 42
  }
}

CLI examples

# Create pack from local HTML docs folder
libscope pack create --name activ-aop --from ~/docs/activ/aop/ --description "ACTIV AOP docs"

# Create pack from markdown files only
libscope pack create --name my-pack --from ~/docs/ --extensions .md,.html

# Create pack excluding assets
libscope pack create --name my-pack --from ~/docs/ --exclude "*.css" --exclude "*.js" --exclude "*.png"

# Create pack from URL
libscope pack create --name api-docs --from https://example.com/docs/

Key considerations

• Reuse existing parsers from src/core/parsers/ — no new parsing logic needed
• Reuse chunkText() from src/core/indexing.ts for consistent chunking
• No database interaction required (in-memory only)
• Output should be identical format to existing pack create (DB export) so pack install works unchanged
• Large folders should show progress (file count, current file)