Skip to content

feat(ucd-store): improve ucd-store#450

Merged
luxass merged 23 commits intomainfrom
feat/ucd-store-fixes-redo
Jan 11, 2026
Merged

feat(ucd-store): improve ucd-store#450
luxass merged 23 commits intomainfrom
feat/ucd-store-fixes-redo

Conversation

@luxass
Copy link
Member

@luxass luxass commented Jan 8, 2026

🔗 Linked issue

📚 Description

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced file operations with local and API fallback support for retrieving files, listing directories, and exploring file trees
    • Improved manifest structure with detailed file metadata (name, path, storage path)
    • Store mirror, sync, and analysis operations for better version management
    • Better CLI formatting and structured output for store operations
  • Refactoring

    • Reorganized internal store architecture for improved modularity
    • Updated CLI command structure and help messaging

✏️ Tip: You can customize this high-level summary in your review settings.

@changeset-bot
Copy link

changeset-bot bot commented Jan 8, 2026

⚠️ No Changeset found

Latest commit: a60a641

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 8, 2026

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

The PR restructures the UCD manifest and store system. It converts manifest file lists from strings to structured objects with name, path, and storePath properties. The ucd-store package is refactored with new context management, file operations organized into files/reports/tasks modules, and improved error handling. CLI store commands are reorganized with new flags and output formatting changes.

Changes

Cohort / File(s) Summary
API Manifest & Setup
.github/workflows/refresh-manifest.yaml, apps/api/scripts/generate-manifest.ts, apps/api/scripts/lib/manifest.ts, apps/api/scripts/setup-dev/setup.ts, apps/api/scripts/setup-dev/setup-worker.ts, apps/api/package.json
Manifest generation refactored to use modular buildManifest and fetchExpectedFilesForVersion helpers; predev script added; tar archive now contains only manifest.json; setup-dev worker implements local manifest upload endpoint with R2 bucket support.
API Routes
apps/api/src/routes/.well-known/ucd-store/$version.ts, apps/api/src/routes/tasks/routes.ts
Per-version manifest routes updated to use UCDStoreVersionManifestSchema; upload-manifest now requires version query parameter instead of extracting from metadata.
Schemas
packages/schemas/src/manifest.ts, packages/schemas/src/index.ts, packages/schemas/test/manifest.test.ts
New ExpectedFileSchema with name, path, storePath fields replacing string-based expectedFiles; updated exports and test fixtures.
CLI Store Commands
packages/cli/src/cmd/store/_shared.ts, packages/cli/src/cmd/store/init.ts, packages/cli/src/cmd/store/sync.ts, packages/cli/src/cmd/store/analyze.ts, packages/cli/src/cmd/store/mirror.ts, packages/cli/src/cmd/store/verify.ts, packages/cli/src/cmd/store/status.ts, packages/cli/src/cmd/store/root.ts
Store commands refactored with new shared flags (LOCAL_STORE_FLAGS, REMOTE_CAPABLE_FLAGS), structured output system replacing console logs, and new versionStrategy/requireExistingStore options.
CLI Output & Tests
packages/cli/src/output.ts, packages/cli/test/cmd/store/...test.ts
List function updated to accept heterogeneous items with filePath property; test assertions swapped from containsInfo to containsLog; mocked API endpoints expanded.
Test Utilities
packages/test-utils/src/mock-store/index.ts, packages/test-utils/src/mock-store/handlers/well-known.ts
New mockStoreSubdomain function for store subdomain mocking; buildExpectedFiles helper constructs ExpectedFile objects from API paths.
UCD Store Core
packages/ucd-store/src/context.ts, packages/ucd-store/src/types.ts, packages/ucd-store/src/store.ts, packages/ucd-store/src/factory.ts
Major restructuring: createInternalContext and createPublicContext added; UCDStoreOptions made generic with BridgeOptionsSchema; context now tracks lockfile state and dual version sources; factory functions refactored for node/HTTP stores.
UCD Store Files Operations
packages/ucd-store/src/files/get.ts, packages/ucd-store/src/files/list.ts, packages/ucd-store/src/files/tree.ts
New file operation modules with overloaded signatures supporting both explicit context and bound this patterns; GetFileOptions/ListFilesOptions/GetFileTreeOptions enable API fallback control.
UCD Store Reports & Tasks
packages/ucd-store/src/reports/analyze.ts, packages/ucd-store/src/tasks/mirror.ts, packages/ucd-store/src/tasks/sync.ts
New analyze, mirror, and sync implementations with structured reporting; AnalysisReport, MirrorReport, SyncResult types introduced; operations track metrics, file counts, and per-version details.
UCD Store Utilities
packages/ucd-store/src/utils/lockfile.ts, packages/ucd-store/src/utils/verify.ts, packages/ucd-store/src/utils/validate.ts, packages/ucd-store/src/utils/reports.ts
New utility modules for lockfile initialization, version verification, validation against API, and report aggregation/formatting.
UCD Store Errors & Index
packages/ucd-store/src/errors.ts, packages/ucd-store/src/index.ts
New error classes UCDStoreFilterError and UCDStoreApiFallbackError; public type exports expanded for AnalyzeOptions, MirrorOptions, SyncOptions and report types.
UCD Store Dependencies
packages/ucd-store/package.json
Replaced pathe dependency with @ucdjs/path-utils workspace import.
UCD Store Removed Modules
packages/ucd-store/src/core/context.ts, packages/ucd-store/src/core/files.ts, packages/ucd-store/src/core/manifest.ts, packages/ucd-store/src/setup/bootstrap.ts, packages/ucd-store/src/setup/verify.ts, packages/ucd-store/src/operations/...
Old context, file path, manifest, bootstrap, verify, and operation modules (analyze, get, list, tree, mirror) removed in favor of new modular structure under context.ts, files/, reports/, tasks/, utils/.
UCD Store Integration Tests (New)
packages/ucd-store/test/integration/node/store.test.ts, packages/ucd-store/test/integration/node/files.test.ts, packages/ucd-store/test/integration/http/files.test.ts, packages/ucd-store/test/config-discovery.test.ts
New integration test suites for Node and HTTP stores, file operations, and config discovery workflows.
UCD Store Unit Tests (New)
packages/ucd-store/test/files/get.test.ts, packages/ucd-store/test/files/list.test.ts, packages/ucd-store/test/files/tree.test.ts, packages/ucd-store/test/reports/analyze.test.ts, packages/ucd-store/test/tasks/mirror.test.ts, packages/ucd-store/test/tasks/sync.test.ts, packages/ucd-store/test/context.test.ts, packages/ucd-store/test/helpers/test-context.ts
Comprehensive new unit tests for file operations, reports, tasks, context management, and test helpers.
UCD Store Tests (Removed)
packages/ucd-store/test/core/...test.ts, packages/ucd-store/test/operations/...test.ts, packages/ucd-store/test/integration/store-operations.test.ts
Old test files for removed modules deleted; test coverage migrated to new module structure.
UCD Store Utility Tests
packages/ucd-store/test/utils/lockfile.test.ts, packages/ucd-store/test/utils/verify.test.ts, packages/ucd-store/test/version-conflict.test.ts
Test updates for new lockfile and verify utility APIs; version-conflict assertions adjusted for partial object matching.
VSCode Extension
vscode/src/lib/files.ts
Updated to use store.files.tree(version) instead of store.getFileTree(version).
Client Tests
packages/client/test/resources/manifest.test.ts
Mock manifest updated with ExpectedFile objects replacing string entries; test validations added for object structure.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • PR #448: Overlapping test-utils mock-store changes (addPathsToFileNodes, mock-store types/utilities, handler updates)
  • PR #347: Matching ucd-store v2 functionality (context factories, manifest helpers, file/analyze/mirror operations)
  • PR #443: Both modify CLI output system (replacing info with log, expanding output helpers, updating test assertions)

Suggested labels

pkg: cli, pkg: api, pkg: schemas, pkg: ucd-store, pkg: test-utils, refactor, feature

Poem

🐰 Whiskers twitch with joy today,
Manifests in objects' way,
Store restructured, clean and bright,
Files now named and pathed just right!
From strings to shapes, a grand redesign,
All the pieces now align! 🎉

✨ Finishing touches
  • 📝 Generate docstrings

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8272127 and a60a641.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (85)
  • .github/workflows/refresh-manifest.yaml
  • apps/api/package.json
  • apps/api/scripts/generate-manifest.ts
  • apps/api/scripts/lib/manifest.ts
  • apps/api/scripts/setup-dev/setup-worker.ts
  • apps/api/scripts/setup-dev/setup.ts
  • apps/api/src/routes/.well-known/ucd-store/$version.ts
  • apps/api/src/routes/tasks/routes.ts
  • apps/api/wrangler-types.d.ts
  • packages/cli/src/cmd/store/_shared.ts
  • packages/cli/src/cmd/store/analyze.ts
  • packages/cli/src/cmd/store/init.ts
  • packages/cli/src/cmd/store/mirror.ts
  • packages/cli/src/cmd/store/root.ts
  • packages/cli/src/cmd/store/status.ts
  • packages/cli/src/cmd/store/sync.ts
  • packages/cli/src/cmd/store/verify.ts
  • packages/cli/src/output.ts
  • packages/cli/test/cmd/store/analyze.test.ts
  • packages/cli/test/cmd/store/init.test.ts
  • packages/cli/test/cmd/store/mirror.test.ts
  • packages/cli/test/cmd/store/status.test.ts
  • packages/cli/test/cmd/store/sync.test.ts
  • packages/cli/test/cmd/store/verify.test.ts
  • packages/client/test/resources/manifest.test.ts
  • packages/schemas/src/index.ts
  • packages/schemas/src/manifest.ts
  • packages/schemas/test/manifest.test.ts
  • packages/test-utils/src/mock-store/handlers/well-known.ts
  • packages/test-utils/src/mock-store/index.ts
  • packages/ucd-store/package.json
  • packages/ucd-store/src/context.ts
  • packages/ucd-store/src/core/context.ts
  • packages/ucd-store/src/core/files.ts
  • packages/ucd-store/src/core/manifest.ts
  • packages/ucd-store/src/errors.ts
  • packages/ucd-store/src/factory.ts
  • packages/ucd-store/src/files/get.ts
  • packages/ucd-store/src/files/list.ts
  • packages/ucd-store/src/files/tree.ts
  • packages/ucd-store/src/index.ts
  • packages/ucd-store/src/operations/analyze.ts
  • packages/ucd-store/src/operations/files/get.ts
  • packages/ucd-store/src/operations/files/list.ts
  • packages/ucd-store/src/operations/files/tree.ts
  • packages/ucd-store/src/operations/mirror.ts
  • packages/ucd-store/src/reports/analyze.ts
  • packages/ucd-store/src/setup/bootstrap.ts
  • packages/ucd-store/src/setup/verify.ts
  • packages/ucd-store/src/store.ts
  • packages/ucd-store/src/tasks/mirror.ts
  • packages/ucd-store/src/tasks/sync.ts
  • packages/ucd-store/src/types.ts
  • packages/ucd-store/src/utils/lockfile.ts
  • packages/ucd-store/src/utils/reports.ts
  • packages/ucd-store/src/utils/validate.ts
  • packages/ucd-store/src/utils/verify.ts
  • packages/ucd-store/test/config-discovery.test.ts
  • packages/ucd-store/test/context.test.ts
  • packages/ucd-store/test/core/config-discovery.test.ts
  • packages/ucd-store/test/core/context.test.ts
  • packages/ucd-store/test/core/files.test.ts
  • packages/ucd-store/test/core/lockfile.test.ts
  • packages/ucd-store/test/core/snapshot.test.ts
  • packages/ucd-store/test/files/get.test.ts
  • packages/ucd-store/test/files/list.test.ts
  • packages/ucd-store/test/files/tree.test.ts
  • packages/ucd-store/test/helpers/test-context.ts
  • packages/ucd-store/test/integration/http/files.test.ts
  • packages/ucd-store/test/integration/node/files.test.ts
  • packages/ucd-store/test/integration/node/store.test.ts
  • packages/ucd-store/test/integration/store-operations.test.ts
  • packages/ucd-store/test/operations/analyze.test.ts
  • packages/ucd-store/test/operations/files/get.test.ts
  • packages/ucd-store/test/operations/files/list.test.ts
  • packages/ucd-store/test/operations/files/tree.test.ts
  • packages/ucd-store/test/operations/mirror.test.ts
  • packages/ucd-store/test/operations/sync.test.ts
  • packages/ucd-store/test/reports/analyze.test.ts
  • packages/ucd-store/test/tasks/mirror.test.ts
  • packages/ucd-store/test/tasks/sync.test.ts
  • packages/ucd-store/test/utils/lockfile.test.ts
  • packages/ucd-store/test/utils/verify.test.ts
  • packages/ucd-store/test/version-conflict.test.ts
  • vscode/src/lib/files.ts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

🌏 Preview Deployments

Application Status Preview URL
API ⏳ In Progress N/A
Website ⏳ In Progress N/A

Built from commit: a60a641fbf1a66783f590458c11988f92f23f73b


🤖 This comment will be updated automatically when you push new commits to this PR.

@luxass luxass force-pushed the feat/ucd-store-fixes-redo branch from 4228b89 to a121117 Compare January 8, 2026 04:24
@luxass luxass force-pushed the feat/ucd-store-fixes-redo branch from 8874128 to e8f3414 Compare January 8, 2026 04:57
@luxass luxass force-pushed the feat/ucd-store-fixes-redo branch 3 times, most recently from eba2f38 to 02199e8 Compare January 11, 2026 05:39
@luxass luxass force-pushed the feat/ucd-store-fixes-redo branch 2 times, most recently from 8d9b3cc to 4381315 Compare January 11, 2026 12:07
luxass added 14 commits January 11, 2026 13:52
Removed the redundant `isContext` function and replaced it with `isUCDStoreInternalContext` for better clarity. Improved the `analyze` function to handle context and options more intuitively.
- Updated command tables in `store/root.ts` for clarity.
- Enhanced output formatting in `store/status.ts`, `store/sync.ts`, and `store/verify.ts` to use consistent logging methods.
- Refactored error handling and success messages for better user experience.
- Adjusted tests in `analyze.test.ts`, `init.test.ts`, `mirror.test.ts`, `status.test.ts`, `sync.test.ts`, and `verify.test.ts` to reflect changes in output methods.
- Introduced a `normalizeApiPath` function to streamline path normalization for mirroring.
- Updated the mirroring process to utilize a "merge" strategy for version handling.
- Improved debug logging for better traceability during file operations.
- Removed unnecessary console error logging in the file listing process.
Updated the file reporting structure in the `_mirror` function to use a `ReportFile` interface for better clarity and consistency. This change includes modifying the types for downloaded, skipped, and failed files, as well as adjusting the normalization of file paths for filtering.
Modified the assertions in the mirror tests to validate the structure of the downloaded files report, ensuring it includes both `name` and `filePath` for each file.
- Introduced custom error classes for better error management.
- Updated `assertLocalStore` and `assertRemoteOrStoreDir` functions to throw specific errors.
- Enhanced error reporting in `runCLI` to utilize the new `CLIError` class.
- Adjusted error message formatting in `runMirrorStore` for clarity.
Updated the file access methods in the `ucd-store` package to utilize the new store subdomain (`ucd-store.ucdjs.dev`). This change simplifies the path structure by removing the `/ucd/` prefix and enhances security by preventing path traversal vulnerabilities.

All related tests have been updated to reflect these changes, ensuring that file operations now correctly reference the new path format.
@luxass luxass force-pushed the feat/ucd-store-fixes-redo branch from 4381315 to ff4184f Compare January 11, 2026 12:53
- Simplified the manifest generation by introducing `ExpectedFile` schema.
- Updated the `upload-manifest` endpoint to accept version as a query parameter.
- Changed the tar creation to only include `manifest.json`.
- Enhanced error handling and logging for better traceability.
- Updated tests to reflect changes in expected file structure.

This refactor improves the clarity and maintainability of the manifest handling process.
@github-actions
Copy link
Contributor

github-actions bot commented Jan 11, 2026

📋 OpenAPI Schema Analysis

Summary

Schema Components

Change Type Count Details
🟢 Added 2 New schema components
🔴 Removed 0
🟡 Modified 0 Changed schema definitions

Overall Status

Status Result
⚠️ Breaking Changes No - No breaking changes detected

Detailed Changes

📋 Schema Components Changes

Added Schemas (2):

  • UCDStoreVersionManifest
  • ExpectedFile

🤖 This comment is automatically updated when you push new commits.

@luxass luxass marked this pull request as ready for review January 11, 2026 15:48
Copilot AI review requested due to automatic review settings January 11, 2026 15:48
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@luxass luxass merged commit 7ac2e43 into main Jan 11, 2026
22 of 27 checks passed
@luxass luxass deleted the feat/ucd-store-fixes-redo branch January 11, 2026 15:48
@codecov
Copy link

codecov bot commented Jan 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the ucd-store package architecture, reorganizing internal operations and updating the public API structure. The changes primarily involve moving from a flat operations structure to a namespaced approach with tasks and reports, updating test assertions for better type safety, and replacing the pathe dependency with a workspace-local @ucdjs/path-utils package.

Changes:

  • Reorganized operations into tasks/ (sync, mirror) and reports/ (analyze) directories
  • Updated public API: store.getFileTree()store.files.tree()
  • Replaced pathe dependency with @ucdjs/path-utils
  • Improved test assertions using expect.objectContaining() for partial matching
  • Updated error messages from "verification" to "validation" for consistency
  • Added comprehensive integration tests for Node.js and HTTP environments

Reviewed changes

Copilot reviewed 84 out of 86 changed files in this pull request and generated no comments.

Show a summary per file
File Description
vscode/src/lib/files.ts Updated API call from store.getFileTree() to store.files.tree()
pnpm-lock.yaml Replaced pathe dependency with @ucdjs/path-utils workspace package
packages/ucd-store/package.json Updated dependencies to use workspace @ucdjs/path-utils
packages/ucd-store/test/version-conflict.test.ts Changed assertions to use expect.objectContaining() for partial matching
packages/ucd-store/test/utils/verify.test.ts Updated function import path and error message text
packages/ucd-store/test/utils/lockfile.test.ts Renamed bootstrapinitLockfile, updated assertions and error messages
packages/ucd-store/test/tasks/*.test.ts New comprehensive test files for sync, mirror operations
packages/ucd-store/test/reports/analyze.test.ts New test file for analyze reporting functionality
packages/ucd-store/test/integration/*.test.ts New integration tests for Node.js and HTTP store implementations
packages/ucd-store/test/files/*.test.ts New test files for file operations (get, list, tree)
packages/ucd-store/test/helpers/test-context.ts Refactored context creation with improved type safety
Multiple deleted test files Removed old test organization under operations/ and core/
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added pkg: cli api Changes related to the API. 🚨 ci Changes related to our CI pipelines labels Jan 11, 2026
@coderabbitai coderabbitai bot mentioned this pull request Mar 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Changes related to the API. 🚨 ci Changes related to our CI pipelines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants