Conversation
This was referenced Apr 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.dbfile per library into./artifacts/instead of inserting directly into the main database. Each artifact carries its ownlib_idin the meta table and is self-contained.cmd/consolidatecommand: Merges all per-lib artifact.dbfiles from./artifacts/into the maindeadzone.dbin a single atomic transaction. Idempotent — replaces existing rows perlib_idwith a full delete-then-insert within one tx.OpenArtifactininternal/db: New entry point for opening per-lib databases withlib_ididentity tracking, validation, and mismatch detection (ErrArtifactLibIDMissing,ErrArtifactLibIDMismatch).schema_versionto the meta table and cross-checks on open, surfacingErrSchemaMismatchfor incompatible databases.scrape,consolidate,serve, andcleanrecipes for the new two-stage pipeline.Motivation
Decouples per-lib scraping from the shared database so individual libraries can be re-scraped independently (
just scrape lib=/org/project) without touching other libs' data. The consolidate step is explicit and atomic — partial failures leave the main DB untouched.Test plan
just test— newconsolidate_test.gocovers validation pass (embedder mismatch, schema mismatch, missing lib_id), merge pass (single/multi artifact, idempotent re-merge), and rollback on mid-merge failurejust build— all three commands compilejust scrape lib=/some/lib && just consolidate— end-to-end artifact creation and mergeFixes #28