fix(deps-graph): preserve workspace lockfile entries on reimport#10318
Conversation
…ev/optional round-trip Covers gaps identified in teambit#10317: - importing a new component into a populated workspace silently drifts the existing workspace's locked dependency versions - re-importing an updated version of an already-imported component drifts unrelated components' locked versions - three-component peer merge: highest-wins on the root edge, plus a known-limitation assertion documenting that lower-version transitive snapshots leak through - dev and optional dependencies round-trip through the graph
…om graph When installPackagesGracefully is called after writing imported components, it only passes the newly-written component IDs to getDependenciesGraphByComponentIds. PnpmPackageManager.dependenciesGraphToLockfile then wrote a lockfile containing only that subgraph, overwriting pnpm-lock.yaml in its entirety. pnpm's subsequent install re-resolved every other workspace dep from manifest specifiers, silently drifting locked versions whenever the registry had a newer matching version. Read the existing wanted lockfile first and overlay the graph-derived subset onto it: graph wins for packages, snapshots, and per-importer per-dep keys it knows; every other entry is preserved verbatim. The graph-emitted sparse importer blocks for unrelated projects no longer erase their existing deps. Also clone the first loaded graph before merging subsequent graphs into it. Version.loadDependenciesGraph caches the deserialized graph on the Version object, so merging into it in place was mutating the cached instance and leaking merged state to later callers. Fixes the two drift assertions in e2e/harmony/deps-graph.e2e.ts that documented this bug; all 20 deps-graph e2e cases now pass.
There was a problem hiding this comment.
Pull request overview
Adds e2e coverage for dependency-graph-based lockfile restoration edge cases and introduces initial fixes to prevent lockfile drift and cached graph mutation during re-import/merge flows.
Changes:
- Add new e2e scenarios covering re-import drift, 3-way graph merge behavior, and dev/optional dependency flag round-tripping.
- Update pnpm lockfile restoration to merge graph-derived lockfile content into an existing wanted lockfile instead of overwriting it.
- Prevent mutation of cached
DependenciesGraphinstances when merging graphs inScope.getDependenciesGraphByComponentIds.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
scopes/dependencies/pnpm/pnpm.package-manager.ts |
Merges graph-derived lockfile data into an existing lockfile to avoid dependency drift on subsequent imports. |
e2e/harmony/deps-graph.e2e.ts |
Adds e2e coverage for re-import drift reproduction and regression scenarios around graph merge/flags. |
components/legacy/scope/scope.ts |
Avoids mutating cached dependency graphs by cloning before merge. |
oxlint --deny-warnings caught these via no-useless-fallback-in-spread. Object spread of null/undefined is a no-op, so the fallbacks are noise.
…equiringBuild Address Copilot review feedback on the overlay merge: 1. Packages and snapshots were shallow-merged by entry key, so any overlap replaced the existing lockfile's entry with the graph's subset. The graph only round-trips 12 fields of LockfilePackageInfo, so fields pnpm managed independently (e.g. `optional`, `transitivePeerDependencies`, `dev`) were silently dropped. Deep-merge per key instead: graph wins for fields it carries, existing entry's other fields survive. 2. The graph-derived lockfile's `bit.depsRequiringBuild` was dropped when merging into an existing lockfile. Union the two lists (deduped, sorted) inside the merge helper so build-required packages from both the re-stated subgraph and whatever was already locked are retained.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
components/legacy/scope/scope.ts:759
getDependenciesGraphByComponentIds()mutatesallGraphfrom within aPromise.all(componentIds.map(async ...))loop. Because the callbacks run concurrently, multiple iterations can observeallGraph == nulland assign it, causing some graphs to be dropped (last writer wins) instead of merged. Collect the per-component graphs first (e.g.const graphs = await Promise.all(...)), then merge them deterministically in a single synchronous loop (or iteratefor...ofwithawait).
public async getDependenciesGraphByComponentIds(componentIds: ComponentID[]): Promise<DependenciesGraph | undefined> {
let allGraph: DependenciesGraph | undefined;
if (!isFeatureEnabled(DEPS_GRAPH)) return undefined;
await Promise.all(
componentIds.map(async (componentId) => {
const graph = await this.getDependenciesGraphByComponentId(componentId);
if (graph == null || graph.isEmpty()) return;
if (allGraph == null) {
// loadDependenciesGraph caches the graph on the Version object; merging into
// it in place would mutate the cached instance and corrupt subsequent callers.
allGraph = DependenciesGraph.deserialize(graph.serialize());
} else {
allGraph.merge(graph);
}
})
);
Copilot review caught that the inline comments still said "this assertion currently fails" — accurate when the tests landed ahead of the fix, misleading now that the fix is in the same PR. Rephrase as regression coverage describing what the merge helper must preserve, without claiming a failing state. The 3-component transitive-leak assertion keeps its "currently" wording because that limitation is not fixed here and is tracked as follow-up in teambit#10317.
Copilot review: convertGraphToLockfile hardcodes lockfileVersion '9.0', so preferring graph.lockfileVersion in the merge helper could silently downgrade workspaces whose pnpm writes a newer schema and trigger a full rewrite on the next install — the opposite of the goal here. Flip the fallback so existing wins whenever present; graph's value is only used when starting from no lockfile at all.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
components/legacy/scope/scope.ts:762
getDependenciesGraphByComponentIdsusespMapPool(..., { concurrency: concurrentComponentsLimit() })but mutates the sharedallGraphinside the concurrent mapper. With concurrency > 1, multiple tasks can observeallGraph == nulland overwrite it, or interleaveallGraph.merge(graph)calls, producing a nondeterministic/incomplete merged graph. Consider collecting the per-component graphs concurrently (e.g., return each graph from the mapper) and then merging them sequentially in a deterministic order, or guard theallGraphassignment/merge with a mutex (or run this mapper with concurrency 1 just for the merge step).
await pMapPool(
componentIds,
async (componentId) => {
const graph = await this.getDependenciesGraphByComponentId(componentId);
if (graph == null || graph.isEmpty()) return;
if (allGraph == null) {
// loadDependenciesGraph caches the graph on the Version object; merging into
// it in place would mutate the cached instance and corrupt subsequent callers.
allGraph = DependenciesGraph.deserialize(graph.serialize());
} else {
allGraph.merge(graph);
}
},
{ concurrency: concurrentComponentsLimit() }
| function mergeEntryRecords<T extends object>( | ||
| existing: Record<string, T> | undefined, | ||
| graph: Record<string, T> | undefined | ||
| ): Record<string, T> | undefined { | ||
| if (!existing) return graph; | ||
| if (!graph) return existing; | ||
| const merged: Record<string, T> = { ...existing }; | ||
| for (const [key, graphEntry] of Object.entries(graph)) { | ||
| const existingEntry = merged[key]; | ||
| merged[key] = existingEntry ? ({ ...existingEntry, ...graphEntry } as T) : graphEntry; | ||
| } | ||
| return merged; |
There was a problem hiding this comment.
mergeEntryRecords() only shallow-merges each packages[...] / snapshots[...] entry ({ ...existingEntry, ...graphEntry }). If a nested object exists on both sides (notably resolution in packages[...]), the graph value replaces the entire nested object and can drop fields that the graph doesn't round-trip (e.g. tarball), even though the goal here is to preserve pnpm-managed metadata. Consider doing a targeted nested merge for known nested objects (e.g. resolution: { ...existingEntry.resolution, ...graphEntry.resolution }) and/or using a deep-merge strategy that keeps existing nested keys when graph omits them.
Summary
pnpm-lock.yamlwith only the imported component's subgraph. pnpm then re-resolved every other workspace dep from the manifest specifiers and silently drifted to newer registry versions.PnpmPackageManager.dependenciesGraphToLockfilemerge the graph-derived subset into the existing wanted lockfile instead of overwriting it.Scope.getDependenciesGraphByComponentIdsthat corrupted the cachedVersion._dependenciesGraph.Context / full analysis: #10317.
Changes
scopes/dependencies/pnpm/pnpm.package-manager.tsdependenciesGraphToLockfilenow reads the existing wanted lockfile and overlays the graph'simporters/packages/snapshotsonto it. The importer overlay merges per-importer, per-dep-type, per-dep-key — which keeps unrelated projects' existing deps intact even thoughconvertGraphToLockfileemits a sparse importer entry for every workspace project.components/legacy/scope/scope.tsgetDependenciesGraphByComponentIdsclones the first loaded graph viaserialize/deserializebefore merging subsequent graphs into it.Version.loadDependenciesGraphcaches the graph on the object, so merging into it in place was mutating the cache and leaking merged state to later callers.e2e/harmony/deps-graph.e2e.tsFour new
describeblocks:importing a component into a workspace that already has an installed component— asserts the existing component's deps aren't re-resolved after importing a second component.re-importing an updated version of an already-imported component— asserts unrelated components' deps aren't re-resolved on update.three components sharing a peer dependency— 3-way merge, highest-wins on the root edge. Includes one assertion documenting a known remaining limitation where lower-version non-peer transitives stay in the lockfile.dev and optional dependencies round-trip through the graph.Test plan
e2e/harmony/deps-graph.e2e.tspass locally.npm run lintandnpm run oxlintclean.Known follow-ups (tracked in #10317, not in this PR)
patchedDependencies,overrides,directory-type resolutions, specifier drift between tag and import.DependenciesGraph.merge(cosmetic today).