Skip to content

fix(deps-graph): preserve workspace lockfile entries on reimport#10318

Merged
zkochan merged 9 commits into
teambit:masterfrom
zkochan:deps-graph
Apr 22, 2026
Merged

fix(deps-graph): preserve workspace lockfile entries on reimport#10318
zkochan merged 9 commits into
teambit:masterfrom
zkochan:deps-graph

Conversation

@zkochan
Copy link
Copy Markdown
Member

@zkochan zkochan commented Apr 20, 2026

Summary

  • Re-importing a component into a populated workspace used to overwrite pnpm-lock.yaml with only the imported component's subgraph. pnpm then re-resolved every other workspace dep from the manifest specifiers and silently drifted to newer registry versions.
  • This PR makes PnpmPackageManager.dependenciesGraphToLockfile merge the graph-derived subset into the existing wanted lockfile instead of overwriting it.
  • Also fixes an in-place mutation in Scope.getDependenciesGraphByComponentIds that corrupted the cached Version._dependenciesGraph.
  • Adds e2e coverage for the drift scenarios, 3-way merge, and dev/optional dep round-trip.

Context / full analysis: #10317.

Changes

scopes/dependencies/pnpm/pnpm.package-manager.ts

dependenciesGraphToLockfile now reads the existing wanted lockfile and overlays the graph's importers / packages / snapshots onto it. The importer overlay merges per-importer, per-dep-type, per-dep-key — which keeps unrelated projects' existing deps intact even though convertGraphToLockfile emits a sparse importer entry for every workspace project.

components/legacy/scope/scope.ts

getDependenciesGraphByComponentIds clones the first loaded graph via serialize/deserialize before merging subsequent graphs into it. Version.loadDependenciesGraph caches the graph on the object, so merging into it in place was mutating the cache and leaking merged state to later callers.

e2e/harmony/deps-graph.e2e.ts

Four new describe blocks:

  1. importing a component into a workspace that already has an installed component — asserts the existing component's deps aren't re-resolved after importing a second component.
  2. re-importing an updated version of an already-imported component — asserts unrelated components' deps aren't re-resolved on update.
  3. three components sharing a peer dependency — 3-way merge, highest-wins on the root edge. Includes one assertion documenting a known remaining limitation where lower-version non-peer transitives stay in the lockfile.
  4. dev and optional dependencies round-trip through the graph.

Test plan

  • All 20 cases in e2e/harmony/deps-graph.e2e.ts pass locally.
  • npm run lint and npm run oxlint clean.
  • CI green.

Known follow-ups (tracked in #10317, not in this PR)

  • Transitive snapshot reconciliation on merge (lower-version non-peer packages still linger after a 3-way merge).
  • Coverage for: missing graph Source, patchedDependencies, overrides, directory-type resolutions, specifier drift between tag and import.
  • Duplicate root edges accumulated by DependenciesGraph.merge (cosmetic today).

zkochan added 2 commits April 20, 2026 13:37
…ev/optional round-trip

Covers gaps identified in teambit#10317:
- importing a new component into a populated workspace silently drifts
  the existing workspace's locked dependency versions
- re-importing an updated version of an already-imported component
  drifts unrelated components' locked versions
- three-component peer merge: highest-wins on the root edge, plus a
  known-limitation assertion documenting that lower-version transitive
  snapshots leak through
- dev and optional dependencies round-trip through the graph
…om graph

When installPackagesGracefully is called after writing imported components, it
only passes the newly-written component IDs to getDependenciesGraphByComponentIds.
PnpmPackageManager.dependenciesGraphToLockfile then wrote a lockfile containing
only that subgraph, overwriting pnpm-lock.yaml in its entirety. pnpm's subsequent
install re-resolved every other workspace dep from manifest specifiers, silently
drifting locked versions whenever the registry had a newer matching version.

Read the existing wanted lockfile first and overlay the graph-derived subset onto
it: graph wins for packages, snapshots, and per-importer per-dep keys it knows;
every other entry is preserved verbatim. The graph-emitted sparse importer blocks
for unrelated projects no longer erase their existing deps.

Also clone the first loaded graph before merging subsequent graphs into it.
Version.loadDependenciesGraph caches the deserialized graph on the Version object,
so merging into it in place was mutating the cached instance and leaking merged
state to later callers.

Fixes the two drift assertions in e2e/harmony/deps-graph.e2e.ts that documented
this bug; all 20 deps-graph e2e cases now pass.
@zkochan zkochan marked this pull request as ready for review April 20, 2026 12:45
Copilot AI review requested due to automatic review settings April 20, 2026 12:45
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds e2e coverage for dependency-graph-based lockfile restoration edge cases and introduces initial fixes to prevent lockfile drift and cached graph mutation during re-import/merge flows.

Changes:

  • Add new e2e scenarios covering re-import drift, 3-way graph merge behavior, and dev/optional dependency flag round-tripping.
  • Update pnpm lockfile restoration to merge graph-derived lockfile content into an existing wanted lockfile instead of overwriting it.
  • Prevent mutation of cached DependenciesGraph instances when merging graphs in Scope.getDependenciesGraphByComponentIds.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
scopes/dependencies/pnpm/pnpm.package-manager.ts Merges graph-derived lockfile data into an existing lockfile to avoid dependency drift on subsequent imports.
e2e/harmony/deps-graph.e2e.ts Adds e2e coverage for re-import drift reproduction and regression scenarios around graph merge/flags.
components/legacy/scope/scope.ts Avoids mutating cached dependency graphs by cloning before merge.

Comment thread scopes/dependencies/pnpm/pnpm.package-manager.ts Outdated
Comment thread scopes/dependencies/pnpm/pnpm.package-manager.ts
Comment thread scopes/dependencies/pnpm/pnpm.package-manager.ts
oxlint --deny-warnings caught these via no-useless-fallback-in-spread.
Object spread of null/undefined is a no-op, so the fallbacks are noise.
@zkochan zkochan changed the title test(deps-graph): e2e coverage for reimport drift and graph merge gaps fix(deps-graph): preserve workspace lockfile entries on reimport Apr 20, 2026
…equiringBuild

Address Copilot review feedback on the overlay merge:

1. Packages and snapshots were shallow-merged by entry key, so any overlap
   replaced the existing lockfile's entry with the graph's subset. The graph
   only round-trips 12 fields of LockfilePackageInfo, so fields pnpm managed
   independently (e.g. `optional`, `transitivePeerDependencies`, `dev`) were
   silently dropped. Deep-merge per key instead: graph wins for fields it
   carries, existing entry's other fields survive.

2. The graph-derived lockfile's `bit.depsRequiringBuild` was dropped when
   merging into an existing lockfile. Union the two lists (deduped, sorted)
   inside the merge helper so build-required packages from both the re-stated
   subgraph and whatever was already locked are retained.
Copilot AI review requested due to automatic review settings April 21, 2026 10:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

components/legacy/scope/scope.ts:759

  • getDependenciesGraphByComponentIds() mutates allGraph from within a Promise.all(componentIds.map(async ...)) loop. Because the callbacks run concurrently, multiple iterations can observe allGraph == null and assign it, causing some graphs to be dropped (last writer wins) instead of merged. Collect the per-component graphs first (e.g. const graphs = await Promise.all(...)), then merge them deterministically in a single synchronous loop (or iterate for...of with await).
  public async getDependenciesGraphByComponentIds(componentIds: ComponentID[]): Promise<DependenciesGraph | undefined> {
    let allGraph: DependenciesGraph | undefined;
    if (!isFeatureEnabled(DEPS_GRAPH)) return undefined;
    await Promise.all(
      componentIds.map(async (componentId) => {
        const graph = await this.getDependenciesGraphByComponentId(componentId);
        if (graph == null || graph.isEmpty()) return;
        if (allGraph == null) {
          // loadDependenciesGraph caches the graph on the Version object; merging into
          // it in place would mutate the cached instance and corrupt subsequent callers.
          allGraph = DependenciesGraph.deserialize(graph.serialize());
        } else {
          allGraph.merge(graph);
        }
      })
    );

Comment thread e2e/harmony/deps-graph.e2e.ts Outdated
Comment thread e2e/harmony/deps-graph.e2e.ts Outdated
zkochan added 2 commits April 22, 2026 12:15
Copilot review caught that the inline comments still said "this assertion
currently fails" — accurate when the tests landed ahead of the fix, misleading
now that the fix is in the same PR. Rephrase as regression coverage describing
what the merge helper must preserve, without claiming a failing state.

The 3-component transitive-leak assertion keeps its "currently" wording because
that limitation is not fixed here and is tracked as follow-up in teambit#10317.
Copilot AI review requested due to automatic review settings April 22, 2026 10:18
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread scopes/dependencies/pnpm/pnpm.package-manager.ts Outdated
Copilot review: convertGraphToLockfile hardcodes lockfileVersion '9.0', so
preferring graph.lockfileVersion in the merge helper could silently downgrade
workspaces whose pnpm writes a newer schema and trigger a full rewrite on the
next install — the opposite of the goal here. Flip the fallback so existing
wins whenever present; graph's value is only used when starting from no
lockfile at all.
@zkochan zkochan requested a review from GiladShoham April 22, 2026 13:42
Copilot AI review requested due to automatic review settings April 22, 2026 14:00
@zkochan zkochan enabled auto-merge (squash) April 22, 2026 14:00
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

components/legacy/scope/scope.ts:762

  • getDependenciesGraphByComponentIds uses pMapPool(..., { concurrency: concurrentComponentsLimit() }) but mutates the shared allGraph inside the concurrent mapper. With concurrency > 1, multiple tasks can observe allGraph == null and overwrite it, or interleave allGraph.merge(graph) calls, producing a nondeterministic/incomplete merged graph. Consider collecting the per-component graphs concurrently (e.g., return each graph from the mapper) and then merging them sequentially in a deterministic order, or guard the allGraph assignment/merge with a mutex (or run this mapper with concurrency 1 just for the merge step).
    await pMapPool(
      componentIds,
      async (componentId) => {
        const graph = await this.getDependenciesGraphByComponentId(componentId);
        if (graph == null || graph.isEmpty()) return;
        if (allGraph == null) {
          // loadDependenciesGraph caches the graph on the Version object; merging into
          // it in place would mutate the cached instance and corrupt subsequent callers.
          allGraph = DependenciesGraph.deserialize(graph.serialize());
        } else {
          allGraph.merge(graph);
        }
      },
      { concurrency: concurrentComponentsLimit() }

Comment on lines +568 to +579
function mergeEntryRecords<T extends object>(
existing: Record<string, T> | undefined,
graph: Record<string, T> | undefined
): Record<string, T> | undefined {
if (!existing) return graph;
if (!graph) return existing;
const merged: Record<string, T> = { ...existing };
for (const [key, graphEntry] of Object.entries(graph)) {
const existingEntry = merged[key];
merged[key] = existingEntry ? ({ ...existingEntry, ...graphEntry } as T) : graphEntry;
}
return merged;
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mergeEntryRecords() only shallow-merges each packages[...] / snapshots[...] entry ({ ...existingEntry, ...graphEntry }). If a nested object exists on both sides (notably resolution in packages[...]), the graph value replaces the entire nested object and can drop fields that the graph doesn't round-trip (e.g. tarball), even though the goal here is to preserve pnpm-managed metadata. Consider doing a targeted nested merge for known nested objects (e.g. resolution: { ...existingEntry.resolution, ...graphEntry.resolution }) and/or using a deep-merge strategy that keeps existing nested keys when graph omits them.

Copilot uses AI. Check for mistakes.
@zkochan zkochan merged commit 5e24a65 into teambit:master Apr 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants