Repository navigation
Let plugins contribute touched-project locators, so affected can see non-npm dependency changes
#36909
Hoffs
started this conversation in
Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
A plugin can emit
externalNodesfromcreateNodesand reference them from an{"externalDependencies": [...]}input, and task hashing honours that per project.nx affectedcannot see any of it, because selection and hashing are separate systems and a plugin has no way into the first one.filterAffected(packages/nx/src/project-graph/affected/affected-project-graph.ts) runs a hardcoded array:The npm path works because both halves exist.
getTouchedNpmPackages(packages/nx/src/plugins/js/project-graph/affected/npm-packages.ts) hardcodesf.file === 'package.json', diffs the JSON, resolves each changed dependency to an external node, and pushes the external node's name intotouchedProjects;@nx/jshas already created project →npm:pkgedges, so the reverse walk infilterAffectedProjectsreaches the dependents.Neither half is reachable from a plugin. Edges alone do nothing, because nothing ever puts a non-npm external node into
touchedProjects. The only lever left isgetImplicitlyTouchedProjects, which matches changed files against each project's{workspaceRoot}/...file inputs — so the workaround is to declare the manifest as a file input, which puts it straight back into the hash and destroys the per-project precision the external nodes were for. One list drives both.This is not only a custom-plugin problem.
@nx/gradleemitsexternalNodesinpackages/gradle/src/plugin/nodes.ts, and a dependency version bump in abuild.gradlemarks nothing affected for the same reason.The case behind this
A single-Go-module monorepo, ~690 Go packages as Nx projects.
createNodesemits ago:<module path>external node per module (Module.Sumasdata.hash) and gives each package a named input listing the external modules its imports reach. Hashing is then exact: perturbinggithub.com/stretchr/testifymoves only the tasks of packages whose test imports reach it and leaves the other ~680 alone.nx affected --files go.modselects one project — the single target in the workspace that happens to name{workspaceRoot}/go.modas a file input. So the two halves are in tension. Precise caching meansaffectedselects nothing on a dependency bump; makingaffectedselect means every Go project rehashes on every bump.What would fix it
A plugin-provided locator, invoked once per run like the JS ones, returning project names and/or external-node names, with the same duplicate-emitting contract the existing locators have:
That is enough for the Go case: parse the changed
go.mod, resolve each changed module to itsgo:<mod>external node, return those names, and let the existing reverse walk finish — the shapegetTouchedNpmPackagesalready has, minus the hardcoding.#36825 looks like groundwork rather than an obstacle. It says the lockfile, npm-package and tsconfig locators stay in TypeScript and "cross the boundary as
ThreadsafeFunctioncallbacks invoked once per run, so the locator set is still extensible from JS rather than hardcoded". The ask is to make that seam public, and it may belong inside NXC-4859 (task-levelnx affected) rather than as its own change.A narrower alternative, if a new plugin hook is too much surface: let an input mark itself as affecting selection but not the hash, so a manifest can drive selection without fanning out every hash. That resolves the tension but not the precision — every project would still bump together — so the locator hook is the better shape.
Environment
nx 23.0.1 and 23.2.0; the relevant code is unchanged between them.
All reactions