Design Nx Plugin Project Crystal

[jogelin]

Hey hey,

I am opening this discussion because after exploring the code of Nx plugins, some reflections came to my mind about the design of the Project Crystal.

For people who don't know what Project Crystal is, I encourage you to have a look at:

• What if Nx Plugins Were More Like VSCode Extensions from @juristr
• 💎 Discovering Nx Project Crystal’s Magic

Current situation in a nutshell

Today, we can consider that if you want to modify the Project Graph, you can do it via Nx plugin by using:

• createNodes
• createDependencies

If we focus on createNodes, we can see that it serves two responsibilities:

• Identifying Projects
• Adding Targets

For example, we can see plugins that are:

• Adding targets: like @nx/jest/plugin which only adds a target test if you have the jest.config.* file
• Identifying projects: like nx/core/project-json that will create an Nx Project if you have a project.json file

We also know that the order of plugins matters because a config generated by one plugin can be overridden by another plugin.

A simplified example of the plugins execution today looks like:

1. @nx/jest/plugin (from nx.json)
2. nx/core/target-defaults (default)
3. nx/core/project-json (default)

But my explanation is wrong!

In fact, it is not possible for a plugin like @nx/jest/plugin to add the test target if it is not linked to an Nx Project. So the @nx/jest/plugin is also responsible for Identifying projects!

This is why many plugins are first testing if the path is related to a project by checking the existence of the project.json.

Reflections

Why is the definition of an Nx Project still linked to a project.json file?

My first reaction was to use the function readProjectsConfigurationFromProjectGraph instead of checking the existence of the project.json. Or to provide the list of projects in the context options of my plugin.

In fact, maybe a previous plugin Identified Projects in a different way. Based on project structure for example.

But it cannot work because all plugins are run in parallel and then the results are merged in order. The execution is already done.

Why don't we split responsibilities?

For example, we could have a plugin nx/core/project-json that is only responsible for Identifying Project and another plugin nx/core/merge-project-json-targets that is only responsible for adding targets.

We could maybe also have three entry points in our plugin called in order:

1. createNodes
2. createDependencies
3. createTargets

But I don't think it's a breaking change and going back to the Nx Plugin v1 in a way :)

Why the plugin functions responsible of target generations are not exposed in the @nx/devkit?

In that way, we could create a new plugin like @org/jest and just extend the @nx/devkit

What do you think?

[MaxKless]

I agree that an Nx project shouldn't be hard-linked to the existence of a project.json file. Everything should also work with a package.json (internal plugins) or whatever really for non-js plugins.

For splitting the createNodes into two, I see the point conceptually. However

• it would be yet another version of our plugin api, we are reluctant to put out a v3 so soon.
• it's not necessary in order to unlink project.json files and Nx projects
• It would be pretty messy in our codebase and a big refactor

So I personally don't think that's going to happen

[jogelin]

Thanks for your answer 🙏

I really get that the split of the plugin will not happen. I don't want a v3 either 😉

Did you talk about exposing the core functions of the plugins in a kind of @nx/jest/dev-kit?

For example, having access to buildJestTargets or buildViteTargets would make it easier to implement custom plugins that extend the default ones and then allow skipping the filters.

I'm eager to remove all my project.json because my monorepo follows conventions.

[MaxKless]

hmmm... shouldn't it be possible to import directly from the plugin? Exposing things via the devkit is always opening up more API surface for us to maintain so it's sometimes a tough decision. But if you maintain a plugin that will be linked closely to the official plugin anyways, it should work, no?
Even in Nx Console we just import things directly from nx even though they aren't in devkit. Have you tried that?

[jogelin]

I agree that it is not a good idea to expose and maintain too much public API 👍

I had the same reflex at the beginning by having a deep import to the function. However, it is not exported right?

Only the createNodes is exported but the createNodes contains some filters that I don't want to have.

Or maybe I misunderstand 😑

[jogelin]

I described my current solution in that article:

🩹 Nx Crystal Plugin Picking the Essentials

I hope I will deprecate that article soon @MaxKless :D