nx show projects should return packages' pathnames

[dmaklygin]

I was trying to extract affected packages by changes I had made.
Unfortunately, nx graph --affected returns deprecated affectedPackages attribute I should not rely on.
So, I have to run both commands in a sequence:

1. nx show projects --affected
2. nx graph

Unfortunately, nx show projects returns only a simple list of packages names, and it's not enough to determine packages location.
If nx show projects --affected returns extended list of packages, it would be totally enough

Motivation

I was trying to ditribute tests execution between runners, so I had to determine affected packages and gradually distribute tests betweeen runners.

Suggested Implementation

Return a list of objects. Each object should have the following:

1. Package name
2. Package location

[Alonski]

So I did something similar to create a custom gpt (go to project) bash function.
Important for you:

monorepo_root=$(git rev-parse --show-cdup)
project_path=$(nx show project "$1" | jq -r .root)
cd "${monorepo_root}${project_path}"

Full script:

# For usage information see inlined usage conditional
gtp() {
    # Show usage if on errors
    showUsage=0

    # Check that an argument was passed
    # https://unix.stackexchange.com/questions/25945/how-to-check-if-there-are-no-parameters-provided-to-a-command
    if [ $# -eq 0 ]; then
        echo "Argument [projectName] not passed"
        showUsage=1
    fi

    # Check that needed programs exist
    # https://stackoverflow.com/questions/592620/how-can-i-check-if-a-program-exists-from-a-bash-script
    if ! command -v git &> /dev/null; then
        echo "git could not be found, please install it globally"
        showUsage=1
    fi

    if ! command -v nx &> /dev/null; then
        echo "nx could not be found, please install it globally"
        showUsage=1
    fi

    if ! command -v jq &> /dev/null; then
        echo "jq could not be found, please install it globally"
        showUsage=1
    fi

    # Print usage
    if [ $showUsage -eq 1 ]; then
        echo "Usage: $0 [projectName]"
        echo "Example: $0 @crowdstrike/mega-slat-table"
        echo "This function uses NX to CD into a project by the project name"
        echo "Required globally installed commands: git, nx, jq"
        return 1
    fi

    # Get monorepo root using git. Ex: ../../../ (Needs to relative in case of worktree)
    monorepo_root=$(git rev-parse --show-cdup)

    # Get NX project path relative to root. Ex: packages/apps/store
    project_path=$(nx show project "$1" | jq -r .root)

    # Show error if project_path failed
    if [ $? -ne 0 ]; then
        echo "NX failed to show the project, make sure you are in the monorepo and the project name is correct"
        # Run gtp again with no arguments to show usage
        gtp
        return 1
    fi

    # CD into the root+path combo
    cd "${monorepo_root}${project_path}"
}

[dmaklygin]

I was able to make it work via nx show projects and nx graph.
But, having a single command would be more optimal.

[raoulkent]

I was able to make it work via nx show projects and nx graph. But, having a single command would be more optimal.

Is it possible for you to share how? 😄

[dmaklygin]

Nothing special, just using JS code running show projects, then managing the output run the nx graph

[aikrez]

For our tooling we relied on the project path name in the workspace.json which is now removed. Currently I am looking for a way to ditch the old workspace.json and get the project path.
It would be nice that nx show projects had an additional argument to return the full detail of the project like nw show project does.

[MaxKless]

Hi!
For automated tooling, I think having a more programatically readable format makes sense. So I think it's a valid feature request.

It needs a little bit more design, though. There is already a --json flag in nx show projects so any changes to that functionality would be a breaking change which I'd like to avoid.

You could read the list of affected projects in JSON format by running npx nx show projects --json --affected and then extract information about the individual projects from .nx/cache/project-graph.json or by using the programmatic API createProjectGraphAsync from @nx/devkit.

I'd love some more concrete thoughts on how you would design this updated API and what use cases you have that aren't solvable with another solution. Let's continue the conversation in a discussion :)

[dmaklygin]

Having package location is necessary to understand the location of the exact package, therefore collect information about number of files (unit tests or cypress). It's helpful to determine the optimal distribution files between groups, therefore between runners.

[raoulkent]

I was running into the exact same need. Building github workflows with reusable components that I want to run as a matrix for affected projects can't be an unlikely usecase.

And since some workflows expect to be run in the root of a project (such as SAST tooling) it would be incredibly useful to have the possibility to output the paths via nx show or nx affected somehow.

Has this discussion been converted into a feature request?

[MaxKless]

Have you looked into this>

You could read the list of affected projects in JSON format by running npx nx show projects --json --affected and then extract information about the individual projects from .nx/cache/project-graph.json or by using the programmatic API createProjectGraphAsync from @nx/devkit.

I'd love some more specific ideas on how you would design the commands :) It seems pretty easy to write your own script for atm, no?

[raoulkent]

And if you meant to ask how one might design the UX of the command:
nx show --projectPath and nx show --path are both decent options if one was to design for just the .data.root node.

The other option would be instead something nx show --data="root" or somesuch if we're talking about getting a generic part of the data object. But that's much more opaque and requires more knowledge about the graph json structure. So good docs are a must if this approach were selected.

[MaxKless]

yes I meant the UX design :) Thanks for the input, I think --data could work.

If you write a node script instead of bash, you can use the @nx/devkit methods like createProjectGraphAsync. They don't require intimate knowledge of the file system structure Nx uses or the like. Give it a try, should be much simpler and you don't have to iterate over everything.

[AgentEnder]

Yeah, my recommendation for users would be to go into a node script and use createProjectGraphAsync directly. We're never going to be able to provide every option for show folks would want, and implementation wise the node script should be easy and portable.

[raoulkent]

I'm sorry for the late reply, but thank you both @MaxKless and @AgentEnder. That's instrumental input, I'll be using it to make my solution more stable for the long term.
I noticed just yesterday that with an upgraded Nx version the path to the graph changed 😄

[raoulkent]

Hello again, @MaxKless and @AgentEnder. Apologies in advance for the necroposting.

Finally, I have some time to spare to do the refactor of the previously mentioned bash script and have it rewritten as a node script.

Now, going through the docs is rather tough; it is not clear to me from any of the recipes, nor just browsing the API docs for devkit what I need to do to be able to access the affected graph.
createProjectGraphAsync is understandable, but I cannot see any obvious way of getting specifically those that are affected.

From looking at the source, all I can identify is the filterAffected function in affected-project-graph.ts. But from what I can identify, this is only used as part of the CLI functionality and not exported/used in the devkit.

Could you possibly point me in the right direction to continue my investigation?
Thank you!