-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run Nx dependency graph backwards (leaves first) #9322
Comments
I need this features as well. my
and each i.e.
The above allows me to build and deploy my AWS cdk stacks... but I want to add a |
I think this could be neatly solved by adding support for
|
@HadiModarres @AgentEnder I can try implementing this feature if it's okay for you 👍 #Hacktoberfest |
we've found that reversing the order of |
@barakcoh Sure but how did you achieve that ? |
@ThomasAribart like so function getAffectedDeployStackApps() {
return new Promise<string[]>((resolve) => {
exec(
'yarn nx affected:apps base=HEAD~1 | grep stack | xargs',
(_, stdout, __) => {
resolve(
stdout
.trim()
.replace('- ', ' - ')
.split(' - ')
.filter(Boolean)
.reverse()
);
}
);
});
} |
@ThomasAribart apparently, the solution we had in place didn't work reliably. we replaced it with a proper topological sort: async function destroyStacks() {
try {
const adjList = new Map<string, string[]>();
const affected = JSON.parse(
(
await $`yarn -s nx print-affected`
).toString()
);
const dependencies = Object.values(
affected['projectGraph']['dependencies']
) as Edge[][];
chain(dependencies)
.flatten()
.forEach(({ source, target }) => {
console.log(`Adding dependency: ${source} -> ${target}`);
adjList.set(source, (adjList.get(source) ?? []).concat(target));
})
.value();
const sortedStacksWithDependencies = findTopologicalSort(adjList);
console.log(
`Destroying stacks with dependencies in order`,
sortedStacksWithDependencies.join(' > ')
);
await $`yarn nx run-many --target=destroy --projects=${sortedStacksWithDependencies.join(
','
)} --args="--infraEnv=${INFRA_ENV} --appsEnv=${APPS_ENV}"`;
} where |
Hi team @jeffbcross @vsavkin @FrozenPandaz @jaysoo, would it be possible to push this feature? 🙏 It's quite important for use-cases when you are using Nx to drive your deployment process which I feel is quite common. 🙂 |
Hey all In theory, it is reasonable to reverse the task graph. I would love to hear more use cases for this to make sure the solution solves the problem.
Is this something that happens often? What's a more concrete example of when you would do this?
I could see this design working but would have to think about it some more. 🤔
I love the enthusiasm but let's gather more info before implementing this. |
Hi @FrozenPandaz 🙂 sounds reasonable. 👍 Let me describe our use-case at @purple-technology: What we have now
What are we migrating to at the moment and why is this issue for us
Nx has this beautiful feature of dependency graph and putting there a flag to make the execution in a reverse order sounds like a reasonable feature. Let me know your any other thoughts on how this could be done in order to meet our needs. 🤔 |
Or maybe just some Nx plugin could be sufficient for this 🤔 I saw in the docs it's possible to manipulate the dependency graph from inside a plugin, which could do the job. Let me know what do you think. |
@FrozenPandaz My need is the same as @FilipPyrek I develop a Serverless application on AWS, with microservices as Cloudformation stacks. We need to remove them from dev accounts every week (this is a policy of the company I work for, to avoid drift). We cannot create a nx script because of the |
This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. |
I did the same thing as @FilipPyrek and @ThomasAribart, but I also think this whole approach of using implicit dependencies for correct deployment order is suboptimal as it affects the |
This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. |
Hey, has any progress been made on this or any further solutions. My usecase is exactly the same as @ThomasAribart and @FilipPyrek, literally to the letter. Thanks |
Sounds reasonable and would solve a use-case for me too. We have to publish packages, some of which depend on others, if there was say a |
I second this for the exact same use-case (AWS stacks) |
I see, that all seems reasonable. This is kind of complicated but the core team does not have bandwidth right now to work on this. I believe reversing the task graph would yield the desired outcome? It would be a quite a bit of work but if someone from the community would like to try implementing this, I believe somewhere around here would be where we would reverse the graph. I'm not sure if it's that simple though. I'm not sure if we could handle "dependent" task inputs for these kinds of tasks 🤔 It seems like for these scenarios, caching is not required though. |
Description
I need to run certain command for my packages in a dependency graph, but backwards. So the tree of dependencies will have commands run around the edges at the leaves first the moving to the root.
Motivation
To successfully remove the deployed stacks on AWS we need to destroy stacks that aren't depended on first. So if I have stacks that depend on each other like: stack1 -> stack2 -> [stack3, stack4] I need stacks 3, 4 to be destroyed first then stack2 then stack1. This is the reverse of deployment in which we want stack1 to be deployed first and so on.
The text was updated successfully, but these errors were encountered: