package.json per app

[comunitius]

First of all,. just like to say thank you! I love nrwl nx and how well organized the code looks like when using nx! Also, it's awesome you folks are investing in React!

Prerequisites

Please answer the following questions for yourself before submitting an issue.
YOU MAY DELETE THE PREREQUISITES SECTION.

• I am running the latest version
• I checked the documentation and found no answer
• I checked to make sure that this issue has not already been filed
• I'm reporting the issue to the correct repository (not related to Angular, AngularCLI or any dependency)

Expected Behavior

Is there a way we could have a package.json per app? If you are using docker, and would like, let's say, have an angular app running in one container, and the nest api into another container, then both containers will have to run npm install over the all the npm packages even when angular packages are only a subset of all the packages (and viceversa with nest). Additionally, if we do npm install inside the containers, huge packages like cypress will be downloaded, even if they have nothing to do with the api (nest) build. This take its toll especially when running a CI build in travis, circleci, etc (yeah, I know there is cache, but it's not 100% reliable).

Please describe the behavior you are expecting:

Would love to see every app having different package.json. Plus I think it is cleaner. Why having one package,json to rule them all? Seems very tightly coupled.

Current Behavior

What is the current behavior?
There is only one package.json for all apps using nx

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

Context

Please provide any relevant information about your setup:

• version of Nx used: 8.4.13

[the-ult]

A related question/problem we are facing with using a node graphql-api is:

• how to create small node_modules just for production and just for the node application.
  Since otherwise node_modules will include the nx, angular, cypress libraries, etc

To solve this, we are using a separate (specific) package.json in the apps/graphql-server.

However, this results in other problems like:

• nx affected:build does not work for the graphql-server (needs to be started from it's own folder)
• npm install order/dependency between main package.json and apps/grahpql-server/package.json
• etc

So the question really is:

• Can we prevent (needing) multiple package.json files when using node applications?

N.B.

• could not find anything in the docs
• We are using separate docker containers as well

[vsavkin]

Thanks for submitting the issue.

You could create a package.json per project, and for instance use something like yarn workspaces to make npm installs relatively fast. We used to do it for one of our internal projects.

We believe in the single-version policy, where, for instance, all applications using NestJS, use the same version of NestJS. Even though it looks a bit crazy at first, we think it is beneficial. Companies like Google do it. So having one node_modules at the root is very similar to what Google does (with a single third_party).

I'll try to find sometime this week to author a guide on single-version policy, where I'll also cover how to add multiple package.json files if you decide that single-version policy doesn't work for you.

[comunitius]

Nice thanks @vsavkin ! In the meanwhile while you create the documentation, do you have an example of a project with different package.jsons?. I just want to decouple my react app from my nest app for now

[the-ult]

Thanks @vsavkin

We love the ONE package.json to rule them all approach and rather have one package.json instead of multiple.

The only reason why we have a separate package.json for our Express (GraphQL-server) app is that we need a small production node_modules for the graphql-server, without all angular dependencies etc.

Would be great if that could be part of the guide.

(each app lives in it's own docker container)

[kylecordes]

Single version policy:

Fantastic, great path for most projects.

Single list of packages:

This is a little harder to deal with as @the-ult suggested. I would love to see something per-app/library that limits which packages (name only, not repeating or varying the version!) the app/library can depend on. This data could also fuel a mechanism to generate a minimal production-only package.json algorithmically when needed.

(I wonder if this might be possible today by omitting package.json from implicitDependencies in nx.dev...)

[wvanderdeijl]

I totally agree with the previous suggestion of generating a minimalistic package.json for production. However, we would also need a package-lock.json to prevent the runtime production app from fetching different versions than the nx workspace.

We mainly need this to get a production app that is as small as possible since it is deployed as a Google Cloud Function

[comunitius]

I'm starting to see the point about one package.json. I've had to deal with multiple lodash versions in the past and it's a pain. Having said that, just like the previous comments, I need a small package.json in production (with the package-lock.json as well) so I can build and deploy multiple small docker images/containers in production.

[aRusenov]

I'm following this discussion and I understand that single-version policy may be an issue for some. But for minimal node server deployments (i.e. not deploying the entire node_modules) why not simply bundle the node app?

[vsavkin]

Quick update: We are working on a course that will cover things like single-version policy. Stay tuned.

[josesaranda]

I think having one package.json per application is useful if you want to publish your libraries or applications separately in the future. Is there anyone who has solved that problem?

[wangzishi]

I'm following this discussion and I understand that single-version policy may be an issue for some. But for minimal node server deployments (i.e. not deploying the entire node_modules) why not simply bundle the node app?

I think it is mainly because there still tons of issues with bundle apps using nx with webpack, nestjs, typeorm, and docker.

#1518
#1393
#1010
#803
#633

Each of them is great. But using those significant parts together is terrible.

[mderoche]

Reading through this because I have a use-case with several "sub-applications" that are lazily loaded into a single core Angular application, and they need to be developed by independent teams. I haven't seen that particular wrench thrown in this thread yet.

While the single package.json strategy works great for a single team, I am struggling with how these large companies handle the seemingly incredible cooperation costs of having multiple teams operate with a single package.json.

I am looking for an architecture where we have a core Angular application that is comprised of several sub-applications (which is now facilitated by Nx, and previously by the vanilla Angular CLI.) I wanted each sub-application to have a private package.json to define dependencies specific to that project, while keeping globally shared dependencies (such as Angular itself) in the root-level package.json.

This dramatically cuts the cooperation costs between teams; if Project A needs this-package@1.0.0, and Project B needs this-package@2.0.0, then Project B doesn't need to petition and wait for Project A to upgrade their application to support this-package@2.0.0.

Of course, we still have the issues of "Okay, everyone upgrade to Angular N+1 on mm/dd/yyyy!", but I think having private package.jsons can remove a vast majority of related versioning issues.

This all works fine for JIT compilation. Issues arise when you try and build it with AoT.

Is there a recommendation on how to handle this type of situation, with multiple teams working on individual sub-projects under the same global project?

[demisx]

I like the idea of having a single master package.json file for enforcing versions across the entire repo, but I’m still struggling to understand how one creates a docker image for each app with the package.json being a subset from the master file? How do you derive an appropriate version of package.json per each app? 🤷🏼‍♂️ Obviously, I don’t need angular packages in my NestJs container and vice versa.

[kraiz]

I use generate-package-json-webpack-plugin to solve this problem.
But I would like to generate a yarn/npm lockfile instead to tighten the dependencies but didnt find a solution for this yet.

[demisx]

@kraiz Thank you for the tip. Do you know if this plugin can be used for Angular apps as well to weed out the backend (NestJs) dependencies? In other words, is it smart enough to parse Angular source code and build a list npm packages that app needs?

[kraiz]

You don't need it for angular because it's webpack configuration is already making sure you only get the stuff you need into your bundle files without the need for an external runtime dependency folder (node_modules).

[demisx]

@kraiz Got it. That makes sense.

[bennymeg]

I use generate-package-json-webpack-plugin to solve this problem.
But I would like to generate a yarn/npm lockfile instead to tighten the dependencies but didnt find a solution for this yet.

@kraiz ,
Is this what you are looking for?

[kraiz]

Ahh, didn't know that the lockfile can be a superset. Thx, will try that!

[MaximeBernard]

How does Google manage to update all (i can imagine they have hundred/thousands) apps at once?

From my experience, you always get:

• one app that requires too much effort/cost/time to be updated (i.e. that won't happen)
• one app that needs to do something before that (i.e. delaying the update for months/years)
• one app that requires to stay at some package version (angular users are often locked in older versions of TS, or some webpack users often rely on some webpack plugin which has not been migrated yet (or never will))

I'm pretty sure I didn't cover all the scenarios but they all lead to late migration / no migration at all.

Even though I agree it's important to encourage every app/team to stay up to date with the latest releases and use the same version, it becomes more and more complex over time until it's not even possible anymore.

One strategy here for nx could be to offer both options:

• 1 package.json (current way)
• multiple package.json (yarn workspaces way)

That being said, it's probably a lot of work to maintain both possibilities and much more work for many features.

On the other hand, keeping the yarn workspaces way and doing some checks when running nx could offer both worlds features with a limited maintenance overhead.

Whatever you decide, keep up the good work 👍

[mderoche]

@MaximeBernard Are you using Angular with multiple package.jsons? If so, how do you manage to get AoT compilation to work with multiple package.jsons?

[boxmein]

@meeroslav At this point, it'll be faster for us to harmonize the Angular versions 😞

It sucks that the documentation says Yarn workspaces (1) /Lerna-style monorepos (2) are fully supported whereas in reality, using any type of workspaces (with separate package.jsons) with Nx creates issues like this one.

Would the alternative be to give Nx its own little box as a separate monorepo package, separate from the actual projects it compiles/manages?

[ianldgs]

@boxmein what do you mean by:

creates issues like this one

?

I have just set up NPM workspaces (not yarn or lerna or pnpm workspaces) in our NX monorepo, and so far, no big issues.
We use React BTW, not angular.

The only 2 things that didn't work for us were:

1. Eslint plugin import's "no-extraneous-dependencies" rule not viewing the other package.jsons;
   • Managed to get around it with a .js eslintrc reading this info from process.argv and adding as packageDir to the config
2. NX unable to run executors or generators of custom plugins in the workspace;
   • Workaround was to exclude them from the workspaces glob patterns: libs/* -> libs/!(nx-plugin-1|nx-plugin-2)/**

[boxmein]

@ianldgs

There is one major pitfall to using a proper Yarn workspace with Angular: the Angular Compiler

• @nrwl/angular implicitly depends on @angular/compiler, which means you have to have @angular/compiler resolvable in project root
  • Changing process.cwd before require("@angular/compiler") is called - resolves the issue, because then the node resolution algorithm would discover the right angular compiler every time
• Because of the above, only one version of Angular compiler can be used across all library code
• Because of the above, there is no feasible way to use different angular versions for different sub applications

So what we were trying to do was the following:

• Use Nx to create 2 Angular applications, and 1 Angular library, eg. "app1", "app2" and "lib1"
• Use Yarn Workspaces without path-mappings to natively use workspace package references
  • You do it by setting dependencies > "@yourorg/lib1": "workspace:*" in app1 and app2
• Use any means possible to correctly run the Angular Compiler on lib1/components/*

Now there are two ways to run Angular Compiler on lib1/:

• Use the Nx Angular Library compiler @nrwl/angular:ng-packagr-lite to compile lib1 into an Angular Library, and refer to it in both app1 and app2

  • ❗@nrwl/angular has implicit dep on ng-packagr and @angular/compiler, which means that the project root must choose only 1 version of @angular/compiler

• Override the configuration for @angular-devkit/build-angular:browser to also run the Angular Compiler on files in the node_modules/@yourorg/lib1/ folder

  • We didn't want to use path mappings because we had a fully functioning yarn workspace, so yarn was taking care of putting everything in the right place for the node module resolver
  • This wasn't really possible because: (1) TypeScript Compiler unraveled the symlink and used the node module resolution algorithm to detect the version of Angular starting from libraries/lib1 - which is wrong, because workspace root did not have any version of Angular
  • And (2) because @angular-devkit/build-angular:browser REALLY does not want to run stuff on files inside node_modules

This meant we did not have a way to run the Angular compiler on lib1 so that it would use @angular/compiler@11 in app1 and @angular/compiler@12 in app2.

As well as that, @nrwl/angular had an implicit (non-declared) peer dependency on ng-packagr so Yarn could not help / detect the issue either.

[boxmein]

@ianldgs

At this time, we chose to stop leveraging Yarn workspaces to properly manage per-library dependencies, merged all package.jsons together into a mega-package.json, downgraded our Angular 12 code to Angular 11, and used "path aliases" to create individual "libraries" like the official Nx doc instructs you to.

[boxmein]

@ianldgs That's why I said above that Yarn workspace support in Nx is not really there.. because in the Angular case, you run into issues when using yarn workspaces instead of the "true Nx way" of having a single mega-package.json. You're essentially limited because the Nx builders have to be in the root project, and there's no real way to make it utilize the version of @angular/compiler defined in a sub-package

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[prodkt]

Can someone link the official NX response and suggested route to do this? I need NX to npm install using different node versions per package. I'm not finding any clear answer and hoping from what the official answer or suggestion was here will be highly beneficial.

[MaxKless]

https://nx.dev/getting-started/package-based-repo-tutorial

[meeroslav]

@boxmein, your use case is too specific, so Nx will never support it in the integrated approach with the @nrwl/* plugins.
But you can use Nx in isolation with whatever workspaces and config system you have set up. The link that @MaxKless sent explains how do this.

Can someone link the official NX response and suggested route to do this? I need NX to npm install using different node versions per package. I'm not finding any clear answer and hoping from what the official answer or suggestion was here will be highly beneficial.

Using different packages per project in monorepo is supported, but how would using different node versions work?
Which version would you use to bundle several projects that use different versions? The suggestion (and the only solution I can think of) would be to align those packages to operate on the same node version at least.

[vicb]

Just a short update for all involved parties.

We are working on a solution that would generate pruned lock file (for all supported package managers) that should solve your issue. Follow our Twitter account or release notes to be notified once this lands.

Thank you for your patience.

@meeroslav I see several "lock file pruning" mentions in the release notes, has this issue been resolved? If not do you have any ETA to share and is this issue the best place to monitor to be informed of the progress?

Thanks for Nx, it's such a great tool!

[meeroslav]

Hey @vicb, the lock file pruning has been released (since 15.3.1 functionality has been out).

The main effort was linked to #9761.

Please note that there are still some minor issues being polished out for Npm and Pnpm so I would wait a bit more before using it for anything else than Yarn.

[vicb]

Thanks a lot @meeroslav

I've found the doc for that feature.

Using yarn is an option for my docker images (it only requires installing one more dependency). It is however not an option for deploying to Google App Engine which only supports npm. It looks like the support is good enough for my need as is and knowing that it will get better is great.

Edit: yarn is actually supported on GAE

👍

[boxmein]

@boxmein, your use case is too specific, so Nx will never support it in the integrated approach with the @nrwl/* plugins.

@meeroslav I understand where you're coming from - however it's not a matter of specific use-case... it's more about the fact that the core feature of a Yarn workspace is that NPM package versions between two projects are not related to each other. They are well-defined and specified in package.json.

I had a bug in my repo that happened because Nx requires all builders (e.g. @nrwl/angular) to be defined in the root project. This means that any dependencies those builders pull in, are transitively included to the root project.

In my case, with this workspace structure:

root/
    nx.json
    workspace.json
    yarn.lock
    package.json
      "dependencies": { "@nrwl/angular": ... }
      "workspaces": ["package1", "package2"]
   package1/
      package.json
           "dependencies": {"@angular/compiler": "11" }
   package2/
      package.json
           "dependencies": {"@angular/compiler": "13" }

The package @nrwl/angular would never run, because it could not resolve @angular/compiler, because it is defined in a subproject.

It doesn't matter which node version it is, just the fact that builders have to be defined in the root project, will cause issues in any Yarn workspace.

A little extra: the root cause will actually be hidden from you, because Yarn will randomly choose one @angular/compiler to hoist to workspace root, which means @nrwl/angular will not immediately error, and you will get interesting unique compile failures.

[meeroslav]

@boxmein I see what you mean.

The problem, in your case, is different from that we require the builders to be in the root (Yarn, anyways, installs all the workspace dependencies recursively in the root). The issue is that we run the builder from the root pointing to your project, which results in always using the same version regardless of what your package requires.

This is an interesting problem we need to take a better look into.

Can you please open another issue explaining this?

cc @Coly010 @leosvelperez @FrozenPandaz

[boxmein]

@boxmein I see what you mean.

The problem, in your case, is different from that we require the builders to be in the root

Yes, that's what we were struggling with and it led to us essentially creating a mega-package.json that contains 3 microfrontends' dependencies. We would still eventually like to decouple them so that they can have smaller, more focused package.json files. This may also cause differing versions of @nrwl/angular to be in the tree in different locations (as yarn is free to reorganize the layout as long as the node resolution algorithm finds the right version in the right place)

(Yarn, anyways, installs all the workspace dependencies recursively in the root).

You're correct in that, unless a workspace requires a different version. Then the node resolution algorithm kicks in, which depends on the cwd and walks up the tree.

The issue is that we run the builder from the root pointing to your project, which results in always using the same version regardless of what your package requires.

This is an interesting problem we need to take a better look into.

Can you please open another issue explaining this?

Sure thing

[meeroslav]

Closing this issue as it has been resolved the lock file parsing, package json generation and lock file pruning.

[LuisCusihuaman]

Closing this issue as it has been resolved the lock file parsing, package json generation and lock file pruning.

@meeroslav could you add a link to the documentation of that? 🙏

[meeroslav]

@LuisCusihuaman the documentation is slightly outdated but most of the information should be there:
https://nx.dev/recipes/ci/ci-deployment#prepare-applications-for-deployment-via-ci

[maksym-yaremenko-simpligov]

Closing this issue as it has been resolved the lock file parsing, package json generation and lock file pruning.

is it work for FE apps as well?

[meeroslav]

It's especially for apps. Published libs don't need lock files and they usually have package.json explicitly specified.

[ianldgs]

@meeroslav would you recommend having an explicit package.json on libs?
I'm currently relying on the auto-generated package.json for them.

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.