feat(core): add monorepo generator to convert from standalone projects #18245

jaysoo · 2023-07-21T17:31:22Z

This PR adds a new @nx/workspace:monorepo generator that converts standalone projects to a monorepo.

For example:

npx create-nx-workspace react-app
cd react-app
nx g monorepo

Then users will see:

.
├── apps
│   ├── e2e
│   │   ├── cypress.config.ts
│   │   ├── project.json
│   │   ├── src
│   │   │   ├── ...
│   │   │   └── support
│   │   └── tsconfig.json
│   └── react2
│       ├── index.html
│       ├── project.json
│       ├── public
│       │   └── favicon.ico
│       ├── src
│       │   ├── app
│       │   ├── assets
│       │   ├── main.tsx
│       │   └── styles.css
│       ├── tsconfig.app.json
│       ├── tsconfig.json
│       ├── tsconfig.spec.json
│       └── vite.config.ts
├── nx.json
├── README.md
├── package.json
└── tsconfig.base.json

generator-monorepo.mp4

Test Cases

Convert a clean standalone project (i.e. one from create-nx-workspace without any changes). Test with React, Next.js, Angular, Node servers, TS.
Convert a standalone project with libs already added.
Convert a standalone project with other apps added.
Convert a standalone project with some apps and libs already in apps and libs directories.
Convert a standalone TS project 'my-lib', but packages/my-lib or libs/my-lib already exists.
Convert a monorepo (idempotency)

Notes

It's hard to move all root project files to their new location under appssince there are workspace files such as nx.json, or even custom files that users create that may or may not belong to the root project. The safest way to handle it is to whitelist the root files we do move, and leave the rest for users to handle.
The existing e2e project is problematic since it is not easy to replace "e2e" string in configuration files with apps/e2e without messing options like "target": "e2e" that should be left untouched. We may need to handle these cases specifically.

vercel · 2023-07-21T17:31:25Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
nx-dev	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 28, 2023 4:59pm

nx-cloud · 2023-07-21T17:32:43Z

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 13f0b2c. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

✅ Successfully ran 8 targets

Sent with 💌 from NxCloud.

JamesHenry

Copying over my slack reply just so it doesn't get lost:

Nice! We have a precedent for conversion generators starting with convert to make it clear it’s mutating existing stuff as its focus, so maybe convert-to-monorepo is preferable?

As Victor notes it’s going to be run very rarely so arguably clarity can be weighted above brevity because it’s not an ongoing source of friction to type?

I would worry nx generate @nx/workspace:monorepo could easily be confused with create-nx-workspace

FrozenPandaz · 2023-07-25T21:01:58Z

packages/workspace/src/generators/convert-to-monorepo/convert-to-monorepo.ts

+  const nxJson = readNxJson(tree);
+  nxJson.workspaceLayout = {
+    // normalize paths without trailing slash
+    appsDir: joinPathFragments(options.appsDir),
+    libsDir: joinPathFragments(options.libsDir),
+  };
+
+  updateNxJson(tree, nxJson);


Just add the directories and it will have the same effect. Do not write to nx.json.

This can move after the move.

If we move an app, we also have to add the libs dir that the user wants

If we move a lib, we don't actually have to do anything because moving the project will create the libs dir

FrozenPandaz · 2023-07-25T21:08:10Z

packages/workspace/src/generators/convert-to-monorepo/schema.json

+    "appsDir": {
+      "type": "string",
+      "description": "The directory where apps are placed.",
+      "default": "apps",
+      "x-prompt": "Where do you want to place your apps?"
+    },


Let's always pick apps if the root project is an app. Do not prompt.

FrozenPandaz · 2023-07-25T21:08:28Z

packages/workspace/src/generators/convert-to-monorepo/schema.json

+    "libsDir": {
+      "type": "string",
+      "alias": "packagesDir",
+      "description": "The directory where libs/packages are placed.",
+      "default": "libs",
+      "x-prompt": "Where do you want to place your libs/packages?"
+    }


Constrain this to ['libs', 'packages']

Turns out, we're restricted to apps-libs vs packages anyway, so removing all the prompts. i.e. you cannot have apps-packages.

FrozenPandaz · 2023-07-25T21:12:06Z

packages/workspace/src/generators/convert-to-monorepo/convert-to-monorepo.ts

+    maybeExtractEslintConfigIfRootProject(tree, project);
+    await moveGenerator(tree, {
+      projectName: project.name,
+      destination: project.name,


Pass destinationRelativeToRoot so that the destination is taken as is.

This turns into join(options.appsDir | options.libsDir, project.name)

FrozenPandaz · 2023-07-25T21:13:29Z

packages/workspace/src/generators/convert-to-monorepo/convert-to-monorepo.ts

+  mayebExtractTsConfigBase(tree);
+  await maybeExtractJestConfigBase(tree);


All of the extract, should go into move. Regardless of if we are moving from a monorepo or not, if a user moves the root app, we should do this stuff.

This could be refactored more later.

FrozenPandaz · 2023-07-25T21:23:16Z

packages/workspace/src/generators/move/lib/move-project-files.ts

    const relativeFromOriginalSource = relative(project.root, file);
    const newFilePath = join(
      schema.relativeToRootDestination,
      relativeFromOriginalSource
    );
+
+    // Prevents moving the same file infinitely for root projects
+    if (seen.has(file)) return;


This isn't needed anymore after the whitelist.

FrozenPandaz · 2023-07-25T21:27:25Z

packages/workspace/src/generators/move/lib/update-project-root-files.ts

-    .relative(path.join(workspaceRoot, project.root), workspaceRoot)
-    .split(path.sep)
-    .join('/');
+  const oldRelativeRoot =


This can be reverted.

FrozenPandaz · 2023-07-25T21:27:46Z

packages/workspace/src/generators/move/lib/update-project-root-files.ts

-    if (file === '.eslintrc.json') {
-      continue;
-    }
+    if (!shouldUpdate(file)) continue;


This should go back to how it was.

FrozenPandaz · 2023-07-25T21:29:19Z

packages/workspace/src/generators/move/move.spec.ts

+    expect(tree.exists('package.json')).toBeTruthy();
+    expect(tree.exists('README.md')).toBeTruthy();
+    expect(tree.exists('.gitignore')).toBeTruthy();
+    expect(tree.exists('other-lib/index.ts')).toBeTruthy();


We should test that root config files are made

FrozenPandaz · 2023-07-25T21:29:57Z

packages/workspace/src/generators/move/move.ts

  formatFiles,
+  ProjectGraph,


Unneeded imports

barbados-clemens

just a question

packages/jest/src/generators/init/init.ts

FrozenPandaz · 2023-07-31T14:16:28Z

docs/generated/packages-metadata.json

@@ -2842,6 +2842,15 @@
        "path": "workspace/generators/remove",
        "type": "generator"
      },
+      {
+        "description": "Convert a Nx project to a monorepo.",


@ndcunningham Can you make this change in a follow up PR please?

github-actions · 2023-08-06T00:07:46Z

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

jaysoo force-pushed the feat/monorepo-generator branch from d712335 to cb0bf1e Compare July 21, 2023 17:47

vercel bot deployed to Preview July 21, 2023 17:50 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from cb0bf1e to d111be9 Compare July 21, 2023 17:52

vercel bot deployed to Preview July 21, 2023 17:55 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from d111be9 to fb0ef75 Compare July 21, 2023 18:09

vercel bot deployed to Preview July 21, 2023 18:12 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from fb0ef75 to 8454d70 Compare July 21, 2023 19:45

vercel bot deployed to Preview July 21, 2023 19:48 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 8454d70 to de33ac7 Compare July 21, 2023 20:06

jaysoo marked this pull request as ready for review July 21, 2023 20:07

jaysoo requested review from FrozenPandaz, AgentEnder and a team as code owners July 21, 2023 20:07

jaysoo requested a review from bcabanes July 21, 2023 20:07

vercel bot deployed to Preview July 21, 2023 20:09 View deployment

JamesHenry requested changes Jul 24, 2023

View reviewed changes

jaysoo requested a review from JamesHenry July 24, 2023 14:16

jaysoo force-pushed the feat/monorepo-generator branch from de33ac7 to d3411ee Compare July 24, 2023 14:21

vercel bot deployed to Preview July 24, 2023 14:23 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from d3411ee to 5fcfd3e Compare July 24, 2023 14:55

vercel bot deployed to Preview July 24, 2023 14:58 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 5fcfd3e to 80ee98a Compare July 24, 2023 16:57

vercel bot deployed to Preview July 24, 2023 17:00 View deployment

jaysoo force-pushed the feat/monorepo-generator branch 2 times, most recently from 9659577 to 7eb9b76 Compare July 24, 2023 17:56

vercel bot deployed to Preview July 24, 2023 18:00 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 130eeef to f0ce95d Compare July 25, 2023 15:52

vercel bot deployed to Preview July 25, 2023 15:54 View deployment

jaysoo requested a review from juristr July 25, 2023 15:54

vercel bot deployed to Preview July 25, 2023 16:01 View deployment

FrozenPandaz requested changes Jul 25, 2023

View reviewed changes

jaysoo force-pushed the feat/monorepo-generator branch from f0ce95d to 3841cab Compare July 26, 2023 13:09

jaysoo requested a review from a team as a code owner July 26, 2023 13:09

vercel bot deployed to Preview July 26, 2023 13:12 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 3841cab to 006c1ea Compare July 26, 2023 13:25

jaysoo requested a review from FrozenPandaz July 26, 2023 13:25

vercel bot deployed to Preview July 26, 2023 13:27 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 006c1ea to 6a25f13 Compare July 26, 2023 13:47

vercel bot deployed to Preview July 26, 2023 13:50 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 6a25f13 to 1541dd5 Compare July 26, 2023 14:26

vercel bot deployed to Preview July 26, 2023 14:30 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from 1541dd5 to b2f3806 Compare July 26, 2023 14:35

vercel bot deployed to Preview July 26, 2023 14:38 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from b2f3806 to d49936f Compare July 27, 2023 00:44

vercel bot deployed to Preview July 27, 2023 00:47 View deployment

jaysoo force-pushed the feat/monorepo-generator branch from d49936f to 3f57363 Compare July 28, 2023 16:18

vercel bot deployed to Preview July 28, 2023 16:24 View deployment

feat(core): add monorepo generator to convert from standalone projects

13f0b2c

jaysoo force-pushed the feat/monorepo-generator branch from 3f57363 to 13f0b2c Compare July 28, 2023 16:54

vercel bot deployed to Preview July 28, 2023 16:59 View deployment

barbados-clemens approved these changes Jul 28, 2023

View reviewed changes

packages/jest/src/generators/init/init.ts Show resolved Hide resolved

FrozenPandaz approved these changes Jul 31, 2023

View reviewed changes

FrozenPandaz merged commit dcefa4a into nrwl:master Jul 31, 2023
15 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Aug 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(core): add monorepo generator to convert from standalone projects #18245

feat(core): add monorepo generator to convert from standalone projects #18245

jaysoo commented Jul 21, 2023 •

edited

vercel bot commented Jul 21, 2023 •

edited

nx-cloud bot commented Jul 21, 2023 •

edited

JamesHenry left a comment

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

jaysoo Jul 26, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

FrozenPandaz Jul 25, 2023

barbados-clemens left a comment

FrozenPandaz Jul 31, 2023

github-actions bot commented Aug 6, 2023

		mayebExtractTsConfigBase(tree);
		await maybeExtractJestConfigBase(tree);

feat(core): add monorepo generator to convert from standalone projects #18245

feat(core): add monorepo generator to convert from standalone projects #18245

Conversation

jaysoo commented Jul 21, 2023 • edited

Test Cases

Notes

vercel bot commented Jul 21, 2023 • edited

nx-cloud bot commented Jul 21, 2023 • edited

☁️ Nx Cloud Report

JamesHenry left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

barbados-clemens left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 6, 2023

jaysoo commented Jul 21, 2023 •

edited

vercel bot commented Jul 21, 2023 •

edited

nx-cloud bot commented Jul 21, 2023 •

edited