Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Monorepo tools #148

Closed
adamziel opened this issue Mar 10, 2023 · 3 comments
Closed

Use Monorepo tools #148

adamziel opened this issue Mar 10, 2023 · 3 comments

Comments

@adamziel
Copy link
Collaborator

Description

Setting up the build and dev mode of separate packages in a monorepo is really challenging. The current amalgamation of vite, esbuild, and rollup doesn't really cut it. Every package has a separate build config with its own resolve.alias, external, and viteStaticCopy setup.

Vite doesn't seem to support merging multiple configurations so the monorepo dependencies must be either:

  • Built before the dev script can be started, ideally in a watch mode (which is slow in Vite)
  • Aliased inside the Playground's vite.config.ts and configured again like in their original vite.config.js

Proposed solution

Let's explore these tools created specifically for managing monorepos and use one of them here.

Requirements:

  • Build all packages in the correct order
  • Fast dev mode without prior build, edit any file in the repo and see the outcome right away.
  • Dev mode setup should reflect all parts of the build setup like copied static assets. If it works in the dev mode, it should work in the production mode with near-100% certainty.
  • Dev mode should use the bundled source and types and also support source maps
@eliot-akira
Copy link
Collaborator

eliot-akira commented Mar 12, 2023

Gutenberg uses Lerna for publishing multiple packages from a single repo. Maybe you've already considered it, or it might not suit all requirements of the Playground - but I've seen it used in a number of projects, so it seems to be a fairly standard tool for this purpose.

https://lerna.js.org/

https://github.com/lerna/lerna

adamziel added a commit that referenced this issue Mar 16, 2023
## Description

Sets up the correct build pipeline for all parts of Playground and PHP.wasm. This enables a public release of the [Playground API](#149) npm package!

I've been [struggling](#146) with [this](#70) for [a while](#150) and couldn't understand what's so hard. NX made it apparent – look at this dependency graph!

<img width="1291" alt="CleanShot 2023-03-16 at 23 16 26@2x" src="https://user-images.githubusercontent.com/205419/225764795-7fa8e972-09f8-41ef-aac2-1c96bd100ea0.png">

No wonder it's been almost impossible to set everything up by hand!

## Usage

Start with `yarn install`

### Shortcuts

To start a copy of `wasm.wordpress.net` locally, run `yarn run dev`.
To build it, run `yarn run build`.

### Fully qualified commands

In reality, these `yarn run` commands are just triggering the underlying project's nx `dev` and `build` commands:

```bash
nx dev playground-website
nx build playground-website
```

Here's a few more interesting commands:

```bash
# Build and run PHP.wasm CLI
nx start php-wasm-cli

# Build latest WordPress releases
nx recompile-wordpress:all playground-remote

# Recompile PHP 5.6 - 8.2 releases to .wasm for web
nx recompile-php:all php-wasm-web

# Recompile PHP 5.6 - 8.2 releases to .wasm for node
nx recompile-php:all php-wasm-node

# Builds markdown files for readthedocs site
nx build docs-site

# Builds the Playground Client npm package
nx build playground-client
```

## NX is the tool Playground needed from the outset

It's ridiculous how many problems this solves:

* The build pipeline is explicitly defined and easy to modify
* Tasks only run once their dependencies are ready
* The dev mode works and is fast
* The build works and is fast
* We get CI checks to confirm the entire build process still works (which solves #150)
* Cross-package TypeScript just works
* There are linters and formatters (which solves #14)
* Documentation is correctly generated from the latest built artifacts
* There are nice generators for bootstraping new packages and moving the existing ones around
* There are checks to ensure the private `php-wasm-common` package is not imported by anything else than `php-wasm-web` and `php-wasm-node`

## Next steps

* Add Lerna to harness package publishing
* Additional developer documentation for the nx-based flow

Related to #148 and #147
@adamziel
Copy link
Collaborator Author

adamziel commented Apr 1, 2023

I just tried using Lerna and found an issue. lerna publish bumps a package version, but it does so in the source directory and not the dist directory. The packages are build before calling lerna publish. I have no solution yet.

Edit: Found a solution: lerna publish from-package. Now Lerna is unhappy about uncommitted changes: Working tree has uncommitted changes

Edit 2: I found a solution to that as well – the build needs to happen during the "prepublishOnly" npm hook.

@adamziel adamziel mentioned this issue Apr 1, 2023
adamziel added a commit that referenced this issue Apr 1, 2023
## Description

It's hard to publish all the npm packages living in this repo without a tool. After considering [changesets](https://github.com/changesets/changesets) and [Lerna](https://lerna.js.org/) this commit moves forward the latter since it's maintained by the authors of nx – another tool used in this project.

Solves #148

This commit also bundles wp-playground/client dependencies

This is useful in itself and makes the client package self-contained. In
this instance we need it to avoid producing an invalid list of
dependencies in the final package.json. This package technically depends
on php-wasm/web, but takes great care not to import anything from it
other than TypeScript types that are then released in a rollup file.
@adamziel
Copy link
Collaborator Author

adamziel commented Apr 1, 2023

Soled in #151 and #184. This project now uses nx and Lerna.

@adamziel adamziel closed this as completed Apr 1, 2023
Pookie717 added a commit to Pookie717/wordpress-playground that referenced this issue Oct 1, 2023
## Description

Sets up the correct build pipeline for all parts of Playground and PHP.wasm. This enables a public release of the [Playground API](WordPress/wordpress-playground#149) npm package!

I've been [struggling](WordPress/wordpress-playground#146) with [this](WordPress/wordpress-playground#70) for [a while](WordPress/wordpress-playground#150) and couldn't understand what's so hard. NX made it apparent – look at this dependency graph!

<img width="1291" alt="CleanShot 2023-03-16 at 23 16 26@2x" src="https://user-images.githubusercontent.com/205419/225764795-7fa8e972-09f8-41ef-aac2-1c96bd100ea0.png">

No wonder it's been almost impossible to set everything up by hand!

## Usage

Start with `yarn install`

### Shortcuts

To start a copy of `wasm.wordpress.net` locally, run `yarn run dev`.
To build it, run `yarn run build`.

### Fully qualified commands

In reality, these `yarn run` commands are just triggering the underlying project's nx `dev` and `build` commands:

```bash
nx dev playground-website
nx build playground-website
```

Here's a few more interesting commands:

```bash
# Build and run PHP.wasm CLI
nx start php-wasm-cli

# Build latest WordPress releases
nx recompile-wordpress:all playground-remote

# Recompile PHP 5.6 - 8.2 releases to .wasm for web
nx recompile-php:all php-wasm-web

# Recompile PHP 5.6 - 8.2 releases to .wasm for node
nx recompile-php:all php-wasm-node

# Builds markdown files for readthedocs site
nx build docs-site

# Builds the Playground Client npm package
nx build playground-client
```

## NX is the tool Playground needed from the outset

It's ridiculous how many problems this solves:

* The build pipeline is explicitly defined and easy to modify
* Tasks only run once their dependencies are ready
* The dev mode works and is fast
* The build works and is fast
* We get CI checks to confirm the entire build process still works (which solves #150)
* Cross-package TypeScript just works
* There are linters and formatters (which solves #14)
* Documentation is correctly generated from the latest built artifacts
* There are nice generators for bootstraping new packages and moving the existing ones around
* There are checks to ensure the private `php-wasm-common` package is not imported by anything else than `php-wasm-web` and `php-wasm-node`

## Next steps

* Add Lerna to harness package publishing
* Additional developer documentation for the nx-based flow

Related to WordPress/wordpress-playground#148 and WordPress/wordpress-playground#147
Pookie717 added a commit to Pookie717/wordpress-playground that referenced this issue Oct 1, 2023
## Description

It's hard to publish all the npm packages living in this repo without a tool. After considering [changesets](https://github.com/changesets/changesets) and [Lerna](https://lerna.js.org/) this commit moves forward the latter since it's maintained by the authors of nx – another tool used in this project.

Solves WordPress/wordpress-playground#148

This commit also bundles wp-playground/client dependencies

This is useful in itself and makes the client package self-contained. In
this instance we need it to avoid producing an invalid list of
dependencies in the final package.json. This package technically depends
on php-wasm/web, but takes great care not to import anything from it
other than TypeScript types that are then released in a rollup file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants