A deploy command

[zkochan]

• I'd be willing to implement this feature (contributing guide)

Describe the user story

I want to deploy a project from a workspace that dependends on other projects from the workspace

Describe the solution you'd like

pnpm should copy any dependencies of the project to the project is being deployed and install its node_modules

Related issues:

• Best practices for using PNPM with docker monorepo #3114
• pnpm monorepo docker support #1637

It is probably best to implement it first as a separate tool. The easiest solution might be to copy any dependents of the project to some subfolder inside the deployed project, create a new workspace and run install.

For example. There is a workspace where b and c are dependencies of a.

<workspace root>
+ a
+ b
+ c
+ d
+ e
+ pnpm-workspace.yaml

The deploy command will copy b and c to a/dependencies, create a pnpm-workspace.yaml inside a and run install:

<workspace root>
+ a
  + dependencies
     + b
     + c
  + node_modules
  + src
  + pnpm-workspace.yaml

Alternative solution

We may use the injected dependencies feature to inject all the workspace dependencies of the project. Run install only for the selected project and set the virtual store directory to be inside the project.

[zkochan]

OK, so I think this workaround works now.

Let's say the package to be deployed is at package/foo.

Create the next .pnpmfile.cjs in the root of the workspace:

module.exports = {
  hooks: {
    readPackage (pkg) {
      pkg.dependenciesMeta = pkg.dependenciesMeta || {}
      for (const [depName, depVersion] of Object.entries(pkg.dependencies)) {
        if (depVersion.startsWith('workspace:')) {
          pkg.dependenciesMeta[depName] = {
            injected: true
          }
        }
      }
      return pkg
    }
  }
}

Run:

pnpm i -F ./packages/foo --virtual-store-dir packages/foo/node_modules/.pnpm

Your project will have all the needed dependencies in its node_modules folder.

[chengcyber]

How about specify a deploy folder and copy everything needed to this folder. Like

root
 |- a (depends on b)
 |- b
 |- c

After run deploy package a

root
 |- a
 |- b
 |- c
 |- deploy_folder
     |- node_module/.pnpm <-- store
     |- pnpm-workspace.yaml <-- configuration
     |- a
     |- b

Under deploy_folder, just run pnpm install, cd a && pnpm run <command> works fine now.

[zkochan]

In order for this to work, I need to make this breaking change in pnpm v7: #4469

[zkochan]

How about specify a deploy folder and copy everything needed to this folder. Like

it would need additional development. My solution that I described in the comment above (#4378 (comment)) doesn't require additional development. It already works by using the injected deps features. So the projects are copied automatically by pnpm into the node_modules of the deployed app.

[kilbiller]

OK, so I think this workaround works now.

Let's say the package to be deployed is at package/foo.

Create the next .pnpmfile.cjs in the root of the workspace:

module.exports = {
  hooks: {
    readPackage (pkg) {
      pkg.dependenciesMeta = pkg.dependenciesMeta || {}
      for (const [depName, depVersion] of Object.entries(pkg.dependencies)) {
        if (depVersion.startsWith('workspace:')) {
          pkg.dependenciesMeta[depName] = {
            injected: true
          }
        }
      }
      return pkg
    }
  }
}

Run:

pnpm i -F ./packages/foo --virtual-store-dir packages/foo/node_modules/.pnpm

Your project will have all the needed dependencies in its node_modules folder.

So I tried this and it doesn't seem to work. I'm using pnpm 7.0.1.
It works if I add the entry dependenciesMeta in package.json manually but not when using this hook.
I also noticed a bug, if I remove dependenciesMeta and run pnpm install, the lockfile keeps a ref to what was injected.

[alec-chernicki]

Came across this issue and we're running into the same issue over at Disney Streaming, we deploy our applications in Docker containers. During CI having multiple node_modules folders becomes cumbersome and having a single source of truth for our deployable's dependencies would be amazing. I also came across this package but I'm getting the following error when trying to use it:

PnpmError: Cannot resolve workspace protocol of dependency "<dependencyName>" because this dependency is not installed. Try running "pnpm install".

When trying to use this workaround when running pnpm install --filter <appName> --virtual-store-dir <appDir>/node_modules/.pnpm I also get the following error because I'm in a CI environment:

 ERR_PNPM_OUTDATED_LOCKFILE  Cannot install with "frozen-lockfile" because pnpm-lock.yaml is not up-to-date with package.json

Note that in CI environments this setting is true by default. If you still need to run install in such cases, use "pnpm install --no-frozen-lockfile"

Ideally, the versions of dependencies in our deployable match those that we're using locally. I'm able to pass --frozen-lockfile=false but would I get the same benefit of inheriting from the root pnpm-lockfile.yaml like in make-dedicated-lockfile?

[maschwenk]

@AlecOrtega Have you tried using @Madvinking 's https://github.com/Madvinking/pnpm-isolate-workspace

[alec-chernicki]

@maschwenk I have not! I'll give that a try as well

[alvarosabu]

Hi there, before creating another ticket, I want to know if this is related.

I have a pnpm-workspace project with a UI package library and a Nuxt app consuming it,

When deploying to Netlify I get this error when I build the Nuxt app:

4:30:01 PM:  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  @pnpm-monorepo-nuxt/blog@ generate: `nuxt generate`
4:30:01 PM: Exit status 1
4:30:01 PM:  ELIFECYCLE  Command failed with exit code 1. (https://ntl.fyi/exit-code-1)
4:30:01 PM:  ELIFECYCLE  Command failed with exit code 1. (https://ntl.fyi/exit-code-1)

I also get something similar if I try to run some Github Actions, is the ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL related to this ticket? If so, what would be the workaround/solution?

Thanks in advance

[zkochan]

I don't know, try node-linker=hoisted

[ragrag]

Greatly appreciate the provided .pnpmfile.cjs workaround in this thread which seems to work perfectly for my use cases.

I also like the isolated deploy folder approach which is kind of similar to how rush handles this with rush deploy

[zkochan]

🚢 7.4.0-3

[iamchathu]

We need to optimize the dependancies with the deploy command only to have dependancies of the package pruned. Needs to remove all other dependancies and dev dependancies.

[zkochan]

To exclude dev dep, use

pnpm --filter=<deployed project name> --no-dev deploy <target directory>

[stefanvanherwijnen]

What is the recommended way to deploy a monorepo with private workspace packages to a Docker image? The packages in node_modules seem to be symlinked according to the monorepo structure, e.g:

- packages
  - app
  - api
- package.json
- pnpm-workspace.yaml

After deploying @monorepo/api, node_modules/@monorepo/app is symlinked to ../../../packages/app.
This doesn't make much sense as the resulting build structure is required to be:

- packages
  - app
- deployed_api

[zkochan]

inject the private stuff: https://pnpm.io/package_json#dependenciesmetainjected

[michael-land]

Hey, is there any example demo the usage? I'm new to docker, all my docker images are same size. I think pnpm deploy might help reduce the bundle size, any help is appreciated!

apps/gateway/Dockerfile
apps/service-identity/Dockerfile

FROM node:18-alpine as builder
RUN apk update
RUN apk add git
RUN corepack enable && corepack prepare pnpm@7.7.0 --activate

WORKDIR /app
COPY . .

RUN --mount=type=cache,target=/.pnpm-store \
    mkdir out && \
    cp package.json .swcrc turbo.json pnpm-workspace.yaml pnpm-lock.yaml tsconfig.json out/ && \
    cd out && \
    pnpm install && \
    pnpm exec turbo run build --filter=gateway && \
    pnpm prune --prod



FROM node:18-alpine as app

ENV NODE_ENV=production

COPY --chown=node:node --from=builder /app/out .

WORKDIR /app/apps/gateway

USER node

CMD TZ=UTC node --experimental-specifier-resolution=node dist/main.js

docke-compose.yml

version: "3.8"
services:
  gateway:
    build:
      dockerfile: ./apps/gateway/Dockerfile
    restart: on-failure:3
    volumes:
      - .:/app
    ports:
      - '5001:5001'
    networks:
      - app-network
    hostname: gateway
  identity:
    build:
      dockerfile: ./apps/service-identity/Dockerfile
    restart: on-failure:3
    hostname: microservice-identity
    volumes:
      - .:/app
    environment:
      - PORT=${GRPC_IDENTITY_PORT}
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

[o-az]

Hey, is there any example demo the usage? I'm new to docker, all my docker images are same size. I think pnpm deploy might help reduce the bundle size, any help is appreciated!

apps/gateway/Dockerfile apps/service-identity/Dockerfile

FROM node:18-alpine as builder
RUN apk update
RUN apk add git
RUN corepack enable && corepack prepare pnpm@7.7.0 --activate

WORKDIR /app
COPY . .

RUN --mount=type=cache,target=/.pnpm-store \
    mkdir out && \
    cp package.json .swcrc turbo.json pnpm-workspace.yaml pnpm-lock.yaml tsconfig.json out/ && \
    cd out && \
    pnpm install && \
    pnpm exec turbo run build --filter=gateway && \
    pnpm prune --prod



FROM node:18-alpine as app

ENV NODE_ENV=production

COPY --chown=node:node --from=builder /app/out .

WORKDIR /app/apps/gateway

USER node

CMD TZ=UTC node --experimental-specifier-resolution=node dist/main.js

docke-compose.yml

version: "3.8"
services:
  gateway:
    build:
      dockerfile: ./apps/gateway/Dockerfile
    restart: on-failure:3
    volumes:
      - .:/app
    ports:
      - '5001:5001'
    networks:
      - app-network
    hostname: gateway
  identity:
    build:
      dockerfile: ./apps/service-identity/Dockerfile
    restart: on-failure:3
    hostname: microservice-identity
    volumes:
      - .:/app
    environment:
      - PORT=${GRPC_IDENTITY_PORT}
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

why are you installing git in your Dockerfile?

[jbanety]

Was able to have working Docker containers with :

• https://github.com/Madvinking/pnpm-isolate-workspace
• node-linker=hoisted
• pnpm install --production --no-frozen-lockfile

COPY ./_isolated_/workspaces ./workspaces/
COPY ./_isolated_/package.json ./package.json
COPY ./_isolated_/node_modules ./node_modules/

Without that, I've never been able to retrieve dependencies from dependencies (deep deps).

[jonaskello]

inject the private stuff: https://pnpm.io/package_json#dependenciesmetainjected

@zkochan Is there some way to do this without having to specify all the internal packages in dependenciesMeta.*.injected in every package.json that uses the private (workspace internal) packages?

I guess also we only want this to happen during docker builds that use pnpm deploy. So the only way to solve it now would be to make some script that patches all package.json files with dependenciesMeta.*.injected during the docker build?

[Falven]

Any progress on a solution that bundles only the necessary modules?