[Bug]: Publishing to verdaccio is very slow

[snowystinger]

What happened?

I work in a monorepo with > 200 packages. When we want to test as though we are using published artefacts, we publish to verdaccio so it's closer to how our users would install our packages. This works fantastically.

Unfortunately it takes several minutes running the publish command. Each package taking ~1s or more

I've checked it with 6.0.5, 6.9.2, and 7.0.0-next-7.23. I couldn't find any information on v9, so I didn't try that. I've had -pt flags set on yarn, I tried without the t as well, no better results.

I'm able to create the verdaccio database directly in a fraction of the time, 11s overall as opposed to the ~2.5min going through verdaccio to build it.

Hopefully I've just missed some setting, but if not, I'd like some way to either speed up the verdaccio publish, or, barring that, a CLI tool that can seed the database for me as though I've used the publishing.

Version

6.x (Stable)

Version details

6.0.5

Output server log info

Here is some of the output

[@react-types/text]: ➤ YN0000: Publishing to http://localhost:4000 with tag latest
[@react-types/text]: ➤ YN0000: README.md
[@react-types/text]: ➤ YN0000: package.json
[@react-types/text]: ➤ YN0000: src/index.d.ts
[@react-types/text]: ➤ YN0000: Package archive published
[@react-types/text]: ➤ YN0000: Done in 0s 510ms
[@react-types/text]: Process exited (exit code 0), completed in 0s 957ms
[@react-types/view]: ➤ YN0000: Publishing to http://localhost:4000 with tag la****
[@react-types/view]: ➤ YN0000: README.md
[@react-types/view]: ➤ YN0000: package.json
[@react-types/view]: ➤ YN0000: src/index.d.ts
[@react-types/view]: ➤ YN0000: Package archive published
[@react-types/view]: ➤ YN0000: Done in 0s 428ms
[@react-types/view]: Process exited (exit code 0), completed in 0s 731ms
[@react-types/well]: ➤ YN0000: Publishing to http://localhost:4000 with tag la****
[@react-types/well]: ➤ YN0000: README.md
[@react-types/well]: ➤ YN0000: package.json
[@react-types/well]: ➤ YN0000: src/index.d.ts
[@react-types/well]: ➤ YN0000: Package archive published
[@react-types/well]: ➤ YN0000: Done in 0s 453ms
[@react-types/well]: Process exited (exit code 0), completed in 0s 722ms

Node.js Version

24.x.x (LTS)

Package manager

yarn

Operating system

mac

Using reverse proxy

• I am using a reverse proxy

Relevant log output

Have I checked other issues?

• I have searched existing issues

[mbtools]

Is this your config, @snowystinger ?

https://github.com/adobe/react-spectrum/blob/main/scripts/verdaccio-config.yaml

[snowystinger]

Yes, that's it

[mbtools]

From email

It looks like the repo is using a default config. Each package needs to be packed, sent to Verdaccio, compared to the npm registry, merged with the manifest from npm, and finally stored. So, in general, the publishing times seem reasonable (using v9 won't change any of this).

Your explanation makes sense, that's a lot of extra network requests I was not thinking of.

We only run verdaccio in our CI to create test app builds as if they were built from the real npm registry. Because of this, we know a few things up front. We will always be publishing a version that doesn't already exist on npm and we never need to know other versions. I assume that's what those requests are doing.

One option could be to cut it off from the internet while constructing the registry database, I don't know how long requests would take to timeout. Or would that cause issues? I know very little about how the npm manifests work.

Otherwise, I was thinking maybe I'd missed something to increase the parallelism other than yarn -p. Or, as I was musing in the Issue, some CLI that allowed users to bypass all of those network requests and directly seed the database. That way I wouldn't need to reverse engineer pakument (I think that was the special thing the AI was recreating for me).

The default makes sure that the version you try to publish does not exist in npm already. If you don't care about this, you can remove the proxy (npm uplink) for those packages. For example

packages:
  '@react-types/*':
    access: $all
    publish: $all

# add other scoped packages or list them individually
#...

  '@*/*':
    # scoped packages
    access: $all
    publish: $all
    proxy: npmjs

  '**':
    access: $all
    publish: $all
    proxy: npmjs

This should make it faster.

[juanpicado]

Another extra optimisation is using JWT, since token is only verified very cheap for E2E envs. https://www.verdaccio.org/docs/best#expiring-tokens

security:
  api:
    jwt:
      sign:
        expiresIn: 60d
        notBefore: 1

[juanpicado]

I added something new to v6.10.0 #6147

if you enable

server:
  legacyAuthCache:
    enabled: true
    ttlMs: 15000 <-- here is 15 seconds but could be your choice to go further
    maxEntries: 1000

It cache the authentication, the problem with publishing is each request is always authenticated, package manager send the token, this takes some ms from each request on publishing, with the cache avoids do this each time for x period of time, very handy for env where authentication is just a required step like E2E, give it a try, benchmark I did look good. @snowystinger