Can't disable modules caching

[PavelSapachev]

My reason:

I can't disable modules caching with cache: false option. Installing 3rd party modules through Verdaccio still store this modules folders in storage (but without tarballs).

App Version:

Latest version of Docker image

Config file:

uplinks:
  npmjs:
    url: https://registry.npmjs.org/
    cache: false

[juanpicado]

The cache property was introduced on #131 and the initial idea if you allow me to paragraph @silkentrance

Since both reside on the same network, it does not make much sense for the proxy to cache the tarballs it gets from the private registry. (and maybe this should be extended to also the metadata)

So, the behavior is correct, metadata is hosted, this might be changed but honestly, I do not see a big problem to host small metadata files. Please feel free to PR any improvement to the current disable cache approach.

[drubin]

I like the idea of being able to at least separate the local internal packages that are only published to the verdaccio server and the proxied packages.

It makes it much easier to backup/store/version.

I was hoping to set cache: false would actually not result in cached files on disk. Not just to save disk space, disk is cheap but to make managing the server easier. If you are using this as a private repository for maybe 2 or 3 packages having 1000's of metadata files makes it much harder to keep in sync and might result in increased latency with the s3 or remote storage backends.

This configuration option is a little confusing. Since it does actually cache things.

I understand this is a breaking change but I would recommend we think about this for v3.

[juanpicado]

thanks @drubin I'll definitely take a look. I'm sure there is a lot room for improvement

[silkentrance]

@juanpicado @drubin I think that this can easily be changed, that is, also dropping the cached meta data from the local repository. I kept the meta data in place in order to speed things up a bit in case that the meta data did not change.

@drubin feel free to provide a PR for this. alternatively I can look at the code an get rid of the metadata also.

[silkentrance]

@drubin and it is not a breaking change at all, since it happens in the background. the user never sees the metadata.

[silkentrance]

@drubin @juanpicado I have just looked at the new code, especially the new API and I think I know the reason for keeping the meta data in place as it will speed things up a lot when querying for package information a/o searching for available packages.

Nonetheless, the new code just reuses the existing storage.get_package() code and the change should be done quite easily. However, it will induce additional latency since the lookup of the remote meta data will take quite some time, especially with the current behaviour of merging the meta data from multiple sources.

[juanpicado]

@silkentrance exactly, otherwise the word proxy wouldn't make sense, also, we modify and update metadata when something new comes up in each library. Furthermore, if you want to scale and you have to deal with multiple processes, each one would have to ask any registries for tarballs (since metadata is updated all the time), when using a proper storage would be faster since, usually, the proxy is in the same network. I think tarballs are mostly the pain of everybody since along the time tends to use a lot of storage, metadata is small and much easier to handle it IMHO.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.