Write tests for publishing

[emmatown]

I'm thinking we could use https://verdaccio.org rather than mocking so we're actually testing against a real npm registry.

I've done a little bit of looking and it seems as though verdaccio doesn't support 2fa so we couldn't test that but we could at least test the core publishing flow.

[Andarist]

Posted by @mitchellhamilton here

tbh, what I'd really like is tests that run with all the different package managers against a local registry (probably https://verdaccio.org) with a way to run the tests against actual registries.

The Verdaccio idea is quite nice and could be setup. How would you run tests against actual registries? Or maybe you have not meant the "official" registries like npm, yarn, and github packages?

cc @belgattitude

[juanpicado]

:) just 🚶🏼 around ... I could support this ( I haven't checked the code yet) but I use a lot changeset :) and pretty happy.

[belgattitude]

Thanks @Andarist and @mitchellhamilton could be nice yes.

How would you run tests against actual registries?

I'm not sure there's a need to ? Verdacio seems to follow the same protocol (some things maybe https://verdaccio.org/docs/cli-registry/ might have to be set either in npmrc, either though npm config). At first sight it seems realistic.

For the multiple package manager tests (npm, yarn, pnpm), I've added a note in #674 (comment) (look section about unit tests). It's worth considering, not saying it should be done.

[Andarist]

given the package manager cli receive the correct params it will do the job or it's a bug on their side. They have an e2e suite that runs on many os... spawn (which I don't know about) has it's own tests too, I guess. So for me what is to test is the command we run, less the fact that npm, pnpm, yarn 1.2.3 will actually publish what what we are feeding to it. Command line is somehow a contract. Same idea when I test a server, do I need to test if my express server works or just my handler/use-case (business...) .... There's not big difference when you write the tests, but running them is much easier when you don't need to start the actual express server (time, flacky...).

I agree with the principles laid out here - although if it's not too hard to pull off I usually prefer e2e/integration tests over units. My users don't care that much how I've implemented a particular thing and how many external dependencies I've used - if they contain bugs then it's affecting my product/tool directly and it's still important for me to test if a particular dependency satisfies my requirements.

Not every e2e scenario is easy to set up and I would classify this one here as something that is definitely not easy to set up. And as I said - I'm OK with testing such things by mocking stuff etc. The problem is that those commands don't only accept the given arguments - if that would be the case then I wouldn't be much worried about the whole thing. Their execution is sensitive to cwd, env variables and the file system structure itself (.npmrc lookup etc). The docs for all of this are scattered through different pages and I think that they are not even exhaustive.

I'm not sure there's a need to ? Verdacio seems to follow the same protocol (some things maybe https://verdaccio.org/docs/cli-registry/ might have to be set either in npmrc, either though npm config). At first sight it seems realistic.

Yes, it seems that this could be enough - I've looked now through the npm docs and it also seems that we can configure both the user and global config files to be used. So we could just "mock" them with a custom Verdaccio-powered registry.

We could even potentially use proxy/https-proxy to spy on the traffic - unsure if that would really be useful with Verdaccio stuff in place, but maybe it would allow us to test the OTP stuff better (since, from what I understand, this is not supported by Verdaccio).

:) just 🚶🏼 around ... I could support this ( I haven't checked the code yet) but I use a lot changeset :) and pretty happy.

@juanpicado would you like to help us out with setting this up? This could give us a good headstart.