feat: Prefer package.yaml (or .json5) to package.json when both are present

[qyurila]

• I'd be willing to implement this feature (contributing guide)

Describe the user story

pnpm supports package.yaml and package.json5 since #1799. But, there are a bunch of npm packages that directly read package.json file, hence cannot be used without additional package.json file.

I think, as an end user, the best way to handle this issue is using some monitoring tools and/or converters to automatically convert package.yaml to package.json in order to allow some packages to find it (and we'd probably want to add package.json in .gitignore). In my opinion, it's barely possible to hope every single packages there to support alternative manifest files because of its little popularity (and the presence of YAML haters).

However, this workaround is currently not very doable in the real life because pnpm tries to read and write package.json file when it's found, regardless package.yaml is at the same place or not. It makes an additional json-to-yaml converter to be required and makes the workaround very complicate.

Describe the solution you'd like

Make @pnpm/read-importer-manifest and @pnpm/write-importer-manifest to find package.yaml and package.json5 before package.json, so that this alternative manifest files are prefered to package.json, which means package.json would be only read by some packages and package.yaml would be modified and read most of the time.

Describe the drawbacks of your solution

I think most of the project which has both of package.yaml and package.json would eventually benefit by this change, but it can introduce breaking change and unexpected behavior.

Describe alternatives you've considered

Adding --manifest-ext (json|yaml|json5) option to pnpm CLI: It'd be inconvenient because it probably should be followed to every pnpm commands.
Adding manifestExt option to pnpm-workspace.yaml: Seems not bad at least for me, but I'm not sure if this is proper config file to have this kind of option.

[gusbemacbe]

Pryvit i dobroho ranku, @zkochan!

Please, let me create a YAML file with a pnpm command. I prefer YAML to JSON or JSON5.

[bmulholland]

FWIW, we're migrating to pnpm specifically for JSON5 support, but this issue prevents us from even using workarounds of compiling package.json for everything that needs it. We're holding off on our migration until then.

[gusbemacbe]

Dobryi den', @zkochan!

Any news about this choice of YAML over JSON?

[rlidwka]

FWIW, we're migrating to pnpm specifically for JSON5 support, but this issue prevents us from even using workarounds of compiling package.json for everything that needs it.

@bmulholland, what other tools are you using that already support it?

[rlidwka]

When both files (package.json5 and package.json) are present, it is very likely that:

• either package.json is directly compiled from package.json5
• or package.json is just a stub for tools that don't support json5

So preferring package.json5 is the right choice from usability standpoint.

Are you're worrying about performance for the extra .stat call? If that's the case, perhaps we should make some effort to change other tools to support json5, so package.json won't ever be needed.

[bmulholland]

@rlidwka We have a package.json5 that is really only used by us, developers. I wrote a short script that compiled the JSON5 to package.json. That's the only tool that really uses or touches it. It adds a step to any change we make to our package.json(5), but it's well worth it -- the whole thing is probably 25% comments and they're all so useful.

FYI: It also means that we can't really use version pinning in our package.json, at least if we use Dependabot upgrades, which goes against the grain of the JS community. That's alright with us, we set version pins to "latest," which is IMHO what all applications (i.e. not packages others depends on) should use. Different discussion, point is that it works for us but has blockers for most people following more standard conventions.

Here's our scripts entry that compiled package.json5 to package.json, complete with explanation and references... feel free to use the script.

    // See https://github.com/nodejs/node/issues/40714
    // Our json5 is sorted sanely, while yarn sometimes sorts it alphabetically
    // (e.g. during upgrades), so the sort package makes it consistent between
    // the json5 conversion and the upgrade-sorting
    // The echo adds a newline to the end of the file; the preceding commands
    // don't, and without it yarn upgrade makes a change to the file to add it
    "compile-package-json": "json5 -o package.json --space 2 package.json5 && npx sort-package-json && echo '' >> package.json",

I think this also makes the point about why comments in these files are useful!

[talr]

We tried to switch from package.json to package.json5, and we found this issue with pnpm version patch
#10372

Can we force pnpm to use json5?

Any thoughts?

[skagedal]

I would also like to the support for this to improved. As it currently is, pnpm has support for package.yaml and package.json5, but in practice both are difficult to use – you end up having tools fighting each other.

I don't think it would be a good idea to change the preference order. That would be a breaking change that seems likely to just cause confusion for anyone who attempts this, also given different versions of pnpm involved (what version is reading the manifest file to resolve which version of pnpm to run for the actual commands, etc).

I think it would be better with an explicit setting, naturally outside the manifest file itself. The pnpm-workspace.yaml file seems like the right place for this. I have drafted a PR to implement this here, with (also draft) docs update PR here. Would love to hear thoughts on the feature design before moving forward on the implementation.

@zkochan mentioned here that he's overall "not happy about all this different package.json file formats bloat", and I totally understand that position. Big fan of keeping things simple myself. As I expanded on there, I still think it's worth it to have a manifest format that allows comments – that's the main thing for me.

Having two alternative formats seems especially a bit over the top. I'd be in favor of deprecating one of them. If it were completely up to me, I'd keep JSON5 – it just seems a much better fit for tooling in the Javascript ecosystem – but since YAML is already used in several places in pnpm, I'd see the argument for choosing that one.

As JSON5 is still a supported format, however, I'd love to help out improving the support for it – see also this PR, which I think is ready to be reviewed.

[zkochan]

i believe bun supports jsonc and we support bun as runtime. so, we can support jsonc format in package.json but it won't work with node.js.

[zkochan]

the big issue with .json5 and .yaml are that the runtimes don't support them. one alternative solution could be to generate package.json from other formats and gitignore package.json.

[danielbayley]

the big issue with .json5 and .yaml are that the runtimes don't support them. one alternative solution could be to generate package.json from other formats and gitignore package.json.

This is what I do, and it works well… (I compile package.yaml to package.json in my prepare lifecycle script, before subsequent scripts run, and add *.json to my .gitignore. Longterm, pnpm could simply supply the appropriate hooks at all the right stages, which would free up anyone to use any format they prefer, with an appropriate devDependency…

[skagedal]

Yes, runtimes and bundlers and various other tools in the ecosystem typically expect to read some things from package.json. But the needs vary between different projects. For some applications, Node.js would basically only need a package.json that contains {"type": "module"}. Having that alongside a package.json5 (or package.yaml) for pnpm would work perfectly, if I could only tell pnpm to prefer the JSON5 file.

I've also tried the approach of generating package.json from package.json5, but that also gets confusing, since now pnpm will treat the package.json5 as the manifest file on a clean repo – but, once that generation has been run, it will prefer package.json. That also means that writes go to that file – a pnpm add will add dependencies to the generated file, rather than the one you want.

This too would be alleviated if there was a way to tell pnpm what manifest format to prefer.