[Bug?]: Checksum mismatch between corepack and non-corepack installs because of newline in package.json

[TuurDutoit]

Self-service

• I'd be willing to implement a fix

Describe the bug

When switching between corepack enable and corepack disable, Yarn complains about a checksum mismatch on a package installed with the git: protocol. This seems to be because the packing happens with a different version of Yarn, which adds / removes an extra newline at the end of the package.json of the dependency.

I had a similar problem before with Yarn 3.2.1 (adding a packageManager field to the package.json, IIRC). I upgraded to v3.2.3, which should have resolved that problem, but now I have this.

To reproduce

Steps to reproduce:

• Create a new, empty directory
• Create the following package.json:

{
  "dependencies": {
    "timecop": "git+https://github.com/jamesarosen/Timecop.js.git#d421df9b00386266bda93eb9c2ea15fedc888761"
  }
}

• yarn set version berry or yarn set version from sources (I could reproduce with both)
• corepack disable
• yarn
• corepack enable (you can switch these around - first enable, then disable fails for me too)
• rm -rf node_modules
• yarn cache clean --all
• yarn -> fails with YN0018: │ timecop@https://github.com/jamesarosen/Timecop.js.git#commit=d421df9b00386266bda93eb9c2ea15fedc888761: The remote archive doesn't match the expected checksum

I tried to make a reproducible case with Sherlock, as suggested - I made this, but it works fine there:

const path = require('path')
const {execFile} = require(`child_process`)
const {promisify} = require(`util`)

const execFileP = promisify(execFile)

async function exec(execPath, args = [], opts = {}) {
  try {
    const {stdout} = await execFileP(execPath, args, opts)
    return stdout
  } catch (error) {
    error.message += `\n${error.stdout}`;
    throw error;
  }
};

await packageJson({
  "dependencies": {
    "timecop": "git+https://github.com/jamesarosen/Timecop.js.git#d421df9b00386266bda93eb9c2ea15fedc888761"
  },
})

const nodePath = process.argv[0]
const corepackPath = path.resolve(nodePath, '../corepack')

await yarn('set', 'version', 'berry')
await exec(corepackPath, ['enable'])
await yarn('install', '--inline-builds')

await exec(corepackPath, ['disable'])
await exec('rm', ['-rf', 'node_modules'])
await yarn('cache', 'clean', '--all')
await expect(yarn('install', '--inline-builds')).resolves.toMatch(/Done in \d+s/)

As an alternative, I put these steps in a Dockerfile, which does fail on the last step:

# Prepare
FROM node:18
WORKDIR /project
COPY package.template.json package.json
RUN yarn set version berry

# Corepack: ENABLED
RUN corepack enable
RUN rm -rf node_modules
RUN yarn cache clean --all
RUN yarn --version
RUN yarn

# Corepack: DISABLED
RUN corepack disable
# With Corepack disabled, we need to get Yarn from somewhere else
RUN npm install -g yarn
RUN rm -rf node_modules
RUN yarn cache clean --all
RUN yarn --version
RUN yarn

with the following package.json:

{
  "dependencies": {
    "timecop": "git+https://github.com/jamesarosen/Timecop.js.git#d421df9b00386266bda93eb9c2ea15fedc888761"
  }
}

Environment

System:
    OS: macOS 12.4
    CPU: (10) arm64 Apple M1 Pro
  Binaries:
    Node: 16.15.1 - /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-4a2199df/node
    Yarn: 3.2.3 - /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-4a2199df/yarn
    npm: 8.11.0 - ~/Library/Caches/fnm_multishells/59621_1664283223571/bin/npm
  npmPackages:
    jest: ^26.6.3 => 26.6.3

Additional context

I managed to get the zip archives from the Yarn cache, as well as the logs from the yarn pack:

• with corepack enabled:
  • timecop-https-21fab97b94-b209b91b1f-corepack-enabled.zip
  • corepack-enabled.log
• with corepack disabled:
  • timecop-https-21fab97b94-fbbbc614ae-corepack-disabled.zip
  • corepack-disabled.log

As you can see, the version used for yarn pack is different:

• with Corepack enabled, it's using v1.22.15
• with Corepack disabled, it's using v2+, I think?

Note that I have a global Yarn install from Homebrew:

$ which yarn
/opt/homebrew/bin/yarn

The same happens when Yarn is installed through npm (as per the Dockerfile repro).

Another interesting finding: no matter if Corepack is enabled or disabled, the Yarn binary & version stays the same:

# From my home directory
# Results are the same from within my project directory, except the version is 3.2.3 there
$ corepack disable
$ which yarn
/opt/homebrew/bin/yarn
$ yarn --version
1.22.19
$ corepack enable
$ which yarn
/opt/homebrew/bin/yarn
$ yarn --version
1.22.19

[yarnbot]

The reproduction case in your issue seems broken (ie it neither pass nor fail due to throwing an unmanaged exception):

Error: spawn /usr/bin/corepack ENOENT

    at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
    at onErrorNT (node:internal/child_process:478:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Remember: any non-Jest exceptions will cause the test to be reported as broken. If you expect something to pass without throwing, you must wrap it into something like await expect(...).resolves.toBeTruthy(). If you instead expect something to throw, you need to wrap it into await expect(...).rejects.toThrow().