Lage crashes (EPIPE) when total byte size of project paths or hash output exceeds 64KB (on MacOS).

[kensternberg-authentik]

Describe the bug

// ./src/getPackageDeps.ts: line 137
    const result = execa.sync(gitPath || "git", ["hash-object", "--stdin-paths"], {
      input: filesToHash.map((x) => path.resolve(packagePath, x)).join("\n"),
    });

The buffers execa uses to execute this command have an upper limit. I don't know if that limit is specified in execa, NodeJS, or by the operating system, but on MacOS Ventura 13.3 it appears to be 64KB.

I am attempting to break up a medium-sized monolith into individual packages. With the existing node_modules collection, that monolith's list of 56,038 hashable files is 3.3MB, far exceeding the 64MB limit.

To Reproduce:

1. Create a simple zero-package project for lage.
2. Into the empty packages folder, place a large and complex project.
3. Configure the root package.json to recognize the new workspace, and lage to build it.
4. Attempt to build the project.
5. See the error:

Running with 11 workers
node:internal/errors:496
    ErrorCaptureStackTrace(err);
    ^

Error: Command failed with EPIPE: git hash-object --stdin-paths
spawnSync git EPIPE
fatal: Unable to hash /Users/ken/projects/dev/web/node_modules/@goauthentik/web
c6ea78f6224fcba40713d28c20319873aa4733e3
06fccc32b1939efb64b92d9ae37bc21fb9118051
552cb0764a8c0cf7a937b25a04f55f0b70ea51cb
31479ed80f5dc571782e0a4d31658ba78adf0108
1b2db5bd40698e7f8467eb0cfe901cf38517f48d
... (many lines of hash output, ending unexpectedly and with an incomplete last line)

Expected behavior

I expected the build to complete.

Environment/Troubleshooting

  System:
    OS: macOS 13.3
    CPU: (12) arm64 Apple M2 Max
    Memory: 273.66 MB / 32.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 18.18.0 - ~/.nvm/versions/node/v18.18.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v18.18.0/bin/yarn
    npm: 9.8.1 - ~/.nvm/versions/node/v18.18.0/bin/npm
  Managers:
    Cargo: 1.71.0 - ~/.cargo/bin/cargo
    Homebrew: 4.2.0 - /opt/homebrew/bin/brew
    pip3: 23.1.2 - ~/Library/Caches/pypoetry/virtualenvs/authentik-BoGTmjdo-py3.12/bin/pip3
    RubyGems: 3.0.3.1 - /usr/bin/gem
  Utilities:
    Make: 3.81 - /usr/bin/make
    GCC: 14.0.3 - /usr/bin/gcc
    Git: 2.39.2 - /usr/bin/git
    Clang: 14.0.3 - /usr/bin/clang
    FFmpeg: 6.0 - /opt/homebrew/bin/ffmpeg
    Curl: 7.87.0 - /usr/bin/curl
  Virtualization:
    Docker: 23.0.5 - /usr/local/bin/docker
  IDEs:
    Emacs: 29.1 - /opt/homebrew/bin/emacs
    Vim: 9.0 - /usr/bin/vim
    Xcode: /undefined - /usr/bin/xcodebuild
  Languages:
    Bash: 3.2.57 - /bin/bash
    Go: 1.20.4 - /usr/local/go/bin/go
    Perl: 5.30.3 - /usr/bin/perl
    Python: 3.12.0 - /Users/ken/Library/Caches/pypoetry/virtualenvs/authentik-BoGTmjdo-py3.12/bin/python
    Python3: 3.12.0 - /Users/ken/Library/Caches/pypoetry/virtualenvs/authentik-BoGTmjdo-py3.12/bin/python3
    Ruby: 2.6.10 - /usr/bin/ruby
    Rust: 1.71.0 - /Users/ken/.cargo/bin/rustc
  Databases:
    PostgreSQL: 14.9 - /opt/homebrew/bin/postgres
    SQLite: 3.39.5 - /usr/bin/sqlite3
  Browsers:
    Chrome: 120.0.6099.129
    Safari: 16.4
  Monorepos:
    Yarn Workspaces: 1.22.19

[kensternberg-authentik]

Closing this in favor of Lage hasher crashes trying to run git hash-object on a symlink. This appears to have been a consequence of that bug, and the fact that it crashed within ~40 bytes or so of processing exactly 64KB was a red herring. It seemed a reasonable assumption that 40 bytes was about the length of a path and so this was "going over" some 64KB limit, but further investigation revealed the symlink to have been the real issue.