Add from github source, in a specific subfolder

[Ploppz]

I want to add a dependency from github, but the source code for the package is only in a subfolder javascript/node/selenium-webdriver in that git repo.

$ pnpm add https://github.com/SeleniumHQ/selenium/tree/trunk/javascript/node/selenium-webdriver
 WARN  GET https://github.com/SeleniumHQ/selenium/tree/trunk/javascript/node/selenium-webdriver error (undefined). Will retry in 10 seconds. 2 retries left.
 ERR_PNPM_MISSING_PACKAGE_NAME  Can't install SeleniumHQ/selenium: Missing package name
Progress: resolved 43, reused 43, downloaded 0, added 0

I guess the reason for this error is that the root package.json does not have a "name" key, and that makes me think this is simply a feature that pnpm does not yet implement.

[folmert]

I've encountered a similar problem and adding "name" to package.json on the downloaded repository side didn't fix it.

[fivethreeo]

Could hack it by making each package a branch, but this feature would be great.

Fork selenium

git clone https://github.com/myusername/selenium
cd selenium
git subtree split -P javascript/node/selenium-webdriver -b selenium-webdriver-only
git push origin  selenium-webdriver-only

# then
pnpm add https://github.com/myusername/selenium#selenium-webdriver-only

https://stackoverflow.com/questions/69364724/turn-a-git-repo-subfolder-into-a-branch

[fivethreeo]

Thinking about this feature. It may not be as useful as it seems because many monorepo packages must be built.

Often you need to do this.

Fork selenium

git clone https://github.com/myusername/selenium
cd selenium
git checkout -b feature/i-need-built feature/i-need 
sed -i -e "s|dist/||" .gitignore
pnpm install
pnpm build
git commit "Built all the packages"
git subtree split -P javascript/node/selenium-webdriver -b selenium-webdriver-only
git checkout selenium-webdriver-only
echo node_modules > .gitignore
echo packages  >> .gitignore
echo src  >> .gitignore
git commit "Ignore leftovers"
git push origin selenium-webdriver-only

# then
pnpm add https://github.com/myusername/selenium#selenium-webdriver-only

# cleanup
git push origin --delete selenium-webdriver-only
git branch -d selenium-webdriver-only
git branch -d feature/i-need-built

[artginzburg]

You should try using https://gitpkg.vercel.app for that. It has a wonderful wizard as a web interface — just paste your wanted dependency link (https://github.com/SeleniumHQ/selenium/tree/trunk/javascript/node/selenium-webdriver) and it'll generate an install command for it.

[willwill96]

Does pnpm not run lifecycle scripts when installing via git url? For example, npm will pull down dev dependencies & run prepack/build automatically. Yarn does the same.

Using something like https://gitpkg.vercel.app/ is fine for public projects, but I think installing via git url is still useful for private repositories.

[Janpot]

In case your subfolder is a workspace, yarn berry supports GitHub dependencies of the form:

• github:<org>/<repo>#workspace=<workspace>

Would love to see support for this in pnpm

[scarf005]

i needed to install specific commit of vite-node as one of its recent commit fixed a major issue i was facing; however vite-node is a package from vitest monorepo, so i had to fork and release my own using pnpm pack. It'd be great if this feature also supported pnpm workspaces.

[ChristopherHaws]

The monorepo I'm trying to add has dist folders in the package's folder so no build would be necessary. Would really love this feature! 👍

[matthew-dean]

I'd also love this for using a forked version of DefinitelyTyped, such that I could pull a specific (pending) types definition.

[zkochan]

We already have the git+https://host/user/repo#semver:^2.0.0 syntax for resolving by semver (using git tags). So I think the right syntax for this one would be git+https://host/user/repo#path:<relative path>. This would be more generic than by workspace package name used by Yarn.

We build the git-hosted packages before adding them to node_modules. pnpm detects the package manager used by the repository and runs pnpm/npm/yarn install. Then it runs the prepare scripts. If we want to support monorepos, we'll have to probably detect if the workspace feature is used and if so, then run install in the root of the repository before taking the subdirectory.

[RexSkz]

If we use the URL hash syntax, users are not able to install by both branch and path.

[zkochan]

Why? You can have multiple # sections. Like

git+https://host/user/repo#next#path:<relative path>

[RexSkz]

It doesn't satisfy the URL standard (there should be at most one fragment, and no unescaped # is allowed in a fragment).

fragment = *( pchar / "/" / "?" )

Also, it looks weird, I haven't seen a URL which has more than one fragment...

---

To my surprise, it seems okay to be parsed using new URL and the returned value is as expected:

[zkochan]

It is a version spec though, not a URL. Like this is also a valid git-hosted dependency: github:drizzle-team/drizzle-orm. Also, the hash in this case is not part of the URL. The hash is only accessible on the client side (on a website), so we can't clone a repo with information in the hash. It is just a coincidence that a hash is used to delimit the git URL from additional info.

[RexSkz]

Okay, I'll make changes to my PR according to the spec. It'll change the way that PNPM handles the hash.

[romeovs]

Is this issue actually closed?

I'm not seeing this feature documented in the pnpm add docs, nor does it actually seem to work as I'm seeing:

$ pnpm -v
8.15.5

$ pnpm add "https://github.com/romeovansnick/util#path:env"

ERROR  Could not resolve path:env to a commit of git@github.com:romeovansnick/util.git.
...

Am I missing something here?

[RexSkz]

It's shipped in pnpm v9; you can upgrade to v9 and try again.

[zkochan]

Correct, it was too big of a change to be added to v8, as we are in the process of shipping out v9. v9 is not stable yet, you can use it but the lockfile format might still have changes to it.

[romeovs]

Ah that's my bad, I should have tried v9.

I'm running into another issue on 9.0.0-alpha.8:

$ pnpm -v
9.0.0-alpha.8

$ pnpm add 'github:romeovansnick/util#path:sitemap'
node_modules/.pnpm                       |  WARN  Cannot find resolution of git+git@github.com:romeovansnick/util.git#fc02ba85407981f591391b4fb9a467e75ee967e7&path:sitemap in lockfile
 WARN  6 deprecated subdependencies found: @babel/plugin-proposal-class-properties@7.18.6, @babel/plugin-proposal-object-rest-spread@7.20.7, debug@4.1.1, fsevents@2.1.3, react-remove-scroll-bar@2.3.5, uuid@3.3.2
 ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY  Broken lockfile: no entry for 'git+git@github.com:romeovansnick/util.git#fc02ba85407981f591391b4fb9a467e75ee967e7&path:sitemap' in pnpm-lock.yaml

This issue is probably caused by a badly resolved merge conflict.
To fix the lockfile, run 'pnpm install --no-frozen-lockfile'.
Progress: resolved 1640, reused 1559, downloaded 0, added 0

(sorry for the bad formatting, but this is the literal pnpm output).

This error seems related but might not be so let me know if you'd like me to open a separate issue.