New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for installing dependencies without manifest #3624

Merged
merged 3 commits into from Jun 22, 2017

Conversation

Projects
None yet
3 participants
@sheerun
Contributor

sheerun commented Jun 12, 2017

This is continuation of #2651. Changes:

  1. Fix extracting default name from url so .git extension won't get into way
  2. Don't write any package.json, store default manifest in .yarn-metadata.json instead.
    Bonus: Cache normalized package.json in in .yarn-metadata.json, so installs should get faster.
  3. Ensure one can install named package like foobar@https://github.com/foo/bar.git
  4. Add support for file: resolver as well
  5. Fix yarn check so it allows for no package.json in installed dependencies

Any comments?

async readManifest(dir: string, priorityRegistry?: RegistryNames, isRoot?: boolean = false): Promise<Manifest> {
const manifest = await this.maybeReadManifest(dir, priorityRegistry, isRoot);
readManifest(dir: string, priorityRegistry?: RegistryNames, isRoot?: boolean = false): Promise<Manifest> {
return this.getCache(`manifest-${dir}`, async (): Promise<Manifest> => {

This comment has been minimized.

@bestander

bestander Jun 16, 2017

Member

What is the reason to move getCache from maybeReadManifest here?

@bestander

bestander Jun 16, 2017

Member

What is the reason to move getCache from maybeReadManifest here?

This comment has been minimized.

@sheerun

sheerun Jun 18, 2017

Contributor

yes, it's because maybeReadManifest was called few times in the same process, and the getCache there memoized "nil" return value. base-fetcher/fetch method writes the manifest between these calls, so nil was returned even after manifest was written.

moving cache to readManifest fixes the issue as getCache doesn't cache rejected promises (as opposed to resolved nil values)

@sheerun

sheerun Jun 18, 2017

Contributor

yes, it's because maybeReadManifest was called few times in the same process, and the getCache there memoized "nil" return value. base-fetcher/fetch method writes the manifest between these calls, so nil was returned even after manifest was written.

moving cache to readManifest fixes the issue as getCache doesn't cache rejected promises (as opposed to resolved nil values)

This comment has been minimized.

@sheerun

sheerun Jun 18, 2017

Contributor

Actually this was always a bug, but didn't surface. With this PR I actually expect manifest to change (be created) across reads, so I needed to fix this.

@sheerun

sheerun Jun 18, 2017

Contributor

Actually this was always a bug, but didn't surface. With this PR I actually expect manifest to change (be created) across reads, so I needed to fix this.

@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Jun 16, 2017

Member

@sheerun, thanks for catching this up.
One test got broken though nstall should write and read integrity file based on lockfile entries

Member

bestander commented Jun 16, 2017

@sheerun, thanks for catching this up.
One test got broken though nstall should write and read integrity file based on lockfile entries

sheerun added some commits Jun 18, 2017

Fix yarn check test for checking integrity with lockfile
Now that we allow for installing packages without package.json,
shallow integrity check will succeed if it's removed from package.

I fix the test by removing installed director instead.
@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jun 18, 2017

Contributor

@bestander I fixed the test. It's behavior changed, as if we allow for installing packages without package.json, simply removing package.json from installed dependency is not enough to fail shallow integrity check, we need to remove whole directory, and that's what I did to fix this test.

Contributor

sheerun commented Jun 18, 2017

@bestander I fixed the test. It's behavior changed, as if we allow for installing packages without package.json, simply removing package.json from installed dependency is not enough to fail shallow integrity check, we need to remove whole directory, and that's what I did to fix this test.

@bestander bestander merged commit b223d51 into yarnpkg:master Jun 22, 2017

2 of 3 checks passed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Jun 22, 2017

Member

Great work, @sheerun!
That is some impressive change across many layers of code.
Anything else you'd like to help out with? :)

Member

bestander commented Jun 22, 2017

Great work, @sheerun!
That is some impressive change across many layers of code.
Anything else you'd like to help out with? :)

GulajavaMinistudio added a commit to GulajavaMinistudio/yarn that referenced this pull request Jun 22, 2017

Merge pull request #58 from yarnpkg/master
Allow for installing dependencies without manifest (yarnpkg#3624)
@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Jun 22, 2017

Oh wow! Thank you for that!

andreypopp commented Jun 22, 2017

Oh wow! Thank you for that!

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jun 22, 2017

Contributor

@bestander How about support for selecting a tag from git repository by semver selector?

Something like yarn add bestander/chrome-app-livereload@0.0.x or yarn add bestander/chrome-app-livereload#0.0.x depending on what you agree on

Contributor

sheerun commented Jun 22, 2017

@bestander How about support for selecting a tag from git repository by semver selector?

Something like yarn add bestander/chrome-app-livereload@0.0.x or yarn add bestander/chrome-app-livereload#0.0.x depending on what you agree on

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jun 22, 2017

Contributor

Probably # makes more sense, as then the source of package can be expressed as

NAME@[REGISTRY#]SEMVER, where registry is optional

in case of git repositories, the registry is repository itself, so

yarn add jquery@jquery/jquery#1.x

makes sense as it adds jquery in version 1.x from "jquery/jquery" registry (git tags), and

yarn add jquery@1.x

installs version 1.x from default "npm" registry

Contributor

sheerun commented Jun 22, 2017

Probably # makes more sense, as then the source of package can be expressed as

NAME@[REGISTRY#]SEMVER, where registry is optional

in case of git repositories, the registry is repository itself, so

yarn add jquery@jquery/jquery#1.x

makes sense as it adds jquery in version 1.x from "jquery/jquery" registry (git tags), and

yarn add jquery@1.x

installs version 1.x from default "npm" registry

@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Jun 23, 2017

Member

The problem is that # is used for git hashes in git resolution logic.

Member

bestander commented Jun 23, 2017

The problem is that # is used for git hashes in git resolution logic.

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jun 23, 2017

Contributor

This probably wouldn't be an issue as hashes look much different than semver. In Bower there's simple check for hash (4+ more word characters), and nobody had any issues: https://github.com/bower/bower/blob/master/lib/core/resolvers/GitResolver.js#L178

Contributor

sheerun commented Jun 23, 2017

This probably wouldn't be an issue as hashes look much different than semver. In Bower there's simple check for hash (4+ more word characters), and nobody had any issues: https://github.com/bower/bower/blob/master/lib/core/resolvers/GitResolver.js#L178

@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Jun 23, 2017

Member

@sheerun, that would be really great to have this feature.
Go ahead!

Member

bestander commented Jun 23, 2017

@sheerun, that would be really great to have this feature.
Go ahead!

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jun 25, 2017

Contributor

@bestander There was some change since my PRs on master branch, and now yarn add bestander/chrome-app-livereload fails with:

An unexpected error occurred: "https://github.com/bestander/chrome-app-livereload.git: Request failed "406 Not Acceptable"".

Contributor

sheerun commented Jun 25, 2017

@bestander There was some change since my PRs on master branch, and now yarn add bestander/chrome-app-livereload fails with:

An unexpected error occurred: "https://github.com/bestander/chrome-app-livereload.git: Request failed "406 Not Acceptable"".

@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Jun 26, 2017

Member

@sheerun, there were not that many changes since the merge and latest CI build shows the tests passing https://circleci.com/gh/yarnpkg/yarn/4198

Member

bestander commented Jun 26, 2017

@sheerun, there were not that many changes since the merge and latest CI build shows the tests passing https://circleci.com/gh/yarnpkg/yarn/4198

@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Jul 5, 2017

Member

@sheerun, looks like the tests at __tests__/package-resolver.js are passing fine but the solution fails at end to end installation.
If you have a chance to have a look and fix the issue that would be really great, I think we need to cover this feature with a proper e2e test.

Member

bestander commented Jul 5, 2017

@sheerun, looks like the tests at __tests__/package-resolver.js are passing fine but the solution fails at end to end installation.
If you have a chance to have a look and fix the issue that would be really great, I think we need to cover this feature with a proper e2e test.

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Jul 7, 2017

Contributor

Yeah I was surprised these tests weren't e2e

Contributor

sheerun commented Jul 7, 2017

Yeah I was surprised these tests weren't e2e

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Oct 2, 2017

Contributor

@bestander @andreypopp Thanks to this patch bower-away can work: https://twitter.com/bower/status/914828104727703553 :)

Contributor

sheerun commented Oct 2, 2017

@bestander @andreypopp Thanks to this patch bower-away can work: https://twitter.com/bower/status/914828104727703553 :)

@bestander

This comment has been minimized.

Show comment
Hide comment
@bestander

bestander Oct 4, 2017

Member

Thanks for sharing, @sheerun!

Member

bestander commented Oct 4, 2017

Thanks for sharing, @sheerun!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment