Pure store #524

merged 57 commits into from Jan 8, 2017


None yet

3 participants

zkochan commented Jan 2, 2017 edited

Details here: #519

Additional notes:

  • Life-cycle events related to installation work on the realpath (which does not have the node_modules folder anymore), so packages that have install scripts, are copied instead of symlinked.
  • Bundled dependencies are ignore for now (they are installed as usual dependencies). To support bundled dependencies some operations have to be made less concurrent (which is probably OK).

Should solve: #400, #397, #315

Should allow implementing of: #508 (Offline support)

@zkochan zkochan referenced this pull request in alexanderGugel/ied Jan 2, 2017

Idea: collaboration with pnpm #182

rstacruz commented Jan 4, 2017

Maybe we'll have to make the dependency trees flat as npm and yarn.

Yeah... in hindsight this seems like a good idea now.

@@ -27,21 +27,37 @@ Follow the [pnpm Twitter account](https://twitter.com/pnpmjs) for updates.
## Background
-`pnpm` maintains a flat storage of all your dependencies in `node_modules/.store`. They are then symlinked wherever they're needed.
+`pnpm` maintains a flat storage of all your dependencies in `~/.store`. They are then symlinked wherever they're needed.
rstacruz Jan 4, 2017 Owner

why not ~/.pnpm/store or ~/.pnpm-store?

zkochan Jan 4, 2017 Collaborator

Because hopefully we will use the same store structure as ied. Though maybe .pnpm-store is a good idea till the specs are not finished

@@ -1,2 +1,2 @@
export const GlobalPath = '~/.pnpm'
-export const GlobalStorePath = GlobalPath + '/.store'
+export const GlobalStorePath = '~/.store'
rstacruz Jan 4, 2017 Owner

hmm, I think usual custom conventions are more like GLOBAL_PATH = '~/.pnpm' but thats just a preference for me :)

zkochan commented Jan 4, 2017

Maybe we'll have to make the dependency trees flat as npm and yarn.

Yeah... in hindsight this seems like a good idea now.

I thought there would be problems with long directory names but I came up with this idea of "node_modules/.resolutions" folder which is kinda the old store with the node_modules included. So flat trees can be avoided in this PR.

@zkochan zkochan added the ready for CR label Jan 7, 2017
@zkochan zkochan referenced this pull request in alexanderGugel/ied Jan 8, 2017

Breaking changes in store spec #193

zkochan added some commits Dec 21, 2016
@zkochan @zkochan zkochan feat: drop Node.js <= 6.3 support

drop support of Node.js pre-6.3
@zkochan @zkochan zkochan refactor: remove the redundant subfolder in store ae00bd0
@zkochan @zkochan zkochan refactor: rename linkPeers 8887f8d
@zkochan @zkochan zkochan refactor: remove not used code 0ee6573
@zkochan @zkochan zkochan feat: link each file separately 7af1f93
@zkochan @zkochan zkochan refactor: change store structure 7b07e01
@zkochan @zkochan zkochan refactor: resolve github dependencies to commits 1afead5
@zkochan @zkochan zkochan chore(package.json): pass --preserve-symlinks to ts-node 93b2110
@zkochan @zkochan zkochan test: temporarily skip failing tests b4060e6
@zkochan @zkochan zkochan refactor(git): use more human-friendly Git IDs 197b2dc
@zkochan @zkochan zkochan refactor: resolve 614a663
@zkochan @zkochan zkochan refactor: divide resolution and fetching 799fe1d
@zkochan @zkochan zkochan refactor(fetch): better variable naming e5bae85
@zkochan @zkochan zkochan feat: add shrinkwrap.yaml 3be672c
@zkochan @zkochan zkochan test(shrinkwrap): shrinkwrap locks dependencies 09fe82c
@zkochan @zkochan zkochan style: fix typing issue 8d91735
@zkochan @zkochan zkochan fix(fetch): properly link node_modules on repeat install bf2e992
@zkochan @zkochan zkochan refactor: comment skipped test with single-line comments 24b965b
@zkochan @zkochan zkochan fix: circular symlinks are avoided 4da3469
@zkochan @zkochan zkochan refactor(fetch): remove redundant type properties d68dba8
@zkochan @zkochan zkochan test: additional check for circular symlinks 924ca8c
@zkochan @zkochan zkochan feat: saving dependency graph in node_modules instead of in store

Dependency graph moved out from store
@zkochan @zkochan zkochan feat: add store.yaml with info about dependent projects d32f1fc
@zkochan @zkochan zkochan refactor: remove name from FetchedPackage 714141f
@zkochan @zkochan zkochan refactor: change the location of packages from npm in store

Structure of store changed
@zkochan @zkochan zkochan refactor: change global store path

Move the store path out of ~/.pnpm
@zkochan @zkochan zkochan refactor: remove justFetched from FetchedPackage 119341c
@zkochan @zkochan zkochan refactor(install): use functional programming to filter deps 193a6eb
@zkochan @zkochan zkochan refactor(install): remove redundant node_modules creation 8dcd49b
@zkochan @zkochan zkochan fix(install): installation of concurent circular dependencies 7c3d6a2
@zkochan @zkochan zkochan feat: copy not link some packages
Packages that have install lifecycle events are copied not linked.
@zkochan @zkochan zkochan fix(store.yaml): don't duplicate records 9ec5cb5
@zkochan @zkochan zkochan perf: resolve a package only once per project 4737fda
@zkochan @zkochan zkochan test: concurrent installation of the same dependency e9d7238
@zkochan @zkochan zkochan fix: packages are not removed in the middle of fetch e8d04cc
@zkochan @zkochan zkochan fix: dependencies are linked only once 8b73918
@zkochan zkochan fix: make linking work on Windows ed2d72b
@zkochan zkochan fix: installing local dependencies on Windows c6ddda1
@zkochan zkochan fix: don't reinstall dependencies of the same package b8747f5
@zkochan @zkochan zkochan fix(bin): the run function always returns a Promise 041adae
@zkochan @zkochan zkochan fix: use hard links to link files to the .resolutions folder a5658f4
@zkochan zkochan fix: peers are linked into the correct location de657e6
@zkochan @zkochan zkochan fix: bundled dependencies are not reinstalled 90121ab
@zkochan @zkochan zkochan refactor: remove unsymlink
It is not used anymore
@zkochan @zkochan zkochan chore(CI): test on Node.js v4 as well 96ac7fd
@zkochan @zkochan zkochan refactor: use UPPER_CASE for constants 4bfe497
@zkochan @zkochan zkochan chore(package): return support of Node.js v4 edc1e57
@zkochan @zkochan zkochan fix: make pnpm Node.js 4 compatible again 6e8b133
@zkochan @zkochan zkochan test: don't run tsnode with --preserve-symlinks
--preserve-symlinks makes tests fail on Node.js 4
@zkochan @zkochan zkochan fix(bin): don't use --preserve-symlinks on Node pre-6.3 a6ba83d
@zkochan @zkochan zkochan test: use global stores in tests b92feee
@zkochan @zkochan zkochan fix: flat-tree installation should not work on Node 4 437cf8a
@zkochan @zkochan zkochan fix: show warning
When trying to install into a node_modules created by older pnpm
@zkochan @zkochan zkochan refactor: change the global store location
From `~/.store` to `~/.pnpm-store`
@zkochan @zkochan zkochan docs: note that some of the info is out of date
@zkochan zkochan test: fix typing error
@zkochan zkochan test: fix frequently failing test
@zkochan zkochan merged commit 38837b1 into master Jan 8, 2017

4 checks passed

continuous-integration/appveyor/branch AppVeyor build succeeded
continuous-integration/appveyor/pr AppVeyor build succeeded
continuous-integration/travis-ci/pr The Travis CI build passed
continuous-integration/travis-ci/push The Travis CI build passed
@zkochan zkochan deleted the pure-store branch Jan 8, 2017

oh, shi~ thats a huge pr. good job, @zkochan!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment