New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paket Install and Large Sub-directories #1782

Closed
nickbabcock opened this Issue Jul 1, 2016 · 6 comments

Comments

Projects
None yet
4 participants
@nickbabcock
Contributor

nickbabcock commented Jul 1, 2016

Description

paket.exe install does not finish when invoked in a directories with large sub-directories

Repro steps

cd /
sudo touch paket.dependencies
sudo /paket.exe install

Known workarounds

cd into /tmp and everything works fine 馃槢

Related information

  • Operating system: 14.04.4 LTS, Trusty Tahr (but should effect all systems)
  • Branch: Paket 3.4.0
  • Mono Version: 4.4.1
@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki Jul 2, 2016

Member

Making this up-for-grabs since I think it will be probably easy to fix if one has a repro.

Member

forki commented Jul 2, 2016

Making this up-for-grabs since I think it will be probably easy to fix if one has a repro.

@forki forki added the up-for-grabs label Jul 2, 2016

@nickbabcock

This comment has been minimized.

Show comment
Hide comment
@nickbabcock

nickbabcock Jul 8, 2016

Contributor

The issue is ProjectFile.FindAllProjects (profiled through mono). It recursively searches all directories, which is not good when the directories are huge (and potentially irrelevant to the install).

Are the project files that make up a solution not known through some other file?

I'll edit the title and description to be a little less sensationalist 馃槈

Contributor

nickbabcock commented Jul 8, 2016

The issue is ProjectFile.FindAllProjects (profiled through mono). It recursively searches all directories, which is not good when the directories are huge (and potentially irrelevant to the install).

Are the project files that make up a solution not known through some other file?

I'll edit the title and description to be a little less sensationalist 馃槈

@nickbabcock nickbabcock changed the title from Paket Install Infinite Loop in Root Directory to Paket Install and Large Sub-directories Jul 8, 2016

@baronfel

This comment has been minimized.

Show comment
Hide comment
@baronfel

baronfel Jul 8, 2016

Contributor

It's important to note that paket doesn't require, and in fact never assumes, the presence of a solution file. It's entirely fixated around projects. Now, that said, it would be possible augment the existing FindAllProjects code to have an optional hot-path if it finds a .sln file to read that file and take the project list from there, but continue to support the search-based method.

Contributor

baronfel commented Jul 8, 2016

It's important to note that paket doesn't require, and in fact never assumes, the presence of a solution file. It's entirely fixated around projects. Now, that said, it would be possible augment the existing FindAllProjects code to have an optional hot-path if it finds a .sln file to read that file and take the project list from there, but continue to support the search-based method.

@nickbabcock

This comment has been minimized.

Show comment
Hide comment
@nickbabcock

nickbabcock Jul 9, 2016

Contributor

A hot path is fine, but the worst case is still pretty bad. I have a node.js project with a node_modules directory of >6k sub-directories, >60k files and paket install won't return after a minute. This could put a limitation on .net/node.js cross applications without a re-org of the project structure.

Also consider the use case when there are no projects (Paket from F# interactive). How could we implement a hot path?

Contributor

nickbabcock commented Jul 9, 2016

A hot path is fine, but the worst case is still pretty bad. I have a node.js project with a node_modules directory of >6k sub-directories, >60k files and paket install won't return after a minute. This could put a limitation on .net/node.js cross applications without a re-org of the project structure.

Also consider the use case when there are no projects (Paket from F# interactive). How could we implement a hot path?

@cdrnet

This comment has been minimized.

Show comment
Hide comment
@cdrnet

cdrnet Jul 9, 2016

Member

Note that the presence of a solution file does not guarantee that it contains a complete list of all projects. I typically have multiple solution files, each of them covering a subset of projects only.

Member

cdrnet commented Jul 9, 2016

Note that the presence of a solution file does not guarantee that it contains a complete list of all projects. I typically have multiple solution files, each of them covering a subset of projects only.

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki Jul 11, 2016

Member

Mhm we probably can black list parts of the tree. Like we don't need to look into packages folder. And we probably don't need to look into node folders.

PRs welcome

Member

forki commented Jul 11, 2016

Mhm we probably can black list parts of the tree. Like we don't need to look into packages folder. And we probably don't need to look into node folders.

PRs welcome

@forki forki closed this in c79c9ce Jul 14, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment