[path_list.rb] Improve performance of read_file_system #5927

Merged
merged 3 commits into from Sep 23, 2016

Projects

None yet

4 participants

@hborders
  • Used a simple String.slice to create relative paths instead of path.relative_path_from(root). Since we already know that all files must exist under root (because we globbed absolute_paths from root), we can just remove the first root.to_s.length + 1 characters from an absolute_path to get a relative_path.
  • Partition dirs and files first since we already get an array of absolute paths back from Pathname.glob, removing unnecessary string concat: File.directory?(root + path)
  • Used lazy arrays to reduce array iteration.

I profiled my code before the change.
I profiled my new code again, and read_file_system practically disappears.

previous:

real     0m59.095s
user     0m46.133s
sys      0m2.360s

now:

real     0m23.712s
user     0m14.492s
sys      0m2.030s
Heath Borders [path_list.rb] Improve performance of read_file_system
* Used a simple `String.slice` to create relative paths instead of `path.relative_path_from(root)`. Since we already know that all files must exist under `root` (because we globbed `absolute_paths` from `root`), we can just remove the first `root.to_s.length + 1` characters from an `absolute_path` to get a `relative_path`.
* Partition `dirs` and `files` first since we already get an array of absolute paths back from `Pathname.glob`, removing unnecessary string concat: `File.directory?(root + path)`
* Used `lazy` arrays to reduce array iteration.

I [profiled my code before the change](https://github.com/CocoaPods/CocoaPods/files/477973/call-stack.zip).
I [profiled my new code again](https://github.com/CocoaPods/CocoaPods/files/478512/call-stack2.html.zip), and `read_file_system` practically disappears.

previous:

```
real     0m59.095s
user     0m46.133s
sys      0m2.360s
```

now:

```
real     0m23.712s
user     0m14.492s
sys      0m2.030s
```
ad94f54
@hborders

This fixes #5890

@segiddins

Other than the one nit, this looks pretty good to me

lib/cocoapods/sandbox/path_list.rb
+ dirs, files = dirs_and_files.partition { |path| File.directory?(path) }
+
+ root_length = root.cleanpath.to_s.length + File::SEPARATOR.length
+ relative_sorted = lambda do |paths|
@segiddins
segiddins Sep 22, 2016 Member

sort_relative_paths?

@hborders
hborders Sep 22, 2016

I want to make it clear that the lambda accepts full paths, but returns relative paths. sort_relative_paths sounds like it accepts relative paths. Maybe sorted_relative_paths_from_full_paths ?

@segiddins
segiddins Sep 22, 2016 Member

๐Ÿ‘

@hborders

Are these failures normal? I don't know what to do to correct them.

@segiddins
Member

Unfortunately, they are :(

@hborders

Do I need to do anything else before this can be merged?

(I'm not trying to pressure you to merge me sooner. I've never contributed to this project before, so I don't know what to expect.)

@DanToml
Member
DanToml commented Sep 22, 2016

@hborders hey, sorry have been super busy this week. This LGTM. I just restarted the system Ruby job on Travis, but will test manually tomorrow if it remains flaky.

@hborders

No worries. I'm just super excited for my first contribution. I'll wait patiently from now on. :)

@DanToml
Member
DanToml commented Sep 22, 2016

@hborders Thanks for the contribution ๐Ÿ’œ - I've wanted to spend some time digging into this for a while.

System Ruby job appears to have passed, running the others.

@DanToml

LGTM with โœ… CI

@dnkoutso
Contributor

@hborders nicely done! This along a couple of other fixes we've done will make pod install super fast.

@DanToml
Member
DanToml commented Sep 23, 2016

The last matrix failure is transitive ๐Ÿ‘๐Ÿ‘

@dnkoutso
Contributor

1.1.0 is shaping up to be a great performance release.

@DanToml
Member
DanToml commented Sep 23, 2016

:shipit: - thanks @hborders

@DanToml DanToml merged commit fffa804 into CocoaPods:master Sep 23, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment