-
Notifications
You must be signed in to change notification settings - Fork 551
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve performance of collate & reduce memorty consumption
We used to read all files into memory - which gets too much if someone runs a massively parallel CI because 400 * 10MB is still ~4GB and that's just pure file size and we do a lot more with it. Breaks the interface of ResultMerger.merge_and_store but it's not intended as a public interface. Will leave a note in the Changelog anyhow. What is attempted to do top leve is perhaps easier to see when looking at the spike code: https://github.com/simplecov-ruby/simplecov/compare/collate-plus-plus?expand=1 Changes go further than just not reading all files in at once, during the merge process we also operate on the raw file structure as opposed to creating SimpleCov::Result. Creating SimpleCov::Result comes with a lot of overhead, notably reading in all source files. So, that's even worse doing ~400 times in a large code base. There's more optimization potential for cases like these which I'll open a ticket about but notably: * Potentially don't create SimpleCov::Result at all until we really produce results (just dump the raw coverage more or less) * allow running without a formatter as only the last one really needs the formatter
- Loading branch information
Showing
11 changed files
with
166 additions
and
124 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# some comment | ||
puts "wargh" | ||
puts "wargh 1" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# some comment | ||
puts "wargh" | ||
puts "wargh 2" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# foo | ||
puts "foo" | ||
# bar | ||
puts "bar" |
Oops, something went wrong.