Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance: Prevent sorting of files in PhpFilesFinder #4164

Merged
merged 2 commits into from
Jun 11, 2023

Conversation

staabm
Copy link
Contributor

@staabm staabm commented Jun 10, 2023

From my understanding the order in which PhpFilesFinder gets its files from sfFinder is not relevant for the final result.

with this PR, I get 25% faster rector run with a 7% safe in memory

grafik

->in($directories);

if ($sortByName) {
$finder->sortByName();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before this PR we sorted everything everytime. this sorting is slow and should only be used when necessary

@TomasVotruba
Copy link
Member

The order is releveant to reproducible run. Windows/*nix/Mac sorts files in a different way. First discovered classes are available to PHPStan reflection.

Yet, I think we can avoid the sorting to get better performance 👍

Btw, can you try to drop the sorting completely? If CI passing well, we should go for it

@staabm
Copy link
Contributor Author

staabm commented Jun 10, 2023

Btw, can you try to drop the sorting completely? If CI passing well, we should go for it

disabling it makes e2e test fail. it seems the change of this PR does not influence the final result order

Yet, I think we can avoid the sorting to get better performance

maybe we could sort the results when rendering the diff, instead of reading the files sorted?
(separate PR)

@staabm staabm changed the title Prevent sorting of files in PhpFilesFinder Performance: Prevent sorting of files in PhpFilesFinder Jun 10, 2023
@TomasVotruba
Copy link
Member

TomasVotruba commented Jun 10, 2023

maybe we could sort the results when rendering the diff, instead of reading the files sorted?
(separate PR)

Sounds good 👍 Feel free to add it to this PR instead, so we have it all finished on merge

@samsonasik
Copy link
Member

@staabm the commit with double quote will broke the scoped build, I think we need to fix that eventually.

@rez1dent3 since you had initialized to use github output at PR:

Could you help verify how escaped get commit message on scoped build for us? The issue is the build will broke if the commit has double quote, eg: Revert "some commit" Thank you 😊

@staabm staabm force-pushed the no-sort branch 2 times, most recently from d4951cf to 4652c4e Compare June 11, 2023 07:00
@staabm
Copy link
Contributor Author

staabm commented Jun 11, 2023

Sounds good 👍 Feel free to add it to this PR instead, so we have it all finished on merge

I had a look into the possible perf impact. removing the sorting alltogehter brings only a negligible perf benefit.
Also the PR as is, is pretty simple and brings already a huge boost.

Could you help verify how escaped get commit message on scoped build for us? The issue is the build will broke if the commit has double quote, eg: Revert "some commit" Thank you 😊

I know we had the same problem in phpstan-src. let me have a look

@staabm
Copy link
Contributor Author

staabm commented Jun 11, 2023

I know we had the same problem in phpstan-src. let me have a look

in phpstan-src this seems to work.. it looks pretty similar to what we have in rector-src though

https://github.com/phpstan/phpstan-src/blob/613c57c3d2301402ca474338959ad8e3bd204000/.github/workflows/phar.yml#L180-L186

@staabm
Copy link
Contributor Author

staabm commented Jun 11, 2023

Fix in #4175

@samsonasik
Copy link
Member

@staabm that doesn't work, I reverted at #4176

@staabm
Copy link
Contributor Author

staabm commented Jun 11, 2023

This PR should be good to go, see #4164 (comment)

@samsonasik
Copy link
Member

Thank you @staabm , let's give it a try 👍

@samsonasik samsonasik merged commit f7b43d4 into rectorphp:main Jun 11, 2023
@staabm staabm deleted the no-sort branch June 11, 2023 09:38
@rez1dent3
Copy link
Contributor

@samsonasik Hello. Got it? If not, then I can look on Wednesday.

@staabm
Copy link
Contributor Author

staabm commented Jun 12, 2023

It was fixed with #4178

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants