Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue] Greatly improve performance by caching subsequent glob calls #34025

Closed
6 tasks done
m2-assistant bot opened this issue Sep 8, 2021 · 10 comments · Fixed by #33998
Closed
6 tasks done

[Issue] Greatly improve performance by caching subsequent glob calls #34025

m2-assistant bot opened this issue Sep 8, 2021 · 10 comments · Fixed by #33998
Assignees
Labels
Area: Framework Component: Filesystem Fixed in 2.4.x The issue has been fixed in 2.4-develop branch Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Priority: P1 Once P0 defects have been fixed, a defect having this priority is the next candidate for fixing. Progress: done Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch Severity: S1 Affects critical data or functionality and forces users to employ a workaround.

Comments

@m2-assistant
Copy link

m2-assistant bot commented Sep 8, 2021

This issue is automatically created based on existing pull request: #33998: Greatly improve performance by caching subsequent glob calls


Description (*)

This pull request introduces an in-memory cache for glob system calls. On systems where file I/O is slow (for example Docker for Mac), glob system calls can be quite slow. This pull requests caches the same glob calls and returns a cached result if possible. In my testing this reduced the glob calls on the category listings from ~8K to ~1.6K.

Blackfire comparison:

Screenshot 2021-09-06 at 09 51 21

Related Pull Requests

None

Fixed Issues (if relevant)

None

  1. Fixes magento/magento2#<issue_number>

Manual testing scenarios (*)

  1. Profile any page before this diff is applied
  2. Apply diff and reprofile
  3. See a reduction in glob() calls

Questions or comments

Contribution checklist (*)

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
  • All automated tests passed successfully (all builds are green)

Resolved issues:

  1. resolves [Issue] Greatly improve performance by caching subsequent glob calls #34024: Greatly improve performance by caching subsequent glob calls
@m2-assistant m2-assistant bot added Component: Filesystem Priority: P1 Once P0 defects have been fixed, a defect having this priority is the next candidate for fixing. Severity: S1 Affects critical data or functionality and forces users to employ a workaround. labels Sep 8, 2021
@m2-community-project m2-community-project bot added this to Pull Request In Progress in High Priority Backlog Sep 8, 2021
@engcom-Hotel engcom-Hotel added Area: Framework Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed labels Sep 8, 2021
@engcom-Hotel
Copy link
Contributor

Added appropriate labels to create a JIRA ticket.

@github-jira-sync-bot
Copy link

✅ Jira issue https://jira.corp.magento.com/browse/AC-1179 is successfully created for this GitHub issue.

@m2-assistant
Copy link
Author

m2-assistant bot commented Sep 8, 2021

✅ Confirmed by @engcom-Hotel. Thank you for verifying the issue.
Issue Available: @engcom-Hotel, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

@engcom-Hotel
Copy link
Contributor

@magento give me 2.4-develop instance

@magento-deployment-service
Copy link

Hi @engcom-Hotel. Thank you for your request. I'm working on Magento instance for you.

@magento-deployment-service
Copy link

Hi @engcom-Hotel, unfortunately there is no ability to deploy Magento instance at the moment. Please try again later.

@TheRealHunzik
Copy link

@magento give me 2.4-develop instance

@magento-deployment-service
Copy link

Hi @TheRealHunzik. Thank you for your request. I'm working on Magento instance for you.

@magento-deployment-service
Copy link

Hi @TheRealHunzik, unfortunately there is no ability to deploy Magento instance at the moment. Please try again later.

@sidolov
Copy link
Contributor

sidolov commented Nov 21, 2021

Hi @m2-assistant[bot]. Thank you for your report.
The issue has been fixed in #33998 by @thlassche in 2.4-develop branch
Related commit(s):

The fix will be available with the upcoming release.

@sidolov sidolov added the Fixed in 2.4.x The issue has been fixed in 2.4-develop branch label Nov 21, 2021
@sidolov sidolov closed this as completed Nov 21, 2021
@m2-community-project m2-community-project bot moved this from Pull Request In Progress to Done in High Priority Backlog Nov 21, 2021
@m2-community-project m2-community-project bot moved this from Done to Pull Request In Progress in High Priority Backlog Nov 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Framework Component: Filesystem Fixed in 2.4.x The issue has been fixed in 2.4-develop branch Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Priority: P1 Once P0 defects have been fixed, a defect having this priority is the next candidate for fixing. Progress: done Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch Severity: S1 Affects critical data or functionality and forces users to employ a workaround.
Projects
High Priority Backlog
  
Pull Request In Progress
Development

Successfully merging a pull request may close this issue.

5 participants