Skip to content

[6.x] Minimize memory usage for retrieving product rewrite URLs#180

Merged
royduin merged 3 commits into6.xfrom
feature/minimize-memory
Feb 13, 2026
Merged

[6.x] Minimize memory usage for retrieving product rewrite URLs#180
royduin merged 3 commits into6.xfrom
feature/minimize-memory

Conversation

@Jade-GG
Copy link
Copy Markdown
Collaborator

@Jade-GG Jade-GG commented Jan 21, 2026

In some projects this command ended up breaking as it was trying to get ~23000 products with all the rewrites which took a lot of memory.

In this PR I've attempted to mitigate this by doing smaller queries that each individually retrieve just the relevant entity IDs and then the relevant rewrite URLs. This also somewhat cleans up and speeds up the code as a result.

ETA: Adding this PR as a patch into the project where this issue occurred has fixed it there.

@Jade-GG Jade-GG requested review from BobWez98 and royduin January 21, 2026 10:30
@royduin royduin requested a review from kevinmeijer97 January 23, 2026 08:27
kevinmeijer97
kevinmeijer97 previously approved these changes Jan 23, 2026
BobWez98
BobWez98 previously approved these changes Feb 10, 2026
@indykoning
Copy link
Copy Markdown
Member

Chunking rows to reduce memory usage is exactly what https://laravel.com/docs/12.x/eloquent#chunking-using-lazy-collections is built for.

Perhaps we can use that, your current fix does reduce memory usage. But it doesn't solve the inefficient memory usage problem. With another bulk of products, or lower memory you may run into it again. Doing it lazily will go in chunks of 1000 by default and remove that chunk from memory before moving on to the next 1000

@Jade-GG Jade-GG dismissed stale reviews from BobWez98 and kevinmeijer97 via 4701fdc February 11, 2026 13:32
@royduin
Copy link
Copy Markdown
Member

royduin commented Feb 13, 2026

There is a conflict 😇 and do we need this on other branches?

6.x = Rapidez v4
7.x = Rapidez v5
8.x / master = Statamic 6

@Jade-GG Jade-GG changed the base branch from master to 6.x February 13, 2026 08:39
@Jade-GG Jade-GG changed the title Minimize memory usage for retrieving product rewrite URLs [6.x] Minimize memory usage for retrieving product rewrite URLs Feb 13, 2026
@Jade-GG
Copy link
Copy Markdown
Collaborator Author

Jade-GG commented Feb 13, 2026

There is a conflict 😇 and do we need this on other branches?

6.x = Rapidez v4 7.x = Rapidez v5 8.x / master = Statamic 6

Fixed and yes, also fixed :)

@royduin royduin merged commit 42ded51 into 6.x Feb 13, 2026
@royduin royduin deleted the feature/minimize-memory branch February 13, 2026 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants