-
Notifications
You must be signed in to change notification settings - Fork 464
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of DB Metadata impl #22779
Comments
Link to subfeature: https://dotcms.productboard.com/feature-board/planning/features/14928643 |
Waiting for Fabro's QA |
PR is #22834 |
I performed a load test using the tool called "siege" The test environment was seeded with 5300 separate file assets the results can be seen here https://docs.google.com/spreadsheets/d/1Qr0EByhNI_RUmtmhd8D6_oXRB0B24dQFyr8zaXq9Ako/edit#gid=0 apparently now it is cheaper to copy the previously generated metadata from the database using the intermediate files than generating them directly from felix and then putting them as files and then also in the cache. |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
* #22779 adding first changes * #22779 the file can holds a binary buffer or a file * #22779 adding more changes * #22779 just adding a comment for future * #22779 adding a normalization of the file repo manager, now it is grouping on directories * #22779 minor changes to handle the hashes on folders * #22779 adding more unit test * #22779 adding more unit test * #22779 adding a fix for an unit test * #22779 adding more doc --------- Co-authored-by: fabrizzio-dotCMS <fabrizzio@dotCMS.com>
When rolling out the metadata changes, we provided two impls, one a fs based impl and one a db impl. The DB impl was intended to be our default impl because it helped lower our dependency on the NFS share but we found that in actuality, it caused a bottleneck because it had to do a DB lookup, then write a temp file before it could return the metadata.
We should improve/tune up this impl so that it can become our default impl and just return the metadata using a single db query without having to write a tmp file.
The text was updated successfully, but these errors were encountered: