New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[#12048] V9 migration verification script optimisation - fetch ReadNotifications for account comparison #12905
Conversation
b519f0c
to
595ce27
Compare
595ce27
to
88a46ff
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this PR is to fetch ReadNotif using join with Account, let's also change the PR title as it is misleading 🙏
CriteriaBuilder cb = HibernateUtil.getCriteriaBuilder(); | ||
CriteriaQuery<teammates.storage.sqlentity.Account> pageQuery = cb.createQuery(sqlEntityClass); | ||
|
||
// sort by createdAt to maintain stable order. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// sort by createdAt to maintain stable order. | |
// sort by id to maintain stable order. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for benchmarking, adding the logging and helping to debug the problem!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thank you!
Part of #12048
Context
When comparing the SQL account to its datastore account for verification, multiple select query for each account was executed to get their readNotifications. This was be because the association between accounts and readNotifications is lazy.
Outline of Solution
Reduce readNotification fetches by fetching the data for a page using a single join, reducing the number of joins. (Multiple joins per account to 1 join for a page of accounts)
~1min2s for 10,000 accounts with 2 readNotifications to ~3 seconds
Fetching