[FW] 16.0 perf 3588401 lul #3209
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@LucasLefevre @rrahir cherrypicking of pull request #3147 failed. stderr: Either perform the forward-port manually (and push to this branch, proceeding as usual) or close this PR (maybe?). In the former case, you may want to edit this PR message as well. More info at https://github.com/odoo/odoo/wiki/Mergebot#forward-port |
This commit caches string normalization used in lookup functions. `normalizeString` is called for every element of lookup ranges. Those ranges can be large. In the production spreadsheet mentionned on the ticket, it was taking more than 13% of the loading time. Now it's 0.0% `normalizeString` was called 19M times while it has only 664 different input values. opw-3588401 Task: 3588401
With this commit, the range arrays are created with their final size. On the ticket spreadsheet, `range` self-time goes from ~21.4% to ~17.4% opw-3588401
When a function references a range `=SUM(A1:B10)`, a new 2d array is rebuilt with the cell values for every range in every functions. However, for LOOKUP functions, it's very likely to have many cells with a LOOKUP function looking for a match in the exact same range. In the customer spreadsheet, `range` self-time goes from ~15% to almost nothing (0.2%). There's another spreadsheet where many cells references the same range with 100k cells. Loading the spreadsheet was taking 26s before and 7s after. opw-3588401 Task: 3588401
LucasLefevre
force-pushed
the
saas-16.1-16.0-perf-3588401-lul-K1Kb-fw
branch
from
d154353
to
00e88e9
Compare
|
fw-bot r+ |
|
@LucasLefevre I can only do this on unmodified forward-port PRs, ask robodoo. |
|
robodoo r+ |
robodoo
pushed a commit
that referenced
this pull request
Nov 17, 2023
This commit caches string normalization used in lookup functions. `normalizeString` is called for every element of lookup ranges. Those ranges can be large. In the production spreadsheet mentionned on the ticket, it was taking more than 13% of the loading time. Now it's 0.0% `normalizeString` was called 19M times while it has only 664 different input values. opw-3588401 Task: 3588401 Part-of: #3209
robodoo
pushed a commit
that referenced
this pull request
Nov 17, 2023
With this commit, the range arrays are created with their final size. On the ticket spreadsheet, `range` self-time goes from ~21.4% to ~17.4% opw-3588401 Part-of: #3209
robodoo
pushed a commit
that referenced
this pull request
Nov 17, 2023
When a function references a range `=SUM(A1:B10)`, a new 2d array is rebuilt with the cell values for every range in every functions. However, for LOOKUP functions, it's very likely to have many cells with a LOOKUP function looking for a match in the exact same range. In the customer spreadsheet, `range` self-time goes from ~15% to almost nothing (0.2%). There's another spreadsheet where many cells references the same range with 100k cells. Loading the spreadsheet was taking 26s before and 7s after. opw-3588401 closes #3209 Task: 3588401 Signed-off-by: Rémi Rahir (rar) <rar@odoo.com> Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
description of this task, what is implemented and why it is implemented that way.
Task: : 3588401
review checklist
Forward-Port-Of: #3147