Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FW][FIX] *: Compute max/min on huge arrays #3952

Closed
wants to merge 1 commit into from

Conversation

fw-bot
Copy link
Collaborator

@fw-bot fw-bot commented Apr 1, 2024

This is an extension of PR #1217. Navigators (or rather javascript engines) have a limit to the number of function arguments 1.

We sometimes have to use Math.max and Math.min functions (which are subject to this limitation) on every cell/column/row which means that we are functionally limited.

E.g. If we take the limitation of Chrome, we could only process 117k rows, which is not such a huge number if we consider the situation of a user importing their data.

This revision introduces wrappers largeMax and largeMin which take an 1-D array (a single argument) and find the maximum (resp. minimum) present in that list. For arrays with les than a 100K cells, the wrapper will still call Math.max as its implementation is most of the time faster.

Task: 3802691

Description:

description of this task, what is implemented and why it is implemented that way.

Task: : TASK_ID

review checklist

  • feature is organized in plugin, or UI components
  • support of duplicate sheet (deep copy)
  • in model/core: ranges are Range object, and can be adapted (adaptRanges)
  • in model/UI: ranges are strings (to show the user)
  • undo-able commands (uses this.history.update)
  • multiuser-able commands (has inverse commands and transformations where needed)
  • new/updated/removed commands are documented
  • exportable in excel
  • translations (_t("qmsdf %s", abc))
  • unit tested
  • clean commented code
  • track breaking changes
  • doc is rebuild (npm run doc)
  • status is correct in Odoo

Forward-Port-Of: #3951
Forward-Port-Of: #3907

Footnotes

  1. Experimentally, that limit is 117k on Google Chrome and 501k on Firefox.

@robodoo
Copy link
Collaborator

robodoo commented Apr 1, 2024

@fw-bot
Copy link
Collaborator Author

fw-bot commented Apr 1, 2024

@rrahir @LucasLefevre cherrypicking of pull request #3907 failed.

stdout:

Auto-merging src/actions/menu_items_actions.ts
Auto-merging src/helpers/chart_date.ts
CONFLICT (content): Merge conflict in src/helpers/chart_date.ts
Auto-merging src/helpers/figures/charts/pie_chart.ts
CONFLICT (content): Merge conflict in src/helpers/figures/charts/pie_chart.ts
Auto-merging src/helpers/misc.ts
CONFLICT (content): Merge conflict in src/helpers/misc.ts
Auto-merging src/plugins/core/header_visibility.ts
Auto-merging src/plugins/core/range.ts
Auto-merging src/plugins/core/sheet.ts
CONFLICT (content): Merge conflict in src/plugins/core/sheet.ts
Auto-merging src/plugins/ui_core_views/evaluation_conditional_format.ts
CONFLICT (content): Merge conflict in src/plugins/ui_core_views/evaluation_conditional_format.ts
Auto-merging src/plugins/ui_feature/renderer.ts
Auto-merging src/plugins/ui_feature/ui_sheet.ts
Auto-merging src/xlsx/functions/charts.ts

stderr:

20:00:31.537378 git.c:463               trace: built-in: git cherry-pick 6e49e438ace52c780b798374445dc96aa3097ffc
error: could not apply 6e49e438a... [FIX] *: Compute max/min on huge arrays
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
----------
status:

Either perform the forward-port manually (and push to this branch, proceeding as usual) or close this PR (maybe?).

In the former case, you may want to edit this PR message as well.

⚠️ after resolving this conflict, you will need to merge it via @robodoo.

More info at https://github.com/odoo/odoo/wiki/Mergebot#forward-port

This is an extension of PR #1217. Navigators (or rather javascript
engines) have a limit to the number of function arguments [1].

We sometimes have to use Math.max and Math.min functions (which are
subject to this limitation) on every cell/column/row which means that
we are functionally limited.

E.g. If we take the limitation of Chrome, we could only process 117k
rows, which is not such a huge number if we consider the situation of a
user importing their data.

This revision introduces wrappers `largeMax` and `largeMin` which take
an  1-D array (a single argument) and find the maximum (resp. minimum)
present in that list. For arrays with les than a 100K cells, the wrapper
will still call Math.max as its implementation is most of the time
faster.

[1]: Experimentally, that limit is 117k on Google Chrome and 501k on
Firefox.

Task: 3802691
X-original-commit: c2a7220
@rrahir rrahir force-pushed the 17.0-15.0-fix-array-size-rar-d1x0-fw branch from 2edbbfe to 4a761e5 Compare April 1, 2024 18:13
@rrahir
Copy link
Collaborator

rrahir commented Apr 1, 2024

@robodoo r+

@robodoo robodoo closed this in c2f6da2 Apr 1, 2024
@fw-bot fw-bot deleted the 17.0-15.0-fix-array-size-rar-d1x0-fw branch April 15, 2024 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants