[IMP] account: dashboard performance

[william-andre]

As the number of account.move, account.move.line,
account.bank.statement.line and account.journal is growing, the
accounting dashboard, which is the entry point of the app, gets slower
and slower.

There are multiple issues being addressed in this commit:

• The data for each journal is computed journal by journal. This means
  that the number of queries run increases linearly with the number of
  journals. While the boilerplate around running multiple queries is
  negligible compared to the running time of the queries in this case,
  some queries take as much time to run for one journal or for many.
  To improve this, all the queries are now batched. This has been done
  by refactoring the code; all these functions are now called on as many
  records as needed¹:
  • _get_journal_bank_account_balance
  • _get_journal_outstanding_payments_account_balance
  • _get_last_bank_statement
  • get_line_graph_datas
  • get_bar_graph_datas
  • get_journal_dashboard_datas
• The gap detection and the entries' count are computed fields
  (has_sequence_holes and entries_count respectively). We don't need
  to display/compute these fields for all types of journals, but since
  they were mentioned by using a <field/> node in the view, they were
  computed for all journals displayed. Instead of using the <field/>
  node, we are now setting the value in the kanban_dashboard field.
• Documents in foreign currencies on journals in foreign currencies need
  to get the rate in order to be aggregated in the journal's currency.
  There are 3 cases:
  • Document in journal's currency
  • Company's currency is the same as the journal's
  • Document, company and journal have 3 different currencies
    Before this commit, the second case will still fetch the daily rate
    for the document in order to do the conversion, but we actually
    already know the conversion; it is stored on the document.

Benchmark

On a populate database with:

• 4 res.company (with accounting enabled)
• 45 account.journal
• 19k account.move
• 140k account.move.line
• 4k account.bank.statement.line
• 4 account.bank.statement

	Query count	Query time	Remaining time
Before fix	279	0.333s	0.375s
After fix (without update)	40	0.120s	0.170s
After fix (with update2)	38	0.100s	0.170s

Note that the currency conversion was disabled because the populate
database doesn't represent a realistic dataset regarding this. Disabling
it only improves the numbers before the fix.

Note

A lot of the time remaining comes from the aggregation of
draft/unpaid invoices with the correct rate done in python instead of in
SQL. This commit doesn't change the behavior but this could be rethought
from a function point of view.

Footnotes

1. the old functions have been kept for compatibility, new ones are
   suffixed by _batched and made private if it wasn't the case. ↩

2. some optimization require the views and indexes to be updated ↩

[robodoo]

[qdp-odoo]

It's a bit confused and I messed up. Or the contrary.

Multi currencies use case to check and discuss ☝️

[qdp-odoo]

I believe that journal.currency_id doesn't have any sense here, although it appears (I check in Odoo 16) you can effectively set a secondary currency on a sale/purchase journal. 🙃
I don't think it has any interest, except maybe to default the secondary currency of invoices created in that journal, and reporting in this very dashboard... I saw it was like that before, but we should probably consider simplifying that, what do you think?

the proposed change doesn't alter the current behavior, so it's not a blocking point

[robodoo]

@william-andre @qdp-odoo linked pull request(s) odoo/enterprise#33165 not ready. Linked PRs are not staged until all of them are ready.