feat: Reconcile Payments in background #34596
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ruthra-kumar
force-pushed
the
auto_reconcile_using_fifo
branch
2 times, most recently
from
8972938
to
d7fc1c4
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## develop #34596 +/- ##
===========================================
- Coverage 63.87% 63.68% -0.19%
===========================================
Files 810 813 +3
Lines 59750 60010 +260
===========================================
+ Hits 38166 38220 +54
- Misses 21584 21790 +206
|
deepeshgarg007
requested changes
Mar 28, 2023
erpnext/accounts/doctype/accounts_settings/accounts_settings.json
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/auto_reconcile_log/auto_reconcile_log.js
Outdated
Show resolved
Hide resolved
ruthra-kumar
changed the title
ruthra-kumar
force-pushed
the
auto_reconcile_using_fifo
branch
from
32bb366
to
e7cb620
Compare
deepeshgarg007
requested changes
Apr 9, 2023
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation_list.js
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
erpnext/accounts/doctype/process_payment_reconciliation/process_payment_reconciliation.py
Outdated
Show resolved
Hide resolved
This was referenced Apr 19, 2023
1. Split reconcile function into 2 2. While reconciling against payment entry, skip a set_missing_ref_details setter method
ruthra-kumar
force-pushed
the
auto_reconcile_using_fifo
branch
from
556aff1
to
dfecf49
Compare
1. Moved the status section to the bottom in parent doc 2. Using alerts to indicate Job trigger status
ruthra-kumar
force-pushed
the
auto_reconcile_using_fifo
branch
from
1b50f57
to
f99d479
Compare
mergify bot
pushed a commit
that referenced
this pull request
Apr 22, 2023
* feat: auto reconcile in background * chore: Option to enable auto reconciliation in settings * refactor: validate if feature is enabled in settings * refactor: check for running job while using reconciliation tool * chore: using doc to get filter values * chore: use frappe.db.get_value in validations * chore: cleanup commented out code * chore: replace get_list with get_all * chore: use block scope variable * chore: type information for functions * refactor: flag to ignore job validation check * refactor: update parent doc status if all reconciled * chore: create test_records file * test: create a bunch of vouchers for testing auto reconcile * chore: renamed auto_reconcile to process_payment_reconciliation * chore: another child doctype to hold payments * chore: remove duplicate field * chore: add fetched payments to log * chore: Popup comment message update * chore: replace get_all with get_value * chore: replace label in settings page * chore: remove unit test and records * refactor: status in reconciliation log * refactor: set status in log as well * chore: fix field name * chore: change triggered job name * chore: use status field in list view of log * chore: status while there are no allocations * refactor: split trigger function into two * chore: adding cancelled status * refactor: function trigger queued docs * chore: cron job scheduled * chore: fixing accouts settings json file * chore: typos and variable scope * chore: use 'pluck' in db call * chore: remove redundant whitelist decorator * chore: use single DB call to fetch values * chore: replace get_all with get_value * refactor: use raw db calls to fetch reconciliation log records Using get_doc on `Process Payment Reconciliation Log` is costly when handling large volumes of invoices. Use raw frappe.db.get_all to selectively pull status and reconciled count * chore: update status on successful batch operation * chore: make payment table readonly * chore: ability to pause the background job * chore: remove isolate_each_allocation * chore: more description in progress bar * refactor: partially working state * refactor: update reconcile flag and setting hard limits for fetching * chore: make allocation editable -- NEED TO REVERT * chore: pause button * refactor: skip setter function in Payment Entry for better performan * refactor: split reconcile function and skip a setter function 1. Split reconcile function into 2 2. While reconciling against payment entry, skip a set_missing_ref_details setter method * chore: increase payment limit * refactor: replace frappe.db.get_all with frappe.db.get_value * chore: remove unwanted doctypes * refactor: make allocation table readonly * perf: update ref_details only for newly linked invoices * chore: rename skip flag * refactor(UI): receivable_payable field should auto populate * refactor: no control statements in finally block * chore: cleanup section and rename checkbox * chore: update new fieldname in code * chore: update error msg * refactor: start and pause integrated into status pause checkbox has been removed * refactor: added cancelled status to the log doctype 1. Moved the status section to the bottom in parent doc 2. Using alerts to indicate Job trigger status (cherry picked from commit ed14d1c) # Conflicts: # erpnext/accounts/doctype/accounts_settings/accounts_settings.json
ruthra-kumar
added a commit
that referenced
this pull request
Apr 24, 2023
* feat: auto reconcile in background * chore: Option to enable auto reconciliation in settings * refactor: validate if feature is enabled in settings * refactor: check for running job while using reconciliation tool * chore: using doc to get filter values * chore: use frappe.db.get_value in validations * chore: cleanup commented out code * chore: replace get_list with get_all * chore: use block scope variable * chore: type information for functions * refactor: flag to ignore job validation check * refactor: update parent doc status if all reconciled * chore: create test_records file * test: create a bunch of vouchers for testing auto reconcile * chore: renamed auto_reconcile to process_payment_reconciliation * chore: another child doctype to hold payments * chore: remove duplicate field * chore: add fetched payments to log * chore: Popup comment message update * chore: replace get_all with get_value * chore: replace label in settings page * chore: remove unit test and records * refactor: status in reconciliation log * refactor: set status in log as well * chore: fix field name * chore: change triggered job name * chore: use status field in list view of log * chore: status while there are no allocations * refactor: split trigger function into two * chore: adding cancelled status * refactor: function trigger queued docs * chore: cron job scheduled * chore: fixing accouts settings json file * chore: typos and variable scope * chore: use 'pluck' in db call * chore: remove redundant whitelist decorator * chore: use single DB call to fetch values * chore: replace get_all with get_value * refactor: use raw db calls to fetch reconciliation log records Using get_doc on `Process Payment Reconciliation Log` is costly when handling large volumes of invoices. Use raw frappe.db.get_all to selectively pull status and reconciled count * chore: update status on successful batch operation * chore: make payment table readonly * chore: ability to pause the background job * chore: remove isolate_each_allocation * chore: more description in progress bar * refactor: partially working state * refactor: update reconcile flag and setting hard limits for fetching * chore: make allocation editable -- NEED TO REVERT * chore: pause button * refactor: skip setter function in Payment Entry for better performan * refactor: split reconcile function and skip a setter function 1. Split reconcile function into 2 2. While reconciling against payment entry, skip a set_missing_ref_details setter method * chore: increase payment limit * refactor: replace frappe.db.get_all with frappe.db.get_value * chore: remove unwanted doctypes * refactor: make allocation table readonly * perf: update ref_details only for newly linked invoices * chore: rename skip flag * refactor(UI): receivable_payable field should auto populate * refactor: no control statements in finally block * chore: cleanup section and rename checkbox * chore: update new fieldname in code * chore: update error msg * refactor: start and pause integrated into status pause checkbox has been removed * refactor: added cancelled status to the log doctype 1. Moved the status section to the bottom in parent doc 2. Using alerts to indicate Job trigger status (cherry picked from commit ed14d1c)
deepeshgarg007
pushed a commit
that referenced
this pull request
Apr 25, 2023
feat: Reconcile Payments in background (#34596) * feat: auto reconcile in background * chore: Option to enable auto reconciliation in settings * refactor: validate if feature is enabled in settings * refactor: check for running job while using reconciliation tool * chore: using doc to get filter values * chore: use frappe.db.get_value in validations * chore: cleanup commented out code * chore: replace get_list with get_all * chore: use block scope variable * chore: type information for functions * refactor: flag to ignore job validation check * refactor: update parent doc status if all reconciled * chore: create test_records file * test: create a bunch of vouchers for testing auto reconcile * chore: renamed auto_reconcile to process_payment_reconciliation * chore: another child doctype to hold payments * chore: remove duplicate field * chore: add fetched payments to log * chore: Popup comment message update * chore: replace get_all with get_value * chore: replace label in settings page * chore: remove unit test and records * refactor: status in reconciliation log * refactor: set status in log as well * chore: fix field name * chore: change triggered job name * chore: use status field in list view of log * chore: status while there are no allocations * refactor: split trigger function into two * chore: adding cancelled status * refactor: function trigger queued docs * chore: cron job scheduled * chore: fixing accouts settings json file * chore: typos and variable scope * chore: use 'pluck' in db call * chore: remove redundant whitelist decorator * chore: use single DB call to fetch values * chore: replace get_all with get_value * refactor: use raw db calls to fetch reconciliation log records Using get_doc on `Process Payment Reconciliation Log` is costly when handling large volumes of invoices. Use raw frappe.db.get_all to selectively pull status and reconciled count * chore: update status on successful batch operation * chore: make payment table readonly * chore: ability to pause the background job * chore: remove isolate_each_allocation * chore: more description in progress bar * refactor: partially working state * refactor: update reconcile flag and setting hard limits for fetching * chore: make allocation editable -- NEED TO REVERT * chore: pause button * refactor: skip setter function in Payment Entry for better performan * refactor: split reconcile function and skip a setter function 1. Split reconcile function into 2 2. While reconciling against payment entry, skip a set_missing_ref_details setter method * chore: increase payment limit * refactor: replace frappe.db.get_all with frappe.db.get_value * chore: remove unwanted doctypes * refactor: make allocation table readonly * perf: update ref_details only for newly linked invoices * chore: rename skip flag * refactor(UI): receivable_payable field should auto populate * refactor: no control statements in finally block * chore: cleanup section and rename checkbox * chore: update new fieldname in code * chore: update error msg * refactor: start and pause integrated into status pause checkbox has been removed * refactor: added cancelled status to the log doctype 1. Moved the status section to the bottom in parent doc 2. Using alerts to indicate Job trigger status (cherry picked from commit ed14d1c) Co-authored-by: ruthra kumar <ruthra@erpnext.com>
frappe-pr-bot
pushed a commit
that referenced
this pull request
Apr 25, 2023
# [14.23.0](v14.22.3...v14.23.0) (2023-04-25) ### Bug Fixes * `PermissionError` in Work Order ([a3568c1](a3568c1)) * Add company field to lower deduction certificate ([#34914](#34914)) ([d732083](d732083)) * add item-code filter for SCR supplied-items batch-no ([7c4f83e](7c4f83e)) * add limit for get_next_stock_reco ([#34937](#34937)) ([7131ff2](7131ff2)) * added validation for extra job card ([a5fde5d](a5fde5d)) * batch qty conversion factor issue fixed in pos transaction ([#34917](#34917)) ([56ef0ba](56ef0ba)) * broken 'set exchange gain/loss' btn in payment entry ([#34940](#34940)) ([00968ba](00968ba)) * Bulk Payment Entry from PO/SO ([#34942](#34942)) ([878d747](878d747)) * click handler should not attempt indexed access of empty array ([#35013](#35013)) ([06f204a](06f204a)) * Common party JV cost center ([#35008](#35008)) ([693007a](693007a)) * duplicate reposting entries of same voucher ([aeac43c](aeac43c)) * FEC report for France accountancy ([#34781](#34781)) ([6aabab2](6aabab2)) * filtering via batch no ([#34951](#34951)) ([3b23fc1](3b23fc1)), closes [#34950](#34950) * Incorrect difference value in Stock and Account Value Comparison report ([e7ca833](e7ca833)) * incorrect OR condition causing timeout error ([c020789](c020789)) * internal Purchase Receipt GL Entries ([f7e436f](f7e436f)) * item not showing in the BOM ([28cd79a](28cd79a)) * pass reference_doctype in link queries (backport [#35038](#35038)) ([#35039](#35039)) ([5cc3978](5cc3978)) * Payment entry with TDS in bank reco statement ([#34961](#34961)) ([f43ea0d](f43ea0d)) * per_billed condition for Payment Entry ([#34969](#34969)) ([f9f42c7](f9f42c7)) * process_loss_percentage in BOM ([4418862](4418862)) * removed depends on for the Employee Detail section ([9d17d3f](9d17d3f)) * respect title_field from doctype to bulk transactions ([#34928](#34928)) ([29aa4a0](29aa4a0)) * set `frappe.flags.company` to call regional code accurately ([789dfd6](789dfd6)) * simplify `erpnext.get_region` ([87595bd](87595bd)) * SLA permissions (backport [#34981](#34981)) ([#34986](#34986)) ([83a1b83](83a1b83)) * stock entry type issue ([33a1608](33a1608)) * **test:** `test_backdated_stock_reco_cancellation_future_negative_stock` ([2ad157b](2ad157b)) * **test:** `test_internal_pr_gl_entries` ([7740ceb](7740ceb)) * Unable to allocate advance against invoice ([#35007](#35007)) ([5045ad6](5045ad6)) * use `functools.wraps` to preserve doc signature ([2ec18eb](2ec18eb)) * use CombineDatetime instead of Timestamp in QB queries ([55da91c](55da91c)) * use filter_by_finance_book instead of only_depreciable_assets in fixed asset register (backport [#35031](#35031)) ([#35035](#35035)) ([5630e81](5630e81)) * Use set instead of db_set as it is called from validate ([#34967](#34967)) ([6650212](6650212)) * v14, Bank Reconcile Tools not cover case JV debit bank ([#35000](#35000)) ([82d8379](82d8379)) * validation for internal transfer entry ([a582354](a582354)) * value of depreciable assets not updating after manual depr entry [v14] ([#35010](#35010)) ([3c75e55](3c75e55)) * wrong qty of remaining work orders to be created when using "Create" > "Work Order" ([#34726](#34726)) ([47df41f](47df41f)) ### Features * Reconcile Payments in background ([#34596](#34596)) ([5923a80](5923a80)) ### Performance Improvements * Journal Entries (backport [#34918](#34918)) ([#35054](#35054)) ([39b5147](39b5147)) ### Reverts * Revert "fix: Rate from LDC in TDS reports (#33699)" ([f44a79f](f44a79f)), closes [#33699](#33699)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For extremely large volumes of Invoices and Payments, Payment Reconciliation tool may timeout. This feature allows for running the same reconciliation tool in the background.
Documentation can be found here
Changes:
Process Payment ReconciliationandProcess Payment Reconciliation Loghave been introduced. These reuse the existing reconciliation logic.Process Payment Reconciliationwill have similar filters as Payment Reconciliation. Once submitted, 'Reconcile in Background' button will be availabe. Once triggered,Process Payment Reconciliation Logwill be created. Payments are allocated in FIFO order.Accounts Settingsto enable/disable this feature