Skip to content

Commit

Permalink
fix: incorrect status on data import (backport #25660) (#25703)
Browse files Browse the repository at this point in the history
* fix: incorrect status on data import

- Used to show the status as "Not Started" if a prior import session was partially successful
- `data_import_list.js` needs some rethinking just displaying the past few logs isn't super helpful
  the import logs aren't individually identifiable on that screen
- could start deleting the logs of a previously failed imported record, if it's successful on a subsequent run
  or atleast flag it to not display it

Refer to internal ticket 9486 and 12166 for further information

(cherry picked from commit f75dd9d)

* fix: Correct logic for comlete success

(cherry picked from commit f221201)

* fix(importer): set `Pending` status as a fallback

Will match scenarios like success == failed == 0

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 2710c28)

* fix: erase logs in case of complete failure

(cherry picked from commit 9f7c385)

---------

Co-authored-by: Arjun Choudhary <contact@arjunchoudhary.com>
Co-authored-by: Ankush Menat <ankush@frappe.io>
Co-authored-by: Akhil Narang <me@akhilnarang.dev>
  • Loading branch information
4 people committed Mar 28, 2024
1 parent 3ddf1f6 commit 0165c75
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 10 deletions.
4 changes: 0 additions & 4 deletions frappe/core/doctype/data_import/data_import_list.js
Expand Up @@ -27,10 +27,6 @@ frappe.listview_settings["Data Import"] = {
if (imports_in_progress.includes(doc.name)) {
status = "In Progress";
}
if (status === "Pending") {
status = "Not Started";
}

return [__(status), colors[status], "status,=," + doc.status];
},
formatters: {
Expand Down
24 changes: 18 additions & 6 deletions frappe/core/doctype/data_import/importer.py
Expand Up @@ -102,9 +102,13 @@ def import_data(self):
log_index = 0

# Do not remove rows in case of retry after an error or pending data import
if self.data_import.status == "Partial Success" and len(import_log) >= self.data_import.payload_count:
if (
self.data_import.status in ("Partial Success", "Error")
and len(import_log) >= self.data_import.payload_count
):
# remove previous failures from import log only in case of retry after partial success
import_log = [log for log in import_log if log.get("success")]
frappe.db.delete("Data Import Log", {"success": 0, "data_import": self.data_import.name})

# get successfully imported rows
imported_rows = []
Expand Down Expand Up @@ -213,13 +217,21 @@ def import_data(self):
)

# set status
failures = [log for log in import_log if not log.get("success")]
if len(failures) == total_payload_count:
status = "Pending"
elif len(failures) > 0:
successes = []
failures = []
for log in import_log:
if log.get("success"):
successes.append(log)
else:
failures.append(log)
if len(failures) >= total_payload_count and len(successes) == 0:
status = "Error"
elif len(failures) > 0 and len(successes) > 0:
status = "Partial Success"
else:
elif len(successes) == total_payload_count:
status = "Success"
else:
status = "Pending"

if self.console:
self.print_import_log(import_log)
Expand Down

0 comments on commit 0165c75

Please sign in to comment.