Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show 100% when import done in info/dashboard. #4181

Merged
merged 8 commits into from May 7, 2024
Merged

Conversation

dafeder
Copy link
Member

@dafeder dafeder commented May 6, 2024

In the output from dataset-info, and by extension in the datastore import dashboard, we have a longstanding issue where when using the mysql import module, the percent done stays at zero (0) even though the import process has finished. This is because the mysql importer performs the entire import at once and provides no feedback on amount complete. It was, in hindsight, not such a good idea to assume any job like this would provide "bites processed" information and build the whole dataset info gathering workflow around that assumption.

While the dataset info and datastore dashboard are probably both due for an overhaul, for now we just want coherent information provided to the user. We had two options:

  1. Remove the percent done measure completely for import, as we did for filefetcher, or
  2. Force the value to be 100 if the status is "done".

Since the percent done is still useful for people using the CSV parser, and the status value seems to be more reliable than the bytes processed value, I went with #2. This is definitely to be considered a stopgap.

QA Steps

  1. Install new vanilla site, enable datastore_mysql_import
  2. Import sample content
  3. Visit datastore import dashboard. Confirm that one any imports that are "done," it reads "100%" not "0%".

@dafeder dafeder marked this pull request as ready for review May 6, 2024 20:52
@janette janette merged commit 6ee0767 into 2.x May 7, 2024
10 checks passed
@janette janette deleted the 16199-percent-done branch May 7, 2024 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants