Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bugs in migration script #4501

Merged
merged 48 commits into from
Oct 10, 2023
Merged

Fix bugs in migration script #4501

merged 48 commits into from
Oct 10, 2023

Conversation

wwwjn
Copy link
Contributor

@wwwjn wwwjn commented Jul 3, 2023

Reasons for making this change

After testing on dev environment, we find more bugs in migration script.

Related issues

Screenshots

Checklist

  • I've added a screenshot of the changes, if this is a frontend change
  • I've added and/or updated tests, if this is a backend change
  • I've run the pre-commit.sh script
  • I've updated docs, if needed

@wwwjn wwwjn self-assigned this Jul 3, 2023
@wwwjn
Copy link
Contributor Author

wwwjn commented Jul 10, 2023

There is still another thing that needs to be fixed:

If a user first uploads a bundle to Azure Blob storage and changes the database pointing to Azure Blob storage, then the user re-runs the script to delete the original bundle storage, the script can not find the original bundle location. Since we migrate step by step, we need to store state information somewhere.

Possible solution:
When changing the database, write (original_bundle_location, target_bundle_location) to the log file. If the user wants to delete the original bundle, read through the log file and remove the original bundle files.

Working on this.

codalab/migration.py Outdated Show resolved Hide resolved
@epicfaace epicfaace merged commit 73b4d7e into master Oct 10, 2023
56 of 63 checks passed
@epicfaace epicfaace deleted the migration2 branch October 10, 2023 01:56
epicfaace added a commit that referenced this pull request Oct 18, 2023
epicfaace added a commit that referenced this pull request Oct 26, 2023
* Revert "Fix bugs in migration script (#4501)"

This reverts commit 73b4d7e.

* Only keeps migration.py changes.

---------

Co-authored-by: Hidy Han <hidyhan5@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants