Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable cleanup thread for cloud object store in set metadata #13102

Merged
merged 2 commits into from Dec 24, 2021
Merged

Disable cleanup thread for cloud object store in set metadata #13102

merged 2 commits into from Dec 24, 2021

Conversation

bioflowy
Copy link

When using cloud object store, set_meta.py never ends

  1. use cloud object store
  2. set metadata_strategy: extended in galaxy.yaml
  3. set_meta.py never ends because of cleanup thread

This bug has already been fixed at s3.py.
Applied same fix to cloud.py.
Added a test to verify that enable_cache_monitor is false in config_to_dict return value .

Cannot download from cloud storage to cache dir

  1. Use cloud object store
  2. Delete object_store_cache content
  3. When galaxy tries to download a file, An Exception will be thrown.

Changed open mode from "w+" to "wb+".
I cannot write a unit test for this change because it require s3.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:

License

@github-actions github-actions bot added this to the 22.01 milestone Dec 23, 2021
@bioflowy bioflowy force-pushed the disable_cleanup_thread_in_set_metadata branch from 51ddf92 to 19dcf22 Compare December 23, 2021 01:12
@KeizoUehara
Disable cleanup thread of cloud objectstore in set_metadata.py
992f18e 
Apply the following fixes for s3.py to cloud.py
Be more explcit about starting cache monitor thread 494c15c
Disable cleanup thread in set_metadata.py 6a56f3a
Fix S3 object store shutdown 7a63fd1
@KeizoUehara
open cache file with binary mode
009760b
@bioflowy bioflowy force-pushed the disable_cleanup_thread_in_set_metadata branch from 19dcf22 to 009760b Compare December 23, 2021 01:35
mvdbeek
mvdbeek approved these changes Dec 24, 2021
View reviewed changes
Copy link
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@mvdbeek mvdbeek merged commit aeddd9c into galaxyproject:dev Dec 24, 2021
@github-actions
Copy link

This PR was merged without a "kind/" label, please correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mvdbeek mvdbeek mvdbeek approved these changes

Assignees
No one assigned
Labels
area/objectstore area/testing kind/bug
Projects
None yet
Milestone
22.01
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@bioflowy @mvdbeek @nsoranzo @KeizoUehara