New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
oss: migrate to ossfs #6307
oss: migrate to ossfs #6307
Conversation
fix 6177
pass all of the tests
1. Experiments shows resumable get/put has little influnence on performance. 2. Just use resumable ones as default
1. Bump ossfs from 2021.6.2 to 2021.7.0 2. add some comments to the verification trigger
Because some problem with oss-emulater 1. bump to ossfs 2021.7.1 2. rewrite upload_fobj
Co-authored-by: Ruslan Kuprieiev <kupruser@gmail.com>
Co-authored-by: Ruslan Kuprieiev <kupruser@gmail.com>
Now that we are using the real OSS, do we need this? dvc/tests/func/test_data_cloud.py Lines 35 to 41 in f45311e
|
@isidentical Great catch, indeed we don't need it anymore. We can safely remove xfail now. |
Because we didn't use emulator, we can remove some pytest XFAILs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thank you @karajan1001 ! π For the record: now waiting for conda package conda-forge/staged-recipes#15612 |
Fix #6177, migrate to ossfs
β I have followed the Contributing to DVC checklist.
π If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.
Thank you for the contribution - we'll try to review it as soon as possible. π
come from #6178
Performance summary:
The results from the current version compare to reuse of sessions optimization.
The results from the current version compare to reuse of sessions optimization.