Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fs: task/rsync: Parallel rsync stress test #28472

Closed
wants to merge 4 commits into from
Closed

fs: task/rsync: Parallel rsync stress test #28472

wants to merge 4 commits into from

Conversation

sidharthanup
Copy link

@sidharthanup sidharthanup commented Jun 10, 2019 •
edited

An rsync task that runs parallel to the workunit task that rsync's the contents of the workunit directory( also snapshot of workunit dir ) to the mount directory.

Fixes http://tracker.ceph.com/issues/17434
Signed-off-by: Sidharth Anupkrishnan sanupkri@redhat.com

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

@sidharthanup sidharthanup added the cephfs Ceph File System label Jun 10, 2019
@sidharthanup sidharthanup force-pushed the wip-task-background-rsync branch 8 times, most recently from f2b8827 to 3d83d4f Compare June 12, 2019 16:49
@batrick batrick self-requested a review June 12, 2019 17:00
@sidharthanup sidharthanup force-pushed the wip-task-background-rsync branch 5 times, most recently from 4b759f6 to 9ca01d8 Compare June 13, 2019 13:16
@batrick
Copy link
Member

batrick commented Jun 14, 2019

Please amend the commits like ff7f339 to create the files in the correct location so the rename is the subsequent commits is not longer necessary. These commits were originally for the defunct ceph-qa-suite repo.

@batrick
Copy link
Member

batrick commented Jun 14, 2019

35f830 needs commit message prefix.

@sidharthanup sidharthanup force-pushed the wip-task-background-rsync branch 11 times, most recently from 1bb02fc to fe53764 Compare July 9, 2019 07:59
@github-actions github-actions bot added the tests label Dec 9, 2020
@sidharthanup
Copy link
Author

jenkins test make check

@sidharthanup sidharthanup force-pushed the wip-task-background-rsync branch 5 times, most recently from 84fa25d to ec63309 Compare December 22, 2020 19:48
@batrick
Copy link
Member

batrick commented Dec 23, 2020 

flake8 run-test: commands[0] | flake8 --select=F,E9 --exclude=venv,.tox
./tasks/cephfs/filesystem.py:583:9: F841 local variable 'snap_shot' is assigned to but never used
./tasks/cephfs/mount.py:661:9: F841 local variable 'snap_shot' is assigned to but never used
ERROR: InvocationError for command /home/jenkins-build/build/workspace/ceph-pull-requests/qa/.tox/flake8/bin/flake8 --select=F,E9 --exclude=venv,.tox (exited with code 1)

@sidharthanup sidharthanup force-pushed the wip-task-background-rsync branch 3 times, most recently from 78ce672 to de140ab Compare January 6, 2021 05:55
@sidharthanup
Copy link
Author

https://pulpito.ceph.com/sidharthanup-2021-01-06_05:55:52-fs-master-distro-default-smithi/

@batrick

@github-actions
Copy link

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

ramkiperiy and others added 4 commits February 18, 2021 19:35
@ramkiperiy @sidharthanup
qa/tasks/rsync: This is new code in task
ffa4541 
Code to rsync data from workunit, snap in parallel, it checks event state from
workunit task.  Task can be used parallel in with workunit and other cephfs
tasks

Signed-off-by: Ramakrishnan Periyasamy <rperiyas@redhat.com>
@sidharthanup
qa/tasks/workunit: add 'observer' functionality
2e2225e 
This is to enable tasks like the forthcoming 'rsync' task to cooperate with
workunit during setup and teardown.

Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
@sidharthanup
qa: add rsync tasks
4861387 
Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
@sidharthanup
Fixed exception handling for rsync and improved cleanup
0dc1bc2 
- Refactored the code for all worknunits/non-all workunits execution
- Handle exception raised by rsync(files vanished and cannot be transferred) during removal of kernel build
- Cleanup of rsync dir when the workunit task is finished

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
@sidharthanup
Copy link
Author

@batrick PTAL

@github-actions
Copy link

github-actions bot commented Jul 2, 2021

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@stale
Copy link

stale bot commented Jan 9, 2022

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@stale stale bot added the stale label Jan 9, 2022
@github-actions
Copy link

github-actions bot commented Jul 7, 2022

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

@github-actions github-actions bot closed this Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@batrick batrick batrick requested changes

@joscollin joscollin joscollin requested changes

Assignees

@sidharthanup sidharthanup

Labels
cephfs Ceph File System needs-rebase stale tests
Projects
None yet
Milestone
No milestone
4 participants
@sidharthanup @batrick @joscollin @ramkiperiy