Skip to content

/ rsbackup

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Concurrent backups of remote hosts #18

Closed
senji-rsbackup opened this issue Aug 31, 2014 · 4 comments
Closed

Concurrent backups of remote hosts #18

senji-rsbackup opened this issue Aug 31, 2014 · 4 comments
Labels
feature performance
Milestone
6.0

Comments

@senji-rsbackup
Copy link

@senji-rsbackup senji-rsbackup commented Aug 31, 2014

When backing up remote hosts it seems that rsbackup is rarely if ever I/O bound on the target device; would it be possible to have a configuration option to run backups on multiple hosts at once?
@ewxrjk
Copy link
Owner

@ewxrjk ewxrjk commented Sep 13, 2015

I've added issue #24 to request concurrent pruning. I'll probably attempt that before concurrent backup, as it's likely to be easier.
@ewxrjk ewxrjk added this to the 4.0 milestone Nov 29, 2015
@ewxrjk ewxrjk modified the milestones: 5.0, 4.0 Mar 25, 2017
@ewxrjk
Copy link
Owner

@ewxrjk ewxrjk commented Apr 14, 2017

https://github.com/ewxrjk/rsbackup/wiki/Concurrency is the design. It's been implemented for pruning but not for backups, which will be a substantial upheaval.
@ewxrjk ewxrjk modified the milestones: 5.0, 6.0 Jan 27, 2018
ewxrjk added a commit that referenced this issue Dec 29, 2018
@ewxrjk
Concurrency group directive
9f9891a 
re #18
ewxrjk added a commit that referenced this issue Dec 29, 2018
@ewxrjk
EventLoop is thread-friendly
222d451 
re #18
ewxrjk added a commit that referenced this issue Dec 29, 2018
@ewxrjk
Perform backups of distinct hosts concurrently
3ba028c 
re #18
ewxrjk added a commit that referenced this issue Dec 29, 2018
@ewxrjk
Avoid concurrent access to a single device
cf14d0b 
re #18
ewxrjk pushed a commit that referenced this issue Jan 14, 2019
@optnfast
Document concurrency rules
Loading status checks…
16d9143 
#18
ewxrjk added a commit that referenced this issue Jan 20, 2019
@ewxrjk
Make EventLoop thread-friendly
46d237d 
re #18
ewxrjk added a commit that referenced this issue Jan 20, 2019
@ewxrjk
Concurrent backup of distinct hosts to distinct devices
Loading status checks…
a716b44 
The current logic is that no resource should be used concurrently with
itself - so at present slow hosts being backed up to a fast device will
still be serialized.

re #18
ewxrjk added a commit that referenced this issue Jan 20, 2019
@ewxrjk
Make EventLoop thread-friendly
8a218c2 
re #18
ewxrjk added a commit that referenced this issue Jan 20, 2019
@ewxrjk
Concurrent backup of distinct hosts to distinct devices
Loading status checks…
5eefcb5 
The current logic is that no resource should be used concurrently with
itself - so at present slow hosts being backed up to a fast device will
still be serialized.

re #18
ewxrjk added a commit that referenced this issue Jan 20, 2019
@ewxrjk
Make EventLoop thread-friendly
9ca44e0 
re #18
ewxrjk added a commit that referenced this issue Jan 20, 2019
@ewxrjk
Concurrent backup of distinct hosts to distinct devices
Loading status checks…
1561eb9 
The current logic is that no resource should be used concurrently with
itself - so at present slow hosts being backed up to a fast device will
still be serialized.

re #18
@ewxrjk
Copy link
Owner

@ewxrjk ewxrjk commented Jan 20, 2019 •
edited

master now supports concurrent backups of distinct hosts to distinct devices - in other words, no concurrent use of any single resource.

Concurrent use of devices and/or hosts could potentially be added, if that's desirable. The former might produce poor locality of reference on backup devices?

#17 would need to be addressed before supporting concurrent use of volumes.
ewxrjk added a commit that referenced this issue Feb 5, 2019
@ewxrjk
Do host availability checks concurrently
6557bf0 
re #18
@ewxrjk ewxrjk mentioned this issue Mar 3, 2019
Open
@ewxrjk
Copy link
Owner

@ewxrjk ewxrjk commented Mar 3, 2019

I'm going to reject idea of concurrency 'within' devices and have created #58 for concurrency with hosts, meaning that this issue is now closed.
@ewxrjk ewxrjk closed this Mar 3, 2019
ewxrjk added a commit that referenced this issue Sep 2, 2019
@ewxrjk
Do host availability checks concurrently
05e93f3 
re #18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature performance
Projects
None yet
Milestone
6.0
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@ewxrjk @senji-rsbackup