[Backup] Limit bandwith utilisation of a backup run

[ghost]

It would be very useful if it was possible to set a limit on the transfer speeds for specific backup schedules. The limit could also be scheduled so that during "hot times" it could be a low limit and high or no limit on "cold" times.

Some examples:

• remote outside company premises and a limited bandwidth on the external line. Limiting in XO would avoid saturating the internet connection
• Disk I/O. Limiting backup would make less strain on I/O limited SR's and remotes.
• Network performance. During certain times a limitation on transfer speeds could improve network performance for other applications or office users.

[lancefogle]

@julien-f @olivierlambert I would like to second this request. Any thoughts on a timeline for this issue? I was going to open a new issue but found this one from a year ago.

I have just 5 VMs in one location that are running nightly Continuous Replication jobs over a 100/100 mbit connection and it saturates it completely making nothing else work (i.e. VPN, Website access, etc).

[julien-f]

Shouldn't be difficult to implement, I'm adding this to the June release roadmap.

[lancefogle]

Awesome! Thank you. It would be a great help.

[antonlindgren]

Any updates on this issue? I really need this because my backups are throttling my line since I use a remote site for backups.

[olivierlambert]

Hi!

No update because it's not a priority right now (improving backup perfs is a priority, but by doing that we might be able to do the opposite too, as requested here). But as usual, contributions are welcome 👍 Also, a support ticket might help to raise the priority.

Until then, take a look at iptables + tc combo to do some network limitations (example here)

[nikade87]

Any updates?
We have a remote site where the connection is shared with others, so it would be great if we could make sure not to use 100% during backups.

[olivierlambert]

@julien-f maybe it's a simple modification for @fbeauchamp to implement this summer?

[julien-f]

I don't see any easy way to implement this.

The more I think about it, the more I believe the best answer is at OS level.

[fbeauchamp]

maybe we can change _outputStream to read chunk manually instead of using pipeline and make a pause if the speed is greater than the max allowed speed.
It won't be exact given the various buffering, but it should respect the order of magnitude. It will probably have a performance cost .
@nikade87 what are the order of magnitude of the limit you want to set : is it in KBps, Mbps , Gbps, 10Gbps , more ?

[olivierlambert]

Also what's the restriction perimeter? Overall backup speed per XOA/Proxy or per backup job? Or per disk?

[nikade87]

For me the limit would be specified in Mbit/s if possible.

[olivierlambert]

And by what? (cf my previous comment)

[nikade87]

In our case we would need to set this at the job, we have a specific job backing up all the VM's on this offsite xcp-ng server.

[fbeauchamp]

the easiest to implement will be by backup transfer. then by backup job execution.
By remote from the same xo / proxy is harder but not off limit
By remote from any combination of proxy/xo/xoa will take much more time

[nikade87]

Yeah that sounds about right, that should do it. Then we'll be able to set like 500Mbit limit on that job and we'll make sure the others still have half the bandwidth available.

[lancefogle]

This needs to be per job for the most part. The reasoning is that when a backup job runs it doesn't eat all available bandwidth. Limiting per disk isn't really helpful as you could be transfering multiple disks. Same for VMs as multiples vould be transferred thus eating more than the requested bamdwidth. If it is limited per job, bandwidth can be predictably controlled while transferring the data over the network during business operations.

…

On Wed, Jun 15, 2022, 4:41 AM Olivier Lambert ***@***.***> wrote: Also what's the restriction perimeter? Overall backup speed per XOA/Proxy or per backup job? Or per disk? — Reply to this email directly, view it on GitHub <#4119 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFVAGWZQEWTEBSQK6GW6NFTVPGJMDANCNFSM4HDQXYQA> . You are receiving this because you commented.Message ID: ***@***.***>

[julien-f]

Ok, we will work on implementing a transfer speed limitation inside a backup run.

All other use cases (global limit, global limit for a specific remote, global limit for a specific pool) will not be implemented, and if needed, should be looked at the OS level.

[nikade87]

That sounds perfect, thanks guys!

[Fohdeesha]

Bumping this as another customer has requested it.

[olivierlambert]

@julien-f why it's reopened?