Making background download block size configurable.

[shuaichang]

What is the version of your Accelerated Container Image

0.6.7

What would you like to be added?

Making overlaybd background downloading request size (currently hardcoded to 256KB) to configurable.

Why is this needed for Accelerated Container Image?

It seems the background download with a fixed 256KB per request https://github.com/containerd/overlaybd/blob/279a7b3dac5c4105780cdce33218d2a359e6e5f4/src/bk_download.cpp#LL188C12-L188C14, which generates 10X number of requests compared to the requests from on-demand load (3-4X more data to download and the ondemand download is 1MB also resulted in less requests)

This behavior causes 2 major issues: a) cost increase to cloud object storage b) more likely to be throttled by cloud provider object storage.

Is it possible to make the request size of background downloading tunnable? We want to make it something like 10+MB per request to reduce QPS.

Thanks!

Are you willing to submit PRs to contribute to this feature?

• Yes, I am willing to implement it.

[yuchen0cc]

Add this feature in containerd/overlaybd#222

• Add a configuration downlaod.blockSize for request block size;
• Download configs can be seen in overlaybd.log with info background download is enabled;
• Audits for background download can be seen in overlaybd-audit.log with label bk_download;

[liulanzheng]

containerd/overlaybd#222

[shuaichang]

Thanks folks for the quick reply!