Export Nydus images as block devices through NBD #1111
Conversation
Refine v6 related code and add two fields to meta info. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Load the mapped_blkaddr field for each data blob, later it will be used compose a RAFS v6 image into a block device. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Add common code to compose a block device from a RAFS image, which then can used exposed through nbd/ublk/virtio-blk/vhost-user-blk etc. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Compose a block device from a RAFSv6 image, so all metadata/data content can be accessed by block address. The EROFS fs driver can be used to directly mount the block device. It depends on the blob_cache subsystem and can be used to implement nbd/ublk/virtio-blk/vhost-user-blk servers. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Implement NbdService which cooperates with the Linux nbd driver to expose RAFSv6 images as block devices. To simplify the implementation, the NbdService will directly talk with the nbd driver, instead of following a typical nbd-server and nbd-client architecture. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Add subcommand nbd to export nydus images as block devices through NBD. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
jiangliu
requested review from
bergwolf,
luodw and
hsiangkao
and removed request for
a team
|
@jiangliu , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53042 |
|
@jiangliu , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53043 |
|
@jiangliu , The CI test is completed, please check result:
Congratulations, your test job passed! |
|||||||||||||||||||||
|
@jiangliu , The CI test is completed, please check result:
Congratulations, your test job passed! |
|||||||||||||||||||||
|
@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53179 |
|
@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53401 |
|
@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53406 |
|
Hi @jiangliu, are there docs / tests that can be given? |
Add method to load BlobCacheConfigV2 from configuration file. Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
|
@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53436 |
In order to support mounting multi-blobs container image as a single block device, add flattened block device feature for EROFS. In this mode, all meta/data contents will be mapped into one block address. User could compose a block device(by nbd/ublk/virtio-blk/ vhost-user-blk) from multiple sources and mount the block device by EROFS directly. It can reduce the number of block devices used, and it's also benefits in both VM file passthrough and distributed storage scenarios. You can test this using the method mentioned by: dragonflyoss/nydus#1111 1. Compose a (nbd)block device from multi-blobs. 2. Mount EROFS on mntdir/. 3. Compare the md5sum between source dir and mntdir/. Later, we could also use it to refer original tar blobs. Signed-off-by: Jia Zhu <zhujia.zj@bytedance.com> Signed-off-by: Xin Yin <yinxin.x@bytedance.com>
In order to support mounting multi-blobs container image as a single block device, add flattened block device feature for EROFS. In this mode, all meta/data contents will be mapped into one block address. User could compose a block device(by nbd/ublk/virtio-blk/ vhost-user-blk) from multiple sources and mount the block device by EROFS directly. It can reduce the number of block devices used, and it's also benefits in both VM file passthrough and distributed storage scenarios. You can test this using the method mentioned by: dragonflyoss/nydus#1111 1. Compose a (nbd)block device from multi-blobs. 2. Mount EROFS on mntdir/. 3. Compare the md5sum between source dir and mntdir/. Later, we could also use it to refer original tar blobs. Signed-off-by: Jia Zhu <zhujia.zj@bytedance.com> Signed-off-by: Xin Yin <yinxin.x@bytedance.com> Reviewed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
|
could we reopen a pr to merge this to main branch now? |
|
substituted by #1139 |
Please follow the step to test
block-nbd: