VDisc is a tool for creating and mounting virtual CD-ROM images backed by object storage.
If you wish to work on VDisc itself or any of its libraries, you'll first need Bazel installed on your machine (version 1.0.0+ is required).
To build the vdisc CLI, you'll need to clone this git repository and build the vdisc command with Bazel.
$ git clone https://github.com/NVIDIA/vdisc.git
$ cd vdisc
$ bazel build --stamp //cmd/vdisc
Burning a vdisc using the CLI is simple. First you need to generate a manifest of your objects, where they should appear in the disc image, and their size.
$ cat << EOF > mnist.csv
"/train-images-idx3-ubyte.gz","https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz",9912422
"/train-labels-idx1-ubyte.gz","https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz",28881
"/t10k-images-idx3-ubyte.gz","https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz",1648877
"/t10k-labels-idx1-ubyte.gz","https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz",4542
EOF
$ vdisc burn -i mnist.csv -o mnist.vdsc
Once you've burned a vdisc, you can mount it
$ mkdir mnist
$ vdisc mount --url=mnist.vdsc --mountpoint=mnist
1.570744835687179e+09 info maxprocs/maxprocs.go:47 maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined
1.570744835729569e+09 info isofuse/isofuse.go:68 mounted iso {"mountpoint": "/home/joeuser/mnist"}
And in another terminal you can examine the files
sh
$ md5 mnist/*
MD5 (mnist/t10k-images-idx3-ubyte.gz) = 074392edd37ac2bd4904c0df2a31c38e
MD5 (mnist/t10k-labels-idx1-ubyte.gz) = dbb5c5e00e8b64dfe161442e122f1c8b
MD5 (mnist/train-images-idx3-ubyte.gz) = b1c2f15e5ea102012fa9da59cd0d6d7c
MD5 (mnist/train-labels-idx1-ubyte.gz) = e538dc41040b558f796a632c4604bbeb
By default, vdisc mount uses fuse, but on linux you can TCMU by specifying --mode=tcmu
.
To learn more about how vdisc works, read through the detailed design.