Move filesystem creation to docker host and add fstype option #597.

[brunotm]

Volumes are formatted as ext4 filesystem, but we mount them as ext2 unless when mocking the ESX server. There is a reason ?

[kerneltime]

The tool is suffixed with ext4 but the fs is ext2
We plan to support other fs types if requested by users.
Do you have a specific need for a FS type and which one? Do let us know.
To double check the FS type, you can run a container image using the volume and on a different console session run the file command

root@photon-VYb05tsNa [ ~ ]# docker volume create --driver=vmdk --name=MyVolume -o size=10gb
MyVolume
root@photon-VYb05tsNa [ ~ ]# docker run -it --rm -v MyVolume:/vol busybox
/ # 

And then run the following command

root@photon-VYb05tsNa [ ~ ]# mount| grep MyVolume
/dev/sdb on /mnt/vmdk/MyVolume type ext2 (rw,relatime,block_validity,barrier,user_xattr,acl)
root@photon-VYb05tsNa [ ~ ]#  file -sL /dev/sdb
/dev/sdb: Linux rev 1.0 ext2 filesystem data (mounted or unclean), UUID=59819b66-7b3b-428a-8b4c-bcc78df08216, volume name "MyVolume" (large files)

[kerneltime]

Changing the title

[brunotm]

No specific needs.

But I think that ext4 would be a reasonable default due to data integrity and performance improvements.

[brunotm]

I've done a small patch #597 changing the default filesystem to ext4, and also noticed that the default mkfs command included is somewhat outdated.

However, handling filesystem creation in the plugin on the docker host seems a more reasonable approach since wouldn't be necessary to maintain filesystem specific tools over the ESX server, and in the future filesystem choice could be determined by the available drivers and tools on the docker host.

What do you think?

[kerneltime]

Taking a look.

[brunotm]

It looks that would be safe to handle the formatting in plugin/Create() (create->attach->mkfs->detach), avoiding nasty verifications or unsafe conditions in plugin/Mount(). And implement Mkfs in utils/fs.

Determining which mkfs.* to call could be done with a fstype option in Request.

[kerneltime]

Reassigning to Mark to look into supporting ext4

[msterin]

@ brunotm - volumes are currently formatted and mounted as ext2. Still - you are correct, plugin/Create() is the right place for the mkfs, and utils/fs is the right place for mkfs wrapper. There is a few historical reasons why we (deliberately) did not do mkfs on docker host - e.g. slow attach/detach on ESXi 6.0 prior to Update2, the need to avoid create() protocol complication and error handling. As of now, these reasons are not very important anymore and it all boils down to priorities - we just did not hear a request for ext4 or other FS yet and other work is on the way, so we did not change it yet. Agreed on the rest.

[brunotm]

@msterin

I have done a patch to move up the filesystem creation to the volume-plugin. If you can please review f315a34a0fcf036acd2b2ac8f97fe5a23c3a74b3

It adds a fstype option, and stores the fstype in the KV for hinting further mounts.

A few things i dislike/would like an advice about:

• security: i currently evaluate the existence of /sbin/mks.FSTYPE, this can potentially be used for privilege escalation/arbitrary code execution if someone manages to write to /sbin/ in the docker host.
• option defaults and validation: while before every option was treated in vmdkops-service, i had to introduce a check in plugin.Create() for the fstype option.
• early check for mkfs existence in plugin.Create(): to avoid having to rollback the volume creation when the host does not have the specified mkfs command
• Getting into Go as i did it: may not be very well crafted.

[msterin]

Resolved by #597 merge