create_close accept multiple paths for durability improvement #39
Conversation
Add new optional create_close request arguments, i.e., devid_X, path_X and dev_count, so that tracker can confirm that enough copies of file are written before ack succession. This way prevents the file loss risk that the host/disk which hold the first copy may broken before replication is done.
MotivationCurrent serving file procedure just writes a copy, and do replication to fit the class setting later. Since the host/disk which hold the first copy may broken before replication is done, this strategy actually trades durability for performance. (It was indeed happened in my production environment, and the issue may observable if the environment has enough capacity of write operations and disks.) Although it's a reasonable design in ten years ago. Since the high performance hardware is available today, the design of almost all of morden DFSs such as openstack swift, ceph, glusterfs, microsoft azure storage, linkedin's Ambry, Dropbox's magic pocket...et al., pays more attention to durability. These DFSs will try to write enough copies of file before ack the serving file procedure. It will be cool to let mogilefs make use of modern hardware, and has the same ability of durability as other modern DFSs. ProposalThis PR wants to give the choices to user. User who focus on durability can have the same level of durability as other modern DFSs, and still keeping mogilefs's simple and easy to use. User who focus on performance can still use mogielfs as before. The PR is simple, it just adds new create_close request arguments, i.e., devid_X, path_X and dev_count, so now create_close can accept multiple paths and devids.
|
Add new optional create_close request arguments, i.e., devid_X, path_X and dev_count, so that tracker can confirm that enough copies of file are written before ack succession.
This way prevents the file loss risk that the host/disk which hold the first copy may broken before replication is done.