Skip to content

Implementation of an asynchronous file system for file deletion and encryption.

Notifications You must be signed in to change notification settings

raomanus/Asynchronous-File-System

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

The following README outlines the implementation details of the project. 
* Brief Overview:
	The following features have been successfully implemented in the code.

	1. Pass the special flags to the clone system call.
	2. Move files deleted by special processes to the trash folder.
	3. Perform encryption, compression synchronously on small files.
	4. Perform encryption, compression asynchronously on large files.
	5. Support /proc entries for setting the trash folder size, encryption key
	   and asynchronous queue size.
	6. Block a process if trash if full.
	7. Ioctl support for restoring files to original location.
	8. Periodically deleting the contents of the trash folder.
	
* Design details.

	a. CLONE system call

		The clone system call has been modified to accept three additional
	flags and create processes with these flags. If we observe the flags passed
	to the clone system call, we notice that there are no unique bit positions
	available to the new flags. So to overcome this drawback we make use of the
	invalid flag combination defined in the clone() system call man page. The 
	new flags have been defined as follows in the include/uapi/linux/sched.h 
	header file. 
	
	#define CLONE_PROT_MV           CLONE_THREAD|CLONE_NEWUSER
	#define CLONE_PROT_ZIP          CLONE_THREAD|CLONE_NEWPID
	#define CLONE_PROT_ENC          CLONE_NEWNS|CLONE_FS	

		The task_struct structure was also modified by adding a new del_flags
	variable. This variable is used to differentiate processes that have these
	special flags set, and is set in the _do_fork() function of kernel/fork.c .
		These flags are then read during the unlink process to determine the
	appropriate actions to take.


	b. Setting /proc entries for the various system parameters

		I've also added the following files to the fs/proc/ folder. Each of 
	these entries corresponds to a proc variable that can be set by the user. The
	command for setting the values is also shown.

	1. encrypt_pass.c - Sets the encrypt_key variable that can be used for file 
	encryption.
		
		echo "encryptionkeyfromuser" > /proc/encrypt_key

	2. trash_size.c - Sets the trash_size variable that determines the number of
	files that can be present in the trash folder. The default values is 10.

		echo 15 > /proc/trash_size

	3. work_queue_size.c - Sets the size of the queue used for asynchronous processing.
	The default value for the queue size is 10.

		echo 15 > /proc/queue_size
	
		We can also read the values of these /proc entries through /bin/cat.

	c. Deleting files and moving to trash folder
		
		To determine the action to take for unlink we make use of the del_flags
	member of the current process that is making the call. If this flag is not set
	then we perform the default unlink action. If it is set, then we perform actions
	based on the values of the set flags.

	1. CLONE_PROT_MV : If this is the only flag set, then we simply move the file to 
	the trash folder. 

	2. CLONE_PROT_ZIP : If this flag is set we first compress the contents of the file
	and them move it to the trash folder. We use the LZ4 compression algorithm to compress
	the files.
	
	3. CLONE_PROT_ENC : If this flag is set, we encrypt the contents of the file and move
	it to the trash folder. If the user has passed a key using the /proc entry described
	earlier, we use that as the encryption key else we use a random key for encryption.

	4. CLONE_PROT_ZIP|CLONE_PROT_ENC : Whenever this flag combination is set, we first 
	encrypt the file and then compress the encrypted file.


	d. Synchronous and Asynchronous unlink.
		
		Whenever a process with the special flags set deletes a file smaller that 4K,
	we perform the operations on the file synchronously. For larger files we perform 
	this asynchronously. The asynchronous functionality is implemented using the linux
	workqueue APIs defined in linux/workqueue.h. The corresponding functions for 
	initializing the workqueue are present in the fs/work_queue.c file. Asynchronous 
	processing of the files is done for the encryption/compression operations and is done
	after moving the file to the trash folder successfully. We initialize a work_struct
	variable for each file to be queued and use a work handler function to perform the 
	operations. The following lists the functions in fs/work_queue.c that perform the
	operations.

	enqueue_work() - Initializes the work queue and the work_struct object and adds to
			 the queue.

	do_async_work() - Work handler function that performs the actual asynchronous work.

		In addition to this we also use items_count variable and the async_wait_count
	counters that allow to synchronize and handle tasks waiiting to access the queue.
		
		We also have a wait queue for blocking processes whenever the trash folder is
	full or if the async queue is full. The variables and initialization functions are 
	defined in the fs/wait_queue.c file. We have a seperate wait queue for processes 
	that are blocking on the trash folder and for processes blocking on the async queue.
	Processes present in these queues are woken up whenever a file is deleted or if an async
	job was processed. 


	e. Blocking and waking up processes.
		
		Blocking of processes is done whenever the trash folder is full or the async
	queue is full. This is done using the following atomic counters.
	
	1. items_count - Number of items present in the async queue.
	2. async_wait_count - Number of processes waiting for the async queue.
	3. waiting_count - Number of processes blocked on the trash folder.
	4. trash_file_count - Number of files currently present in the trash folder.
	
		During unlink we first check if we have space in the trash folder and if not 
	we block the process. Everytime we delete an entry from the trash folder we check
	the counter and wake up one of the processes. Similarly we check the asnyc queue size
	when adding work to it and block the process if the queue is full. We then wake up
	the process once one of the entries in the queue has been processed.

	f. Periodic trash cleanup
	
		I've also implemented a module which when inserted starts a kernel thread that
	deletes the oldest file in the recycle bin at intervals of 30 seconds. 

	g. Files add/changed
	
	   In the fs folder
	1. namei.c 
	2. wait_queue.c
	3. work_queue.c
	4. custom.h
	5. wq.h
	6. ioctl.c
	
	   In the kernel folder
	7. fork.c

About

Implementation of an asynchronous file system for file deletion and encryption.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published