GitHub - karthick18/io_cache: A fast read-only IO cache thats expected to be used for processing large data-sets or files preferably > 1GB atleast without worrying about the maintenance overhead associated with processing a large data-file. The user need not focus on managing the reads of such large files and could instead focus on the core-processing of his large data-sets stored in the file..

karthick18 / io_cache Public

Notifications You must be signed in to change notification settings
Fork 0
Star 3

A fast read-only IO cache thats expected to be used for processing large data-sets or files preferably > 1GB atleast without worrying about the maintenance overhead associated with processing a large data-file. The user need not focus on managing the reads of such large files and could instead focus on the core-processing of his large data-sets …

View license

3 stars 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 7 Commits
LICENSE.txt		LICENSE.txt
Makefile		Makefile
README		README
io_cache.c		io_cache.c
io_cache.h		io_cache.h
list.h		list.h
task_pool.c		task_pool.c
task_pool.h		task_pool.h
test_iocache.c		test_iocache.c

Repository files navigation

A fast io-cache for post-processing and reading large files preferably over 1GB and above. User needn't focus on efficiently managing and reading the data off the file and maintain his own read buffers. He could instead focus on processing his large data-sets in such files where he could be expected to randomly iterate and read over his data-sets stored in the file for processing. The cache can be initialized upto a maximum of 4 GB. The limit specified should also be less than 50% of available memory in the system. The APIs to initialize/read/lock the io_cache can be found in io_cache.h. A sample usage can be seen in test_iocache.c.

-A.R.Karthick (a.r.karthick@gmail.com, a_r_karthic@users.sourceforge.net)