Skip to content
Userspace implementation of file APIs using persistent memory.
C C++ Python CMake Shell Perl
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
debian Add FUSE driver using libpmemfile-posix. Sep 19, 2017
doc doc: Minor fixes in libpmemfile-posix.3.md Oct 4, 2017
include
src posix: better error message when there's ref leak Nov 17, 2017
tests tests: update "match" to the latest version Nov 2, 2017
utils common: fix Coverity patch path May 14, 2018
.clang-format
.gitignore docker: add ltp scripts Sep 15, 2017
.mailmap Add .mailmap Nov 16, 2017
.travis.yml docker: add ltp scripts Sep 15, 2017
CMakeLists.txt libpmemfile-pop - a library which translates libpmemfile-posix functi… Oct 13, 2017
ChangeLog Make it build! Feb 16, 2017
LICENSE Mention origin of cmake_uninstall_license.cmake.in Apr 21, 2017
README.md libpmemfile-pop - a library which translates libpmemfile-posix functi… Oct 13, 2017
STATUS.md antool: add fcntl's F_SETFL(0x04) flag to unsupported ones Sep 28, 2017
cmake_uninstall.cmake.in Mention origin of cmake_uninstall_license.cmake.in Apr 21, 2017
codecov.yml Allow code coverage to drop by 0.2%. Sep 13, 2017
pmemfile-debug.spec
pmemfile.spec

README.md

pmemfile

Build Status Coverage Status Coverity Scan Build Status

Pmemfile project's goal is to provide low-overhead userspace implementation of file APIs using persistent memory. It consists of 2 major compoments:

  • libpmemfile-posix - provides syscall-like API which can be used by applications
  • libpmemfile - provides transparent access to libpmemfile-posix pools

How to build

Requirements:

$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=/usr
$ make
$ make install

When developing:

$ ...
$ cmake .. -DCMAKE_BUILD_TYPE=Debug -DDEVELOPER_MODE=1 -DTEST_DIR=/mnt/pmem/pmemfile-tests
$ ...
$ ctest --output-on-failure

Note that in Debug mode "make install" installs only debug libraries.

Pmemfile-specific cmake variables:

  • BUILD_LIBPMEMFILE=0 - disables building of libpmemfile.so
  • BUILD_LIBPMEMFILE_POP=1 - builds tests using with libpmemfile-posix-over-POSIX library which imitates libpmemfile-posix.so, but uses POSIX functions provided by OS
  • DEVELOPER_MODE=1 - enables coding style, whitespace, license checks and enables fail-on-warning flags
  • LONG_TESTS=1 - enables tests which take much more time
  • TEST_DIR=/mnt/pmem/test_dir - provides directory where tests will create its pools
  • TRACE_TESTS=1 - dumps more info when test fails (requires cmake >= 3.4)
  • TESTS_USE_FORCED_PMEM=1 - allows tests to force enable or force disable use of optimized flush in libpmemobj (to speed them up)
  • USE_ASAN=1 - enables AddressSanitizer (only for debugging)
  • USE_UBSAN=1 - enables UndefinedBehaviorSanitizer (only for debugging)
  • LIBPMEMFILE_VALIDATE_POINTERS=1 - build-in support for PMEMFILE_PRELOAD_VALIDATE_POINTERS environment variable

Package for Debian-based distros

$ ./utils/build-deb.sh

Resulting packages will be in ./build-deb/release and ./build-deb/debug

Package for RPM-based distros

$ ./utils/build-rpm.sh

Resulting packages will be in ~/rpmbuild/RPMS.

Note: Before 1.0 pmemfile-posix API and ABI is considered unstable. We reserve the right to change the on-media layout without warning.

Description:

Example:

$ mkfs.pmemfile /dev/dax0.0 0
$ mkdir /tmp/mountpoint

$ sudo pmemfile-mount /dev/dax0.0 /tmp/mountpoint
$ alias pf='LD_PRELOAD=libpmemfile.so'

# or if you don't have root access:

$ alias pf='LD_PRELOAD=libpmemfile.so PMEMFILE_POOLS=/tmp/mountpoint:/dev/dax0.0'

# now all commands prefixed with 'pf' will see files under /tmp/mountpoint,
# but files will be stored in file system backed by /dev/dax0.0
$ pf mkdir /tmp/mountpoint/dir_in_pmemfile
$ pf cp README.md /tmp/mountpoint/dir_in_pmemfile
$ pf ls -l /tmp/mountpoint/
total 0
drwxrwxrwx 2 user group 4008 Feb 16 17:46 dir_in_pmemfile
$ pf ls -l /tmp/mountpoint/dir_in_pmemfile
total 16
-rw-r--r-- 1 user group 1014 Feb 16 17:46 README.md
$ pf cat /tmp/mountpoint/dir_in_pmemfile/README.md | wc -c
1014
# verify that files are stored in pmemfile pool:
$ ls -l /tmp/mountpoint/
total 0
$ ls -l /tmp/mountpoint/dir_in_pmemfile
ls: cannot access '/tmp/mountpoint/dir_in_pmemfile': No such file or directory

Limitations:

  • libpmemfile[-posix].so does not support multiple processes accessing the pool (libpmemobj limitation)
  • libpmemfile.so supports Linux only (other UNIX-like systems could be supported)
  • libpmemfile.so supports x86_64 only (libsyscall_intercept, libpmem and libpmemobj limitation)
  • libpmemfile.so is not safe with remotely replicated pool (libpmemfile-posix.so has no such limitation)

Debugging:

Environment variables:

libsyscall_intercept.so:

  • INTERCEPT_LOG - log file for all intercepted syscalls (default: none)
  • INTERCEPT_LOG_TRUNC - when set to 0, the above log file is not truncated (default: 1)
  • INTERCEPT_HOOK_CMDLINE_FILTER - intercept syscalls only when process name matches this variable (used to skip gdb from intercepting) (default: empty)

libpmemobj.so:

  • PMEMOBJ_LOG_FILE - log file (default: none)
  • PMEMOBJ_LOG_LEVEL - logging level (default: 0)

libpmemfile-posix.so:

  • PMEMFILE_POSIX_LOG_FILE - log file (default: none)
  • PMEMFILE_POSIX_LOG_LEVEL - logging level (default: 0)

libpmemfile.so:

  • PMEMFILE_PRELOAD_LOG - log file (default: none)
  • PMEMFILE_PRELOAD_LOG_TRUNC - when set to 0, the above log file is not truncated (default: 1)
  • PMEMFILE_EXIT_ON_NOT_SUPPORTED - when set to 1, aborts an application which uses unsupported syscall (default: 0)
  • PMEMFILE_PRELOAD_PAUSE_AT_START - pauses initialization of pmemfile until debugger is attached (default: 0)

Other variables:

  • PMEMFILE_BLOCK_SIZE - forces one block size (default: dynamic)
  • PMEMFILE_CD - performs early chdir() to specified directory, used as a workaround for missing multi-process support when application must start from pmemfile-backed directory (default: none)
  • PMEMFILE_IGNORE_INODE_FREE_ERRORS - when set to 1, disables abort() when freeing inode's metadata fails (it defers freeing to the next application start) - can be used to get out of out-of-space situations (default: 0)
  • PMEMFILE_OVERALLOCATE_ON_APPEND - when set to 0, disables allocation of more space than required (default: 1)
  • PMEMFILE_PRELOAD_PROCESS_SWITCHING - when set to 1, enables VERY slow emulation of multi-process support, used for testing pmemfile with file system test suites (default: 0)
  • PMEMFILE_PRELOAD_VALIDATE_POINTERS - when set to 1, verifies memory reaching libpmemfile through syscall arguments is accessible; it's very slow, so it should never be used in production for non-buggy applications

Other stuff

You can’t perform that action at this time.