NVML Version 1.2 - Windows Technical Preview

@krzycz krzycz released this Dec 30, 2016 · 282 commits to master since this release

* Version 1.2 - Windows Technical Preview #1
This is the first Technical Preview release of NVML for Windows.
It is based on NVML 1.2 version, but not all the 1.2 features
are ported to Windows. In particular, Device DAX and remote access
to persistent memory (librpmem) are not supported by design.

NOTE: This release has not gone through the full validation cycle,
but only through some basic tests on Travis and AppVeyor.
Thus, it cannot be assumed "Production quality" and should not
be used in production environments.

Besides several minor improvements and bug fixes, all the other
changes since NVML 1.2 release were related to Windows support:
- win: port libvmem (and jemalloc)
- win: benchmarks Windows port
- win: fix mapping files of unaligned length
- win: clean up possible race condition in mmap_init()
- win: enable QueryVirtualMemoryInformation() in pmem_is_pmem()
- test: check open handles at START/DONE
- test: port all the remaining unit tests
  (scope, pmem_map, obj_debug, util_poolset, pmempool_*)
- win: add resource files for versioning

Known issues and limitations of Windows version of NVML:
- Unicode support is missing.  The UTF/UCS-encoded file paths
  or pool set files may not be handled correctly.
- The libvmmalloc library is not ported yet.
- The on-media format of pmem pools is not portable at the moment.
  The pmem pools created using Windows version of NVM libraries
  cannot be open on Linux and vice versa.
- Despite the fact the current version of NVML would work
  with any recent version of Windows OS, to take full
  advantage of PMEM and NVML features and to benefit from
  the PMEM performance, the recommended platforms needs be
  equipped with the real NVDIMMs hardware and should support
  the native, Microsoft's implementation of DAX-enabled file
  system (i.e. Windows Server 2016 or later).
  In case of using NVML with older versions of Windows or with
  the custom implementation of PMEM/DAX drivers, the performance
  might not be satisfactory.  Please, contact the provider
  of PMEM/DAX drivers for your platform to get the customized
  version of NVML in such case.


NVML Version 1.2

@krzycz krzycz released this Dec 16, 2016 · 430 commits to master since this release

* Version 1.2
This NVML release causes a "flag day" for libpmemobj.
The pmemobj pools built under NVML 1.1 are incompatible with pools built
under NVML 1.2 and later.  This is because an issue was discovered with
the alignment of locks (#358) and, although rare, the issue potentially
impacts program correctness, making the fix mandatory.
The major version number of the pmemobj pool layout and the version
of the libpmemobj API is changed to prevent the use of the potentially
incorrect layout.

Other key changes introduced in this release:
- Add Device DAX support, providing that "optimized flush" mechanism
  defined in SNIA NVM Programming Model can safely be used, even
  if PMEM-aware file system supporting that model is not available,
  or if the user does not want to use the file system for some reason.
- Add a package for libpmemobj C++ bindings.
  C++ API is no longer considered experimental.
  Web-based documentation for C++ API is available on http://pmem.io.
- Add "sync" and "transform" commands to pmempool utility.
  The "sync" command allows to recover missing or corrupted part(s)
  of a pool set from a healthy replica, while the "transform" command
  is a convenient way for modifying the structure of an existing
  pool set, i.e. by adding or removing replicas.
- Add experimental support for remote access to persistent memory and
  basic remote data replication over RDMA (librpmem).  Experimental
  support for remote replicas is also provided by libpmemobj library.

New features:
- common: add Device DAX support (#197)
- obj: add C++ bindings package (libpmemobj++-devel)
- obj: add TOID_OFFSETOF macro
- pmempool: add "sync" and "transform" commands (#172, #196)

Bug fixes:
- obj: force alignment of pmem lock structures (#358)
- blk: cast translation entry to uint64_t when calculating data offset
- obj: fix Valgrind instrumentation of chunk headers and cancelled
- obj: set error message when user called pmemobj_tx_abort()
- obj: fix status returned by pmemobj_list_insert() (#226)
- obj: defer allocation of global structures

- obj: fast path for pmemobj_pool_by_ptr() when inside a transaction
- obj: simplify and optimize allocation class generation

Experimental features:
- rpmem: add support for remote access to persistent memory and basic
  remote data replication over RDMA
- libpmempool: add pmempool_sync() and pmempool_transform() (#196)
- obj: introduce pmemobj_oid()
- obj: add pmemobj_tx_xalloc()/pmemobj_tx_xadd_range() APIs and
  the corresponding macros
- obj: add transaction stage transition callbacks


NVML Version 1.1

@krzycz krzycz released this Jun 26, 2016 · 1711 commits to master since this release

* Version 1.1   
This NVML release introduces a new version of libpmemobj pool layout.
Internal undo log structure has been modified to improve performance
of pmemobj transactions.  Memory pools created with older versions
of the libpmemobj library must be converted to the new format using
"pmempool convert" command.  See pmempool-convert(1) for details.

A new "libpmempool" library is available, providing support for
off-line pool management and diagnostics.  Initially it provides only
"check" and "repair" operations for log and blk memory pools, and for
BTT devices.

Other changes:
- pmem: deprecate PCOMMIT
- blk: match BTT Flog initialization with Linux NVDIMM BTT
- pmem: defer pmem_is_pmem() initialization (#158)
- obj: add TOID_TYPEOF macro

Bug fixes:
- doc: update description of valid file size units (#133)
- pmempool: fix --version short option in man page (#135)
- pmempool: print usage when running rm without arg (#136)
- cpp: clarify polymorphism in persistent_ptr (#150)
- obj: let the before flag be any non-zero value (#151)
- obj: fix compare array pptr to nullptr (#152)
- obj: cpp pool.get_root() fix (#156)
- log/blk: set errno if replica section is specified (#161)
- cpp: change exception message (#163)
- doc: remove duplicated words in man page (#164)
- common: always append EXTRA_CFLAGS after our CFLAGS

Experimental features:
- Implementation of C++ bindings for libpmempobj is complete.
  Web-based documentation for C++ API is available on http://pmem.io.
  Note that C++ API is still considered experimental.  Do not use it
  in production environments.
- Porting NVML to Windows is in progress.  There are MS Visual Studio
  solution/projects available, allowing to compile libpmem, libpmemlog,
  libpmemblk and libpmemobj on Windows, but the libraries are not fully
  functional and most of the test are not enabled yet.