New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

META: libpmemobj 1.6 release #932

Open
pbalcer opened this Issue Oct 2, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@pbalcer

pbalcer commented Oct 2, 2018

  • Performance Improvements
    • Allocator transient state improvements
      • Run tracking improvements #906
    • ulog invalidation instead of data clobbering #935
    • asynchronous RDMA replica writes (flush/drain model) #870
  • Under the hood changes
    • Remove expensive linear reservations runtime tracking #934
    • New multiple consumer single producer hashmap #871
  • Functionality
    • Expanded Actions API #880
    • More allocator/transaction statistics #888
    • thread-safe pmemobj_create(), pmemobj_open(), pmemobj_close() #872
    • switchable transaction context (TLS) pmem/pmdk#1965
    • allocation classes delete operation #936
    • Improved allocator arenas configurability #937
  • RAS
    • new OS interfaces / libndctl adoption
      • linux permission problem
  • Tools / Benchmarks / Tests
    • Fragmentation and long-term stability focused benchmarks #886
    • initial python test framework integration pmem/pmdk#3209 (stretch)
    • new build system in CMake (stretch)
    • error handling/error injection/coverage
  • Documentation / Examples
    • Technical paper describing the library in detail #887
  • C++
    • Change atomic make persistent to allow nesting #909
    • CTL/allocation flags #959
    • pmembench/C++ benchmark
    • Persistent Containers #910
      • pmem::obj::vector
      • pmem::obj::string
      • pmem::obj::map (stretch)
      • pmem::obj::unordered_map (stretch)
      • pmem::obj::concurrent_hash_map

@pbalcer pbalcer added this to the 1.6 milestone Oct 2, 2018

@pbalcer pbalcer referenced this issue Oct 2, 2018

Closed

META: libpmemobj 1.5 release #869

17 of 29 tasks complete
@pbalcer

This comment has been minimized.

pbalcer commented Oct 2, 2018

Just like last time, I created a list of things that I think we should deliver for the next release. This one is significantly smaller, and because only one thing here touches the on-media layout (#935), we might actually end up splitting this list into two separate, even tinier, releases.

Feel free to suggest new things ;)

@RobDickinson

This comment has been minimized.

Member

RobDickinson commented Oct 3, 2018

@pbalcer two questions on scope/goals:

  1. Does #910 include merging Sergey's TBB-based concurrent hashmap? (Of course we expect this to be marked "experimental" and not be fully supported) I'm hoping we can get to this fairly early in the release since this pull request has already been waiting 7+ weeks. If we cannot commit to merge these changes for 1.6, would be better to admit this sooner than later.

  2. Several times I've heard that #905 isn't needed and can be accomplished another way. Should we be counting on #905 for the concurrent hashmap implementation, or performing nested allocations using a different approach? Currently the concurrent hashmap has hardcoded limits on string size which is not practical or efficient.

Thanks for your consideration -- we'll be able to take a big leap forward with pmemkv this fall if we can navigate these issues!

Regards,
RobD

@pbalcer

This comment has been minimized.

pbalcer commented Oct 4, 2018

  1. I think it's realistic, but it's not going to be a blocker for our main release. libpmemobj-cpp after 1.5 is most likely going its separate way w.r.t. release cadence. Personally, I want it to be merged, but I acknowledge the issues that the team raised, and we simply need to work on resolving them.
  2. There are workarounds, but they are inconvenient or have slightly different semantics (you can achieve nested allocs with transactions for example). This does not mean however that we are not going to implement this feature, it just means it's not a big priority for us. I want this to be done, because it's one of those things that are unintuitive and fail silently.
@RobDickinson

This comment has been minimized.

Member

RobDickinson commented Oct 4, 2018

Thanks for clarifying that 1 & 2 are not high priority changes for PMDK. But obviously these are blocking issues for pmemkv. Sergey and I acknowledge the issues that the team raised, and we've committed to doing the design & code reviews requested, but I'm worried that there is no commitment beyond that to move forward. I'm not sure how to build a schedule for pmemkv with this level of uncertainty, so I've asked Marek to try and help us on this.

Can I suggest changing the list of features above to indicate which changes are committed and which are stretch? It's not as helpful to have a big list of proposed changes and leave everybody wondering what subset will actually make it into the release.

Thank you for your attention on this matter!
RobD

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment