Skip to content

Latest commit

 

History

History
60 lines (55 loc) · 2.8 KB

README.md

File metadata and controls

60 lines (55 loc) · 2.8 KB
Raw

wterl is an Erlang interface to the WiredTiger database, and is written to support a Riak storage backend that uses WiredTiger.

Remaining work includes:

TODO:

  • Find/fix any code marked "TODO:"
    • Why do we see {error, {eperm, _}} result on wterl:cursor_close/1 during fold_objects/4?
    • Why do we see {error, {eperm, _}} result on wterl:cursor_close/1?
    • Why do we see {error, {eperm, _}} result on wterl:cursor_next/1 during is_empty/1?
    • Why do we see {error, {eperm, _}} result on wterl:cursor_next_value/1 during status/1?
    • Why do we see {error, {ebusy, _}} result on wterl:drop/2?
    • Determine a better way to estimate the number of sessions we should configure WT for at startup in riak_kv_wterl_backend:max_sessions/1.
  • Make sure Erlang is optimizing for selective receive in async_nif_enqueue/3 because in the eLevelDB driver there is a comment: "This cannot be a separate function. Code must be inline to trigger Erlang compiler's use of optimized selective receive."
  • Provide a way to configure the cursor options, right now they are always "raw,overwrite".
  • Add support for Riak/KV 2i indexes using the same design pattern as eLevelDB (in a future version consider alternate schema)
  • If an operation using a shared cursor results in a non-normal error then it should be closed/discarded from the recycled pool
  • Cache cursors based on hash(table/config) rather than just table.
  • Finish NIF unload/reload functions and test.
  • Test an upgrade, include a format/schema/WT change.
  • When WT_PANIC is returned first try to unload/reload then driver and reset all state, if that fails then exit gracefully.
  • Currently the riak_kv_wterl_backend module is stored in this repository, but it really belongs in the riak_kv repository.
  • wterl:truncate/5 can segv, and its tests are commented out
  • Add async_nif and wterl NIF stats to the results provided by the stats API
  • Longer term ideas/changes to consider:
    • More testing, especially pulse/qc
    • Riak/KV integration
      • Store 2i indexes in separate tables
      • Store buckets, in separate tables and keep a <<bucket/key>> index to ensure that folds across a vnode are easy
      • Provide a drop bucket API call
      • Support key expirey
      • An ets API (like the LevelDB's lets project)
      • Use mime-type to inform WT's schema for key value encoding
  • Other use cases within Riak
    • An AAE driver using WT
    • An ability to store the ring file via WT

Deploying

You can deploy wterl into a Riak devrel cluster using the enable-wterl script. Clone the riak repo, change your working directory to it, and then execute the enable-wterl script. It adds wterl as a dependency, runs make all devrel, and then modifies the configuration settings of the resulting dev nodes to use the WiredTiger storage backend.