Releases · rabbitmq/khepri

14 Nov 16:09

dumbbell

v0.6.0

7c0a4e5

Khepri 0.6.0

What's new in Khepri 0.6.0

This release focuses on several important new features.

Very fast queries thanks to "projections"

Projections are a new mechanism to allow fast queries with a low latency.

Projections use ETS tables to work as a cache. The user is in control of the way data is stored in these ETS tables:

He selects what is to be cached using path patterns.
He provides an anonymous function to transform the data payload to a record ready to be stored in ETS.

This obviously comes at a cost. The cache being implemented as ETS tables means data duplication between the store itself and the ETS tables and an increase in memory footprint. Also, queries are therefore eventually consistent (because of the time it may take to reflect a store change in the cache).

Here is a copy of the exemple in the pull request:

You need to declare the projection first:

%% `ProjectionName' will be the name of the ETS table.
ProjectionName = wood_stocks,

%% `ProjectionFun' defines how a data payload is transformed into an ETS
%% record.
ProjectionFun = fun([stock, wood, Kind], Stock) -> {Kind, Stock} end,

%% This line creates an opaque term from the variables above.
Options = #{type => set, read_concurrency => true},
Projection = khepri_projection:new(ProjectionName, ProjectionFun, Options),

%% You can finally associate the created projection with a path pattern.
StoreId = stock,
PathPattern = "/:stock/:wood/*",
RegisterOptions = #{},
khepri:register_projection(StoreId, PathPattern, Projection, RegisterOptions).

The ETS table named wood_stocks will be created and filled with the existing tree nodes matching the path pattern, after they are transformed using the given anonymous function.

You can start to query the ETS table for cached values:

%% Imagine you write a new value in the store, whether it is before or
%% after creating the projection.
khepri:put(StoreId, "/:stock/:wood/Oak", 100),

%% After the projection was defined, you can query the ETS table:
100 = ets:lookup_element(ProjectionName, <<"Oak">>, 2).

See #135 for more information.

Import and export of a store

This release introduces an import/export mechanism.

This feature relies on import/export backend modules. These modules follow the same API as the "backup and restore" Mnesia API, therefore it is possible to re-use existing backup modules for Mnesia.

Unlike Mnesia however, it is possible to export a part of the store by specifying a path pattern to select tree nodes to export.

During import, everything is written to the store. Note that other existing tree nodes are left untouched: the store is not emptied before an import.

Khepri 0.6.0 comes with an import/export backend module called khepri_export_erlang which uses a plaintext file where tree nodes are formatted as Erlang terms. The benefit is it doesn't add any dependencies to the library.

Here is an example:

Export the wood stocks from an existing store:

ok = khepri:export(
   StoreId,
   "/:stock/:wood/**",
   khepri_export_erlang,
   "export-wood-stock-1.erl").

export-wood-stock-1.erl contains:

{put, [stock, wood, <<"Oak">>], {p_data, 100}, #{}}.
{put, [stock, wood, <<"Maple">>], {p_data, 230}, #{}}.

Import the wood stocks back:

ok = khepri:export(
   StoreId,
   khepri_export_erlang,
   "export-wood-stock-1.erl").

See #157 for more information.

New `maps:fold/3`-like functions

To make it easier to work on many tree nodes, Khepri 0.6.0 brings the common iterative functions we are used to use in lists and maps modules:

khepri:fold/5
khepri:foreach/4
khepri:map/4
khepri:filter/4

They will all loop on tree nodes matching the given path pattern and run the given anonymous function for each one of them. Except for khepri:foreach/4, the return value of this anonymous function will be used to build a new Erlang term return by the khepri:fold/5 and friends.

Example:

Fun = fun
          (Path, #{data := Stock}, Acc) ->
              Wood = lists:last(Path),
              Acc#{Wood => Stock};
          (_Path, _NodeProps, Acc) ->
              Acc
      end,

#{<<"Oak">> := 100},
 {<<"Maple">> := 230}} = khepri:fold(StoreId, "/:stock/:wood/*", Fun, #{}).

See #163 for more information.

Stored procedures as transaction functions

Transactions in Khepri can be expensive in terms of CPU and memory because of the need to extract the given anonymous function, verify what it does and create a standalone module out of it to be executed on all Ra nodes.

To "pay" this extraction cost once, it is now possible to use stored procedures as transaction functions. In other words, a caller needs to:

store the transaction anonymous function once in the store
call khepri:transaction/2 with the path to that stored procedure instead of passing the anonymous function directly

To make transactions more flexible and allow the use of more generic stored procedures, transaction functions can now take arguments.

See #150 for more information.

Other changes

Automatically reclaim useless tree nodes which were automatically created (#155).
Support empty binaries in string-based paths (#156).
Fix handling of khepri:delete_many("**") (#158).
Rename delete_payload() back to clear_payload().
Fix application definition when building Khepri with mix(1) (#146).
Add module_info/{0,1} functions to extracted stored procedures and transaction functions, as required by debugging tools (#148).
Fix several issues around function extraction: #149, #161, #165, #168, #169, #170.
Acknowledge trigger executions asynchronously to fix a race with store shutdown (#151).
Fix regression with the use of #if_has_data{} with node props (#154).
Implement ra_machine:overview/1 for khepri_machine (#159).
Reduce the size of the "put" Ra commands by merging two maps (#166).
Accept machine_config in a Ra server config map (#167).

Download

Khepri 0.6.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.6.0

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.6.0"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.6.0

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.6.0 is available on Hex.pm.

Assets 2

07 Oct 13:05

dumbbell

v0.5.0

90efd62

Khepri 0.5.0

What's new in Khepri 0.5.0

Rework the public API again

In Khepri 0.3.0, the public API already saw a lot of changes to make it more straightforward. We were still unhappy with the result and this release is another attempt at providing a nice API. The goals are:

to have a simple API with no surprises when you read a program using Khepri
to keep the ability to handle more advanced use cases

Now, the khepri module exposes the simple API mentionned above, while a new module, khepri_adv covers the advanced use cases. Likewise inside transactions with khepri_tx and khepri_adv.

Here is an example of the changed API:

Up-to Khepri 0.4.x:

%% Store and get a value:
{ok, _} = khepri:put("/:stock/:wood/Oak", 120),
{ok, #{[stock, wood, <<"Oak">>] := #{data := 120}}} = khepri:get("/:stock/:wood/Oak"),

%% Get the entire stock:
{ok, #{[stock, wood, <<"Oak">>] := #{data := 120},
       [stock, wood, <<"Maple">>] := #{data := 45}}} = khepri:get_many("/:stock/:wood/*"),

%% Update the stock after a fire in the warehouse...
%% -> Use a transaction going through all elements.

%% Or:
{ok, _} = khepri:delete("/:stock/:wood").

Starting from Khepri 0.5.0:

%% Store and get a value:
ok = khepri:put("/:stock/:wood/Oak", 120),
{ok, 120} = khepri:get("/:stock/:wood/Oak"),

%% Get the entire stock:
{ok, #{[stock, wood, <<"Oak">>] := 120,
       [stock, wood, <<"Maple">>] := 45}} = khepri:get_many("/:stock/:wood/*"),

%% Update the stock after a fire in the warehouse...
ok = khepri:put_many("/:stock/:wood/*", 0),

%% Or:
ok = khepri:delete("/:stock/:wood").

Everything is described in greater details in pull request #145.

Errors from `khepri:exists()`, `khepri:has_data()` and `khepri:is_sproc()`

The functions khepri:exists(), khepri:has_data() and khepri:is_sproc() are checking a particular property of the targeted tree node. They used to return a boolean(). However, when there was an error in the process, like an issue with the network, the errors were ignored and false was returned.

This is not the truth and the functions can't know. Therefore, they now return that error.

See #126.

Other changes

Handle nodedown when handling queries (#125).
Several bug fixes around function extraction: #130, #131, 134, #138 and #139.
Fix a bug with triggers on grandchildren (#137).
Improve documentation (#128 and #133).
Drop support for Erlang 23 (#132).

Download

Khepri 0.5.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.5.0

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.5.0"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.5.0

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.5.0 is available on Hex.pm.

Assets 2

19 Jul 10:59

dumbbell

v0.4.3

cd2d894

Khepri 0.4.3

Bugfixes in Khepri 0.4.3

Allow badrecord instructions (#120).
Work around several badly decoded instructions from beam_disasm (#119, #121, #122).

Download

Khepri 0.4.3 is available from Hex.pm: https://hex.pm/packages/khepri/0.4.3

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.4.3"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.4.3

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.4.3 is available on Hex.pm.

Assets 2

28 Jun 08:52

dumbbell

v0.4.2

d68beb4

Khepri 0.4.2

Bugfixes in Khepri 0.4.2

Fix a possible deadlock when a transaction is executed during the start or stop of Erlang applications (#117).

Download

Khepri 0.4.2 is available from Hex.pm: https://hex.pm/packages/khepri/0.4.2

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.4.2"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.4.2

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.4.2 is available on Hex.pm.

Assets 2

16 Jun 15:49

dumbbell

v0.4.1

0f55c10

Khepri 0.4.1

Bugfixes in Khepri 0.4.1

Fix a crash when a Khepri store joins a cluster after being restarted, i.e. was created by a previous call to khepri:start() (#113).
Update Ra from 2.0.13 to 2.1.0; this fixes a race condition in Ra which affects clustered Khepri stores when one of them is stopped (#115).

Other changes

Improve cluster_SUITE integration testsuite (#114, #116).

Download

Khepri 0.4.1 is available from Hex.pm: https://hex.pm/packages/khepri/0.4.1

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.4.1"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.4.1

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.4.1 is available on Hex.pm.

Assets 2

08 Jun 14:25

dumbbell

v0.4.0

2d7695b

Khepri 0.4.0

What's new in Khepri 0.4.0

Add a logo :-)

Let's start with the fanciest change: we added the logo you can spot on the right ;-)

This logo is inspired by the drawing representing the god Khepri: a dung beetle holding the Sun. Here we use a rabbit (for the RabbitMQ-originated project) holding a database diagram. I know, creativity turned to 11.

Revamp of the clustering API

The clustering API was completely revamped in Khepri 0.4.0. Like the rest of the public API in Khepri 0.3.0, the goal is again to make it more straightforward and consistent.

The #105 pull request gives more details about the changes and how existing code can be adapted. But here are some of the highlights:

It is possible to start a Khepri store with a data directory as argument. No need to configure a Ra system yourself.
Most functions can now wait for a Ra leader to be elected. They take a timeout as argument to limit that wait. You don't have to handle a retry loop anymore.
There is a khepri:stop() function now to stop a store.

Here is an example of an old code and its newer version:

Up to Khepri 0.3.0:

%% `my_ra_system' is configured using `ra_system:start/1'.
DataDir = "/var/lib/khepri",
RaSystemConfig = RaSystemConfig0#{name => my_ra_system,
                                  data_dir => DataDir,
                                  wal_data_dir => DataDir},
{ok, _} = ra_system:start(RaSystemConfig),

%% The Khepri store is started using that Ra system.
{ok, StoreId} = khepri:start(my_ra_system).

Starting from Khepri 0.4.0:

%% The Khepri store is started with a data directory. A Ra system will be
%% configured and started automatically.
{ok, StoreId} = khepri:start("/var/lib/khepri").

See #105.

Several improvements to make it easier to use Khepri for Elixir developers

We had great feedback on the Erlang forums following a question we asked about what should Khepri provide to make it nice to Elixir developers.

As a result, we implemented the following improvements:

Khepri path can be Erlang binaries (#93). Elixir and other languages like Gleam running on top of the BEAM runtime use Erlang binaries to implement the "string" type. With this change, their native strings can be used directly in the Khepri API.
```
iex> :khepri.get_data("/:foo")
:value
```

The khepri module exports "bang functions" such as khepri:'get!'(StoreId, Path) (#98). They don't look as nice in Erlang, but in Elixir, they are a common practice to have a version of a function which returns its result directly (if successful) or throws an error if not. They are especially handy when using pipelines.

iex> :khepri.get!(~p"/:foo")
%{
  [:foo] => %{
    child_list_length: 0,
    child_list_version: 1,
    data: :value,
    payload_version: 1
  }
}

iex> :khepri.get!(:non_existing_store, ~p"/:foo")
** (ErlangError) Erlang error: :noproc
    (khepri 0.3.0) /home/dumbbell/Projects/pivotal/khepri/src/khepri.erl:2629: :khepri.unwrap_result/1

The khepri_path modules exports ~p and ~P sigils (#98). They are constructs which make the parsing of Khepri Unix-like string/binary-based paths very easy in Elixir.
```
iex> import :khepri_path

iex> ~p"/stock/wood/:oak"
["stock", "wood", :oak]

iex> ~P"/stock/wood/:oak"
["stock", "wood", :oak]
```

See #93, #98.

EEP-48 doc chunks

In addition to rebar3_edoc_extensions, the plugin we use to produce documentations which are nicer to the eye compared to the default EDoc HTML output, we now also use the EDoc doclet and layout modules to produce EEP-48 doc chunks. People using Khepri should now get documentation and specs right from their compatible IDE and text editors.

► rebar3 shell
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling khepri
Erlang/OTP 24 [erts-12.3.2] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit] [dtrace]

Eshell V12.3.2  (abort with ^G)
1> h(khepri).

   khepri

  Khepri database API.

  This module exposes the database API to manipulate data.

  The API is mainly made of the functions used to perform simple direct atomic operations and queries on the database. In addition to that, transaction/1 are the starting point to run transaction functions. However the API to
  use inside transaction functions is provided by khepri_tx.

  This module also provides functions to start and stop a simple unclustered Khepri store. For more advanced setup and clustering, see khepri_cluster.

Other changes

Added khepri:count() functions (#102).
Added support for ETS pattern conditions in #if_data_matches{} conditions (97).
Improved support for Erlang/OTP 25 (#101).
Ra was updated from 2.0.9 from 2.0.13 (#110).
Clarified the difference between RDBMS and Khepri's triggers (#65).

Download

Khepri 0.4.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.4.0

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.4.0"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.4.0

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.4.0 is available on Hex.pm.

Assets 2

0 Join discussion

25 Apr 13:39

dumbbell

v0.3.0

4a32368

Khepri 0.3.0

What's new in Khepri 0.3.0

Revamp of the public API

The public API was completely revamped in Khepri 0.3.0, with the goal to make it more straightforward and consistent. Hopefully it will be easier to understand for both old and new users.

The #79 pull request gives more details about the changes and how existing code can be adapted. But here are some of the highlights:

The high- vs low-level API distinction is now gone. The public API is now exposed by khepri only. khepri_machine becomes an internal private module. As part of that khepri grew several new functions for common use cases and we will certainly add more in the future, based on the feedback.
Unix-like path are first-class citizen: all functions taking a native path ([stock, wood, <<"oak">>]) now accept Unix-like paths ("/:stock/:wood/oak"). In the process, the syntax of Unix-like paths evolved: atoms are represented as :atom and binaries are represented as-is, binary. The main reason is that using <<binary>> for binaries was difficult to read and type. See #73 and #74.
Payload and event filter records are now private. Payload types are automatically detected now, likewise for event filters. That said, it is still possible to use functions to construct the internal structures, but it should rarely be necessary.
khepri_tx, the module to perform Khepri calls inside transactions, will now expose the same API as khepri, except when functions don't make sense in a transaction.

Here is an example of an old code and its newer version:

Up to Khepri 0.2.1:

%% `khepri_machine' had to be used for "advanced" use cases, though
%% `khepri' would have been fine in this example.
case khepri_machine:get(StoreId, [stock, wood, <<"oak">>]) of
    %% Accessing the data in the node payload was a bit complicated,
    %% requiring to pattern-match on the node properties map inside the
    %% result map.
    {ok, #{Path := #{data := Quantity}}} when Quantity < 100 ->
        %% We would also have to construct a payload record.
        Payload = #kpayload_data{data = 500},
        {ok, _} = khepri_machine:put(StoreId, [orders, wood, <<"oak">>], Payload),
        ok
    _ ->
        ok
end.

Starting from Khepri 0.3.0:

%% Now we have helpers for common use cases like simply accessing the data
%% of a single tree node. The piece of data is returned directly, returning
%% a default value if there is no data and bypassing error handling if we
%% don't care.
%%
%% Unix-like paths are used for the demonstration. Native paths, like in
%% the previous example, would work as well.
Quantity = khepri:get_data_or(StoreId, "/:stock/:wood/oak", 0),
if
    Quantity < 100 ->
        %% The payload record is automatically constructed internally. No
        %% need to mess with records.
        {ok, _} = khepri:put(StoreId, "/:orders/:wood/oak", 500),
        ok;
    true ->
        ok
end.

See #79.

Options to play with consistency and latency

Queries, including read-only transactions, now accept a favor option to select if Khepri should favor consistency or low latencies:

consistency: it means that a "consistent query" will be used in Ra. It will return the most up-to-date piece of data the cluster agreed on. Note that it could block and time out if there is no quorum in the Ra cluster.
compromise: it performs "leader queries" most of the time to reduce latency (though latency will still be higher than "local queries"; see below), but uses "consistent queries" every 10 seconds to verify that the cluster is healthy. It should be faster but may block and time out like consistency and still return slightly out-of-date data.
low_latency: it means that "local queries" are used exclusively. They are the fastest and have the lowest latency. However, the returned data is whatever the local Ra server has. It could be out-of-date if it has troubles keeping up with the Ra cluster. The chance of blocking and timing out is very small.

%% Favor low latency, even if it means out-of-date data compared to what the consensus is.
khepri:get(StoreId, Path, #{favor => low_latency})

See #64.

Asynchronous updates

Updates, including read-write transactions, now accept an async option to configure asynchronous commands:

true to perform an asynchronous low-priority command without a correlation ID.
false to perform a synchronous command.
A correlation ID to perform an asynchronous low-priority command with that correlation ID.
A priority to perform an asynchronous command with the specified priority but without a correlation ID.
A combination of a correlation ID and a priority to perform an asynchronous command with the specified parameters.

%% Configure a correlation ID for this update.
Ref = erlang:make_ref(),
ok = khepri:put(StoreId, Path, Data, #{async => Ref}),

%% Later, consume the reply from Khepri/Ra.
receive
    {ra_event, _, {applied, [{Ref, Reply}]}} ->
        do_things(Reply)
end.

See #69.

Other changes

Introduced support for Erlang/OTP 25 (#81).
Ra was updated from 2.0.2 to 2.0.9 (#70, #71, #80, #89).
Introduced a compiled standalone functions cache (#72). Note that this cache is never cleaned up for now!
Numerous fixes and improvements to the function extraction code (khepri_fun) from @the-mikedavis (#63, #66, #67, #68 and probably more).

Download

Khepri 0.3.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.3.0

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.3.0"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.3.0

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.3.0 is available on Hex.pm.

Contributors

A warm thank you to contributors outside of the RabbitMQ team, this is invaluable for such a young project!

Kian-Meng Ang (@kianmeng)
Michael Davis (@the-mikedavis)

Contributors

kianmeng and the-mikedavis

Assets 2

0 Join discussion

18 Feb 08:50

dumbbell

v0.2.1

d708d3d

Khepri 0.2.1

What's new in Khepri 0.2.1

This release brings bug fixes only.

Many new assembly instructions supported in the function extraction code (commit b140d8c, commit 0836441, commit a801495, #54).
Complete "Line" beam chunk decoding (#55).
Use a valid SPDX license identifier in rebar.app.

Download

Khepri 0.2.1 is available from Hex.pm: https://hex.pm/packages/khepri/0.2.1

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.2.1"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.2.1

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.2.1 is available on Hex.pm.

Assets 2

15 Feb 16:42

dumbbell

v0.2.0

98ea160

Khepri 0.2.0

What's new in Khepri 0.2.0

Stored procedures and triggers

This version introduces stored procedures and triggers. They allow to store code in the database itself and automatically execute it after some event occurs.

Store an anonymous function in the tree:

StoredProcPath = [path, to, stored_procedure],

Fun = fun(Props) ->
          #{path := Path},
            on_action => Action} = Props
      end,

khepri_machine:put(
  StoreId,
  StoredProcPath,
  #kpayload_sproc{sproc = Fun}))}.

EventFilter = #kevf_tree{path = [stock, wood, <<"oak">>]},

ok = khepri_machine:register_trigger(
       StoreId,
       TriggerId,
       EventFilter,
       StoredProcPath))}.

In the example above, as soon as the [stock, wood, <<"oak">>] node is created, updated or deleted, the anonymous function will be executed.

See #47.

Use records to describe payloads instead of macros

Payloads are now described as Erlang records. This allows to benefit from more compile-time checks in particular.

Here are the modifications you will have to do in your code:

% Up until 0.1.1:
khepri_machine:put(StoreId, Path, ?DATA_PAYLOAD(Data)).

% Starting from 0.2.0:
khepri_machine:put(StoreId, Path, #kpayload_data{data = Data}).

See #20.

`keep_until` conditions renamed to `keep_while`

The keep_until option of khepri_machine:put/3 was renamed to keep_while. The reason is that the tree node is kept while the condition is met. As soon as the condition evaluates to false, the tree node is deleted.

Here are the modifications you will have to do in your code:

% Up until 0.1.1:
khepri_machine:put(StoreId, Path, Payload, #{keep_until => #{Path => Condition}}).

% Starting from 0.2.0:
khepri_machine:put(StoreId, Path, Payload, #{keep_while => #{Path => Condition}}).

See #18.

Stacktrace frames pointing to an extracted function code have line numbers

Before, when a transaction function encountered an exception, the stacktrace would miss the source file and line number for frames pointing to the extracted code:

%% Based on the following function:
%%     crashing_fun() ->
%%         throw("Expected crash").

1> Fun = khepri_fun:to_standalone_fun(fun mod:crashing_fun/0, #{}).
2> khepri_fun:exec(Fun, []).
** exception throw: "Expected crash"
     in function  kfun__mod__crashing_fun__33048370:run/0

Now, khepri_fun decode lines information and the stacktrace frames provide the source file and the line number:

1> Fun = khepri_fun:to_standalone_fun(fun mod:crashing_fun/0, #{}).
2> khepri_fun:exec(Fun, []).
** exception throw: "Expected crash"
     in function  kfun__mod__crashing_fun__33048370:run/0 (.../mod.erl, line 30)

See #51.

Other changes

Ra was updated from 2.0.2 to 2.0.4.
Several functions of the re Erlang module are allowed in transaction functions (#41).
Patch several assembly instructions incorrectly decoded by beam_disasm (#39).
Many new assembly instructions supported in the function extraction code.
Annotate registers with type information to fix compilation failures related to function extracted (#36, #49).
The workaround to make it possible to run Dialyzer on testsuite was removed (#23). Now, one just has to use rebar as test dialyzer.
Modules to generate the nicer documentation were moved to https://hex.pm/packages/rebar3_edoc_extensions.

Download

Khepri 0.2.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.2.0

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.2.0"}]}.

Run rebar3 upgrade khepri.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.2.0

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.2.0 is available on Hex.pm.

Contributors

A warm thank you to contributors outside of the RabbitMQ team, this is invaluable for such a young project!

Michael Davis (@the-mikedavis)
Viacheslav Katsuba (@vkatsuba)

Contributors

vkatsuba and the-mikedavis

Assets 2

0 Join discussion

16 Nov 17:35

dumbbell

v0.1.1

283c31c

Khepri 0.1.1

What's new in Khepri 0.1.1

`ReadWrite` argument to `khepri_tx:transaction/2` changes

The transaction API changed: khepri_tx:transaction/2 now takes the atoms ro, rw or auto as its second argument, instead of a boolean or auto. It makes it a bit clearer what the argument means out of context, compared to a boolean. This impacts the following APIs:

khepri:transaction/2
khepri:transaction/3
khepri_machine:transaction/2
khepri_machine:transaction/3

Here are the modifications you will have to do in your code:

%% Read/write transaction:
khepri:transaction(StoreId, Fun, true). % Up until 0.1.0
khepri:transaction(StoreId, Fun, rw).   % Starting from 0.1.1

%% Read/only transaction:
khepri:transaction(StoreId, Fun, false). % Up until 0.1.0
khepri:transaction(StoreId, Fun, ro).    % Starting from 0.1.1

The use of the auto argument remains unchanged.

See commit d7893f3.

Auto-detection of R/W transaction fixed

When a transaction function used another anonymous function which was declared outside of the transaction, that second anonymous function was not analyzed to determine the R/W nature. Therefore, if the transaction function didn't use khepri_tx:put/2 or khepri_tx:delete/1, it would be considered a read-only transaction. Unfortunately, if the second anonymous function did use one of those two APIs, the transaction would abort with a store_update_denied error.

The transaction below would be detected as read-only even though it calls khepri_tx:delete/1:

Fun = fun() -> khepri_tx:delete([foo]) end,
khepri_tx:to_standalone_fun(
  fun() -> Fun() end,
  auto),

Pull request #7 fixes this problem.

Other changes

Ra was updated from 2.0.1 to 2.0.2.
Several improvements and fixes to the testsuites and the specifications.
To be able to run Dialyzer on the testsuites, they are compiled at the same time as the actual source code of Khepri. Unfortunately, it means that PropEr becomes a regular dependency (instead of a test dependency), even though it is not used at runtime. That said it is a regular dependency from Rebar point of view, but not Erlang and the PropEr application won't be started or used at runtime.

Download

Khepri 0.1.1 is available from Hex.pm: https://hex.pm/packages/khepri/0.1.1

Upgrade

Using Rebar:

Update your rebar.config:

%% In rebar.config
{deps, [{khepri, "0.1.1"}]}.

Run rebar3 upgrade.

Using Erlang.mk:

Update your Makefile:

%% In your Makefile
dep_khepri = hex 0.1.1

Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.1.1 is available on Hex.pm.

Assets 2

Releases: rabbitmq/khepri

Khepri 0.6.0

What's new in Khepri 0.6.0

Very fast queries thanks to "projections"

Import and export of a store

New maps:fold/3-like functions

Stored procedures as transaction functions

Other changes

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.5.0

What's new in Khepri 0.5.0

Rework the public API again

Errors from khepri:exists(), khepri:has_data() and khepri:is_sproc()

Other changes

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.4.3

Bugfixes in Khepri 0.4.3

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.4.2

Bugfixes in Khepri 0.4.2

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.4.1

Bugfixes in Khepri 0.4.1

Other changes

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.4.0

What's new in Khepri 0.4.0

Add a logo :-)

Revamp of the clustering API

Several improvements to make it easier to use Khepri for Elixir developers

EEP-48 doc chunks

Other changes

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.3.0

What's new in Khepri 0.3.0

Revamp of the public API

Options to play with consistency and latency

Asynchronous updates

Other changes

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Contributors

Contributors

Khepri 0.2.1

What's new in Khepri 0.2.1

Download

Upgrade

Using Rebar:

Using Erlang.mk:

Documentation

Khepri 0.2.0

What's new in Khepri 0.2.0

Stored procedures and triggers

New `maps:fold/3`-like functions

Errors from `khepri:exists()`, `khepri:has_data()` and `khepri:is_sproc()`

`keep_until` conditions renamed to `keep_while`

`ReadWrite` argument to `khepri_tx:transaction/2` changes