Skip to content

v0.7.0

Latest

Choose a tag to compare

@GlenDC GlenDC released this 13 Nov 21:45
0.7.0
fe75960

A new engine, with the same API on top of it.

Breaking Changes:

  • Bump MSRV to 1.96;
  • venndb::__internal is gone, the types a database is made of are now public:
    RowSet, KeyIndex, BoolColumn, ValueColumn, Plan, Filter, Matches and RuntimeColumn;
  • <DB>QueryResult::iter now returns impl Iterator;
    • the <DB>QueryResultIter types are gone, which only breaks you if you named them;
  • a database holds at most u32::MAX rows;
  • <DB>Query and <DB>QueryResult are now generated for every database, also for one without filters;
  • filters can no longer be named and, any or any_where, or <filter>_where and <filter>_ref
    for a filter that exists as well,
    as query methods of that name are now generated;
    • Migration is as easy as renaming the property, or to #[venndb(skip)] it;
  • querying a filter map on a value no row has, next to values that do have rows,
    now returns the rows of the latter;
    • Prior to this release it returned nothing at all.

New Features:

  • [#17]: support filter maps with many values:
    • values are stored as a list of rows as long as few rows have them,
      and as a bitmask only once many do;
    • a database of 100 000 rows with 16 000 distinct values is created in 12 milliseconds
      instead of 5 seconds, and takes 6.5 MB instead of 330 MB;
  • support properties with multiple values, of which a row matches each: #[venndb(filter, set)];
    • for HashSet, BTreeSet and types implementing venndb::FilterSet;
    • or any other type using #[venndb(filter, members = path)];
  • support filtering on what the values of a property look like: <DB>Query::<property>_where(predicate);
    • the predicate is called once per distinct value, not once per row;
    • rows with an any value always match;
  • [#16]: support columns that are no property of a row:
    • <DB>::column(predicate) is computed once;
    • a venndb::RuntimeColumn can be changed at any time and from any thread,
      and grows along with the database;
    • add them to a query with <DB>Query::and;
  • [#21]: support keys computed from the row by a function: #[venndb(key = path)];
  • support wildcards decided by a function instead of venndb::Any: #[venndb(filter, any = path)];
  • add #[derive(Any)] to implement venndb::Any for an enum, with #[any] on the variants that match any value;
  • add <DB>Query::any() and <DB>Query::any_where(predicate) to pick a random row without
    finding all rows first: random rows are tried if many are expected to match;
    • three bool filters on 1 000 000 rows: 0.2 microseconds instead of 12;
  • add <DB>Query::<property>_ref(&value) to filter on a value without giving it away;
  • add <DB>QueryResult::any_where(predicate) and <DB>QueryResult::len();
  • add <DB>::index_by_<key>, <DB>::get and <DB>::shrink_to_fit;
  • add the derive feature, enabled by default, to be able to use the engine without the macro.

Example:

use venndb::{Filter, RuntimeColumn, VennDB};

#[derive(Debug, VennDB)]
pub struct Employee {
    #[venndb(key)]
    id: u32,
    #[venndb(filter, set)]
    skills: HashSet<Skill>,
    #[venndb(filter, any = Country::is_any)]
    country: Country,
}

let db = EmployeeDB::from_rows(/* .. */).unwrap();
let on_leave = RuntimeColumn::new();
on_leave.set(db.index_by_id(&42).unwrap(), true);

let mut query = db.query();
query
    .skills_where(|skill| matches!(skill, Skill::Language { native: true, .. }))
    .country("BE")
    .and(Filter::Runtime(&on_leave, false));
let employee = query.any();

Other Changes:

  • Replace hashbrown and rand crate deps with the standard library;
    • foldhash is used as the hash function, the one dependency left next to the macro;
  • Queries count and pick rows a machine word at a time, and start from the smallest set of rows;
  • Rewrite benchmarks around a database shaped like the one of rama,
    compared with roaring and an indexed sqlite;
  • The fuzzer now compares queries with a scan of the rows;
  • Add examples, and test that the code shown in the README and on the website is part of them.