Typedb 3.0 wishlist

[lveillard]

Hello! Here is a little compendium of features we are building in our abstraction layer (www.blitznocode.com) that would be nice to have in typeDB natively:

TOP 3 Key features: optional mutations, hooks and structs

1. Batched & optional mutations

• Similar to fetch but including optional insert & delete clauses on each nested branch related to the match clause (or the fetch)
• Unlike fetch, it would be nice to have multiple entry points at the root level, so we can mutate two things optionally from the root level (two simultaneous match-insert at the root level which can share $vars)

2. Pre-hook & post-hooks:

• This enables a ton of usecases: Data validation, default values, data transformation, cascade deletes... [Feature request] OnWrite / OnRead hooks  #6944

3. Structs

• Store JSON-like structured embedded in other things (attribute-in-attribute current pattern)
• Related to cardinality features, as arrays might be needed to be stored here and they might require to store its order or have repeated vlaues

Functions

• 4.1. More functions & operators: [Feature Request] More operators and basic functions typeql#290
• 4.2. Custom functions => like typedb rules but with params and reusable
• 4.3. HTTP functions / cloud lambdas => Having the logic layer really close to the database [Feature request] Cloud lambdas  #6847. Related to pre & post-hooks

Cardinality

5.1. Validations

• This includes Card(min,max) Introduce cardinality restriction typeql#34 (comment)
• An alternative more expressive cardinality validation: Introduce cardinality restriction typeql#34 (comment)
• Please make this one the default: @card(0,1). @Card(1) is RARE.
• 📝This is also related to the "replace" clause and also to order & repetition feature requests

Ordered & repeated attributes & vectors

• 5.2. Being able to store ordered attributes and ordered vectors with potential repetititions
• 5.3. Not only being able to define them but also manipulate them with precision, including shift, insert at, remove at etc: Add vectors as an attribute type. #6327 (comment)

Cardinality reasoning

• 5.4. Right now match -(insert/delete) clauses are run N times (one per match) sometimes it makes sense to store matched values in an array, and run the insertion once for all the array. An example of issue with that is this one Keep relations without players alive until the end of a transaction #6920. It has been "workarounded" keeping relations alive. But there should be a way to store the edges in an array and run it in a single edge connection. Example:

match
$bigBoss isa Person, has name "Ann";
[$everyoneElse] isa Person; # store them as array
not {$bigBoss in [$everyoneElse]};

insert
$myRelation (boss: $bigBoss, employees: [$everyoneElse] isa myrelation, has id 1; #this is run only Once. Not once per item in $everyoneElse as we packed them into an array

Array Functions

5.5.

- IN // Checks whether a value is contained within another value
- NOT IN // Checks whether a value is not contained within another value
- CONTAINS or ∋ // Checks whether a value contains another value
- CONTAINSNOT or ∌ // Checks whether a value does not contain another value
- CONTAINSALL or ⊇ // Checks whether a value contains all other values
- CONTAINSANY or ⊃ // Checks whether a value contains any other value
- CONTAINSNONE or ⊅ // Checks whether a value contains none of the following values
- INSIDE or ∈ // Checks whether a value is contained within another value
- NOTINSIDE or NOT IN or ∉ // Checks whether a value is not contained within another value
- ALLINSIDE or ⊆ // Checks whether all values are contained within other values
- ANYINSIDE or ⊂ // Checks whether any value is contained within other values
- NONEINSIDE or ⊄ // Checks whether no value is contained within other values
- OUTSIDE // Checks whether a geometry type is outside of another geometry type
- INTERSECTS // Checks whether a geometry type intersects another geometry type

Example:

match $people isa Person, has id in [1,2,3,4,12]

Extra

6. Orphan attributes

• Fix the problem, not the symptom: Automatic/periodic cleanup of disconnected attributes #6389 (comment)

7. Composite ids

• Multikey indexes: Feature request: Unique vs Id & composite IDs #6658

8. Replace statement

Simplify replaces by having a way to replace attributes and edges without matching prior attributes or edges

match 
$a isa Person, has id 7;
$management (employee: $a);

insert 
$newManager isa Person, has id 8;

replace
$management( manager: $newManager); #Removes all current managers if any , replaces by new one

9. native geo-spatial types (point, polygon...)

• https://dgraph.io/docs/graphql/schema/types/

[maydanw]

Rule scoping - Sending the list of rules to apply (or exclude) when running a query (an alternative to reduce transmission costs is being able to set tags to rules during their definition and send the tags as part of the query). Example: There is a need for inference but no need for permission check when running this query or no need to check if the information was reviewed and approved as correct (as it is used for training or the user has a curator role, etc.,). These scenarios are where queries run frequently and repeatedly and redundant operations is more apparent.

[maydanw]

Atomic increase and decrease - This allows putting a counter on entities and impacts many algorithms that use probabilities.

[maydanw]

Private/local attributes - Most attributes are local and private to the entity there is no need to search every entity with this attribute value. Allowing defining attributes to be local to the entity simplifies the basic CRUD operation on entities and saves mistakes. Example: Most boolean attributes (e.g., Gender, ActiveUser, ...)

[maydanw]

OnWrite/Mutating Rules - When a certain change in the graph occurs also do this and that. It keeps the data integrity inside the database and does not rely on external application level. If you want, think about the current situation as if I would make a class with data properties but make the process of keeping data integrity in other classes maintained by other groups of developers. It is possible but not the best practice (post-hooks is also a sort of solution here). Eqvivantly when I define my schema I want to define, as best as I can, various change consequences as part of my data layout view and definitions and not worry on every step if a developer is going to know and maintain it.

[maydanw]

I think it would be beneficial to have a "list" and a "set" attributes types with their basic operations management in rust. It means I can define an attribute that is of type list (we can begin with simple values like int and string and maybe later allow structs and entities). To these attributes I would like to do the expected operation (e.g., add, remove, check containment, for list by indexes) in a single query along with the match operation.

This will allow adding metrics and accumulating data attached to the entity or relation and retrieving it without multiple roundtrips and/or complex rules or inefficient driver-side processing. The consequence is lowering the entrance bar for many users and use-cases as well as supplying superior performance in many common use-cases.

Another point I want to add is that one of TypeDB strengths is reducing the gap between the programming logic and the data storage engine and any programming language has these basic types so it is aligned with the vision.

[brettforbes]

Also,

Structs must support time, so that one can input a datetime object, but in the background it gets broken into elements (i.e. integers or day, month, year, hour, min, sec, millisecs, microsecs, nanosecs, timezone). Then all datetime retrieval by default uses these indexes, so time query's, retrievals etc become super fast. Further date maths also uses these elements.

Making time fast, and including date maths, is an essential advantage for winning competitive deals against Elastic search or other db's

[brettforbes]

Also,
Vector support (ala LLM vectors) would also be a huge win, and at the moment it is a negative for TypeDB, as most vendors support it

[sjpritchard]

Vector support (ala LLM vectors) would also be a huge win, and at the moment it is a negative for TypeDB, as most vendors

Can't emphasise this enough. You are missing a HUGE opportunity here. Graph databases are enormously useful combined with LLMs, and TypeDB is nowhere to be seen.