Hypothetical database as part of the Time Traveling features

[chuanqisun]

Hypothetical database allows user to dynamically add/modify relations only for the duration of the query. For example: I'm building a text search engine based on vectors (embeddings) and a knowledge graph. When user submits a query, I want to hypothetically add that user query into the knowledge graph and observe the results from several graph algorithms. Without hypothetical database support, I would have to either construct more complex queries to handle how the query "enters" the graph or I must manually add the relation before the query and remove/invalidate the information after the query. It become labor intensive as the query becomes more complex.

Conceptually, I can see this being a Time Traveling feature because the hypothetical relation could be true at some point in the future.

For those who are using Cozo for formal logic/ontology based AI, this feature can be very handy for hypothesis testing. Here is a hypothetical syntax for hypothetical query 🙃

hypotheticalRows[] <- [["Socrates", "instance of", "human"], ["all humans", "has property", "mortal"]]
?[object] := *db{subject: "Socrates", predicate: "has property", object}, with(*db, hypotheticalRows[])

Here is a concept talk on how hypothetical database query could look like: https://youtu.be/oo-7mN9WXTw?t=2184
Here are some papers on this topic:

• https://ieeexplore.ieee.org/document/9177715/
• https://www.sciencedirect.com/science/article/pii/0304397590900116
• http://www.fdi.ucm.es/profesor/fernan/FSP/Sae13c.pdf

[gisborne]

A generalisation of this idea would be separate theories that can be combined for different purposes. Each query would be executed in a context consisting of a chain of theories, each serving a different purpose. eg system, business logic, user.

eg Cyc calls this "microtheories"

[zh217]

You can open a multi-statement transaction and abort it at the end: https://github.com/cozodb/pycozo#multi-statement-transaction (note: RocksDB engine is highly recommended if you want to use it, otherwise expect lots of locks to get in your way).

Or do you think something more is needed?

[chuanqisun]

Interesting, that seems to be python only feature that the node.js wrapper doesn't support? Also, I probably misunderstood how multi-statement transaction works. I thought unless I commit a query, I won't be able to observe its results.

[zh217]

It's available in the Rust API. It's just a bit messy to write the interop code for it, but if there is demand I can make it available to NodeJS as well. I use python the most so it is there.

Within a transaction you can observe your own writes, including those cascading writes to indices. However, triggers are not called (they are only called on successful commits.)

If you are thinking about WASM, unfortunately I don't know how to do it without threads without tearing everything apart.

[chuanqisun]

Understood. That would be a nice addition to the node.js API. I have a personal interest in consuming the node.js API because our application stack is all javascript. Definitely up to you to gauge the interest from the community and prioritize. Thanks for providing the details and keeping this project going!

[matthiasautrata]

Isn’t this very similar to what diatomic also does: you get your own virtual copy of the database, frozen at the time when you asked for it. If you make changes, you can worry about committing and merging them back later?

…

On May 1, 2023 at 14:00, Ziyang Hu ***@***.***> wrote:  You can open a multi-statement transaction and abort it at the end:  https://github.com/cozodb/pycozo#multi-statement-transaction    Or do you think something more is needed?    —    Reply to this email directly, view it on GitHub  [#85 (comment)],  or unsubscribe  [https://github.com/notifications/unsubscribe-auth/AL52JJTZXD5NVG2CSZLSOA3XD722VANCNFSM6AAAAAAXSABI5I].    You are receiving this because you are subscribed to this  thread.Message ID: ***@***.***>

[zh217]

Yes, in fact for every query in CozoDB you get a frozen world. That's why for every engine except RocksDB you get big locks every time you write, since the other engines do not support true snapshots.

[gisborne]

This may well be a longer-term idea. Alternatively, it might be implemented on top of the basic store.. That said, the idea would be that different theories could be permanently stored and combined as needed. Two namespaces can be merged in an obvious way, the values in one taking precedence over the other in a collision. In the same way, two theories (sets of rules and assertions) can be combined. eg I have a basic system with facts and business rules about my company. On top of this, I put rules concerning the customer interaction for different purposes (“Applications”). On top of that, for each user that interacts with a system, they get their own theory in which I keep their data and eg their password hash. Assertions could appear in multiple models. One could build a security model around this. eg the user interface generation and user interaction library interacts with the system through a theory having very basic permissions. When the user supplies their password, this raises the system’s permissions (visibility, what can be changed) on just that user’s data. A different application, warehouse say, could share that same user’s theory, but combine it with a different business model theory, giving suitable users the ability to place orders and so on. I’m working on such a system now (frest.substack.com <http://frest.substack.com/>). (There is considerably more to it than this.) Cozo is a godsend to my work. I can build this system on top of Cozo, but it would be better if some of this was available in the base system (particularly security guarantees). Perhaps as I get my system working, it can inspire features for Cozo. The advent of Cozo has made my project much simpler. I can’t tell you how grateful I am that you shared this with the world.

…

On May 1, 2023, at 11:00, Ziyang Hu ***@***.***> wrote: You can open a multi-statement transaction and abort it at the end: https://github.com/cozodb/pycozo#multi-statement-transaction Or do you think something more is needed? — Reply to this email directly, view it on GitHub <#85 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAG7X6QI2K7RMESOG3CVHTXD722TANCNFSM6AAAAAAXSABI5I>. You are receiving this because you commented.

[chuanqisun]

Might be related to your use case. I work on a knowledge management system and having the ability to dynamically hide/expose/merge distributed knowledge graphs is a game changer. Some of those knowledge graphs are private to the user and I must deal with network partitions all the time.