Support for foreign keys

[teohhanhui]

I'd like to be able to reconcile / hydrate a related object as a foreign key in the document.

Currently, I'm running into a roadblock in the API:

In Reconcile::reconcile I'd need to be able to access the doc, to be able to determine the ObjId for the related object, in order to call reconcile_prop for it.

For example:

fn reconcile<R: Reconciler>(&self, mut reconciler: R) -> Result<(), R::Error> {
    let mut m = reconciler.map()?;
    // ...
    let Found(related_key) = self.related.key() else {
        // ...
    };
    let related_key = related_key.to_string();
    m.put("related", related_key)?;
    // no access to `doc`, and need to get `related_obj_id` from `doc` as well
    reconcile_prop(&mut doc, related_obj_id, &*related_key, &self.related)?;
    // ...
    Ok(())
}

[alexjg]

I'm not quite sure what you're imagining, is it something like the following data model:

struct Project {
    owner: User,
    name: String
}

struct User {
    id: String,
    name: String,
}

And you would like to reconcile this into a document with something like the following structure:

{
    "projects": {
        "id1": {
            "name": "Some project",
            "owner": "user1"
        }
    },
    "users": {
        "user1": {
            "name": "Some user",
            "id": "user1"
        }
    }
}

I.e. the reconciliation for the user within the Project references the user by ID rather than including the whole object.

[teohhanhui]

the reconciliation for the user within the Project references the user by ID rather than including the whole object.

Indeed.

[alexjg]

So you can achieve this on reconciliation with the current API:

use automerge_test::{assert_doc, map, list};
use autosurgeon::{Reconcile, Reconciler, reconcile_prop};

#[derive(Reconcile)]
struct Project {
    #[autosurgeon(reconcile="reconcile_userid")]
    owner: User,
    name: String
}

fn reconcile_userid<R: Reconciler>(u: &User, mut reconciler: R) -> Result<(), R::Error> {
    reconciler.str(&u.id)
}

#[derive(Clone, Reconcile)]
struct User {
    id: String,
    name: String,
}

fn main() {
    let mut doc = automerge::AutoCommit::new();
    let users = vec![User{
        id: "user1".to_string(),
        name: "some user".to_string(),
    }];
    let projects = vec![Project{
        owner: users[0].clone(),
        name: "some project".to_string(),
    }];

    reconcile_prop(&mut doc, automerge::ROOT, "projects", &projects).unwrap();
    reconcile_prop(&mut doc, automerge::ROOT, "users", &users).unwrap();

    assert_doc!(
        doc.document(),
        map!{
            "projects" => { list! {
                { map! {
                    "name" => { "some project" },
                    "owner" => { "user1" },
                }}
            }},
            "users" => { list! {
                { map! {
                    "id" => { "user1" },
                    "name" => { "some user" },
                }}
            }}
        }
    );
}

This is great but you can't Hydrate into such a structure. The problem is that you would not be able to construct a User from the ID when constructing the owner property. We could extend the Hydrate trait to allow this, but I'm not sure that this is the right place to do things. Your data model would end up with two copies of the given user in it, which seems not at all like what you would like. It seems to me more likely that you would want to have something like a data transfer object for going from your domain model to the normalized representation in the automerge document. What do you think?

[teohhanhui]

So you can achieve this on reconciliation with the current API

But this means no transitive / cascading persistence, which I guess might not be such a problem...

It seems to me more likely that you would want to have something like a data transfer object for going from your domain model to the normalized representation in the automerge document.

That of course makes the whole thing less ergonomic to use, and we still end up kicking the problem to the application layer, which seems worse to me...

[alexjg]

My initial design goals for this library were not to support arbitrary mappings of domain logic to an automerge document, but to make expressing the semantics of automerge data types in Rust easy. Much as serde helps map data types to and from serialized formats. In particular I have tried to follow the principle that if you have a type that implements Reconcile you can reconcile it with whatever property you like in the document (much as you can stick anything that implements Serialize anywhere you like in a JSON object) and likewise if you have a type that implements Hydrate you can pull it out of any property in a document, in neither case do you need to know anything about the shape of the whole document. I think this is a very useful compositional property.

It seems to me that what you are interested in are properties of the entire document rather than of individual types. I think it would be interesting to explore how we might express those but my gut says that it would be better to think about doing that in some other trait (I'm imagining a function similar to reconcile but which takes a different trait as an argument which describes how all the types in a document are constrained - much like Sea ORM's RelationTrait trait). One thing that might be interesting in that vein is that in order to support incremental updates (by way of a Patchable trait) I am splitting the key related functionality of Reconcile out into it's own trait called HasKey, which I think would also be useful for the constraint descriptions you are talking about. I'm interested in working on this but my current priority is getting incremental updates working.

What do you think? Do you have an idea of what an API for describing these constraints should look like?

[teohhanhui]

Thanks for your insight.

What do you think? Do you have an idea of what an API for describing these constraints should look like?

Unfortunately I don't think I'm at the level of understanding to be able to give any constructive input here.

[teohhanhui]

If I understand correctly, there are plans (or it's at least being considered) to allow storing the ObjId in the hydrated object struct, right?

If so, I wonder if it's possible to reuse that ObjId to implement support for foreign keys? i.e. in my example above, if the self.related object has an ObjId (perhaps identified through a HasObjId trait), and we have let's say #[foreign_key] on this related field, then we can know to reconcile this as (some representation of) a foreign key in the document.

Or perhaps would it even be possible to support foreign keys in automerge itself, i.e. ObjId aliasing? (No idea if it can be done in a CRDT-safe way...)