Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
RFC: Generic Record Sync #658
rfk left a comment
Thanks for sharing out out, @thomcc! I probably need to read over it a few times to really understand, but left a couple of preliminary comments based on my first read.
In general, I think it's fine to ask the server to be smarter here rather than trying to fit all the changes into the existing server API, we need to be in a position where we're OK evolving our server infrastructure just as much as we do for our client infrastructure.
mhammond left a comment
I think this proposal is very solid and has obviously taken much thought and effort. It's also a very important discussion for our team - thanks! And sorry for the delay - I needed time to make my thoughts coherent - hopefully I haven't failed in that :)
However, I'm a little skeptical that the entirety of this will offer net value. I'm concerned it will implement something that has more complexity than we need, and even then, may fail to handle edge-cases we find outselves facing. I suspect Mentat would have put us in the same boat too, but that's a different conversation. IOW, I fear more pain than gain for our foreseeable future.
Don't get me wrong - I do think that some of the ideas expressed here are valuable - particularly the local schema ideas. A simple way of "versioning" a schema and avoiding custom (but boilerplate) schema upgrades has value, as does a library which makes the implementation of new simple stores almost trivial.
My skepticism is mainly around server-based-data-driven merging and duping, particularly in a world where we end up with a single Rust implementation everywhere. In particular:
With the correct library to leverage, I think it would be fine to require collection-specific code in some cases - most notably deduping and merging. Concepts like
I guess a reasonable tl;dr of my position is:
Again, thanks for starting this valuable discussion and for such a well thought out RFC.
I've just pushed up a major change to this based on a bunch of thoughts I've had since the last time I wrote it. The expanded commit message at 1e2ba10 says all of them, but essentially:
@mhammond Responded inline:
For something like this, there's a balance to be struck between ease of implementation and ease of use. I agree that mentat is probably too far on the side of implementation complexity, but I feel that this design is much simpler than Mentat. I expect it to be around the size of pre-bookmarks places, maybe on the order of 10kloc or so. (Famous last words, but I'd be very surprised if it were larger than places-with-bookmarks)
I also think that this design document makes it come off as a bit more complex than it would end up being. That's mostly because I've made an effort to spell everything out explicitly and handwave minimally. I also wrote some of the design concurrently with a rough WIP prototype, which I'll try to push up eventually.
In my updated version of this, I've been more clear about things we're not doing, so maybe it's easier to see why it's much simpler than mentat.
I've tried to come up with something that is flexible enough to support the use cases we've seen or wanted in practice. Most of these are too complex to do with naive roundtripping.
I broadly agree that we could do it, but I don't think having the sync team implement these has been successful. As far as I can tell, teams are very reluctant to talk to us about this, and tend to just use easy off the shelf solutions instead (which also means they don't have to wait on us to get started).
I agree. That's more or less explicitly one of the main ideas in remerge. We want to be able to break old clients, without breaking new ones. This is why there are two version numbers (the current schema version and the required schema version)
This is probably a reasonable complaint, and does seem likely that new features we add for teams would be , but given that in a custom-code world all the code is used exactly once, I don't think this is that much of a problem so long as the feature is vaguely sensible.
The alternative is either have us write it (which I know you're arguing, but see below on why I don't think that's good), or to allow some sort of programability to this, which would make this substantially more complex, to the point where I'd probably flip back to your side on it being not worth the effort, (not to mention the security concerns).
Additionally, given how reluctant teams are to talk to us now, it's possible that the only thing we'd be missing are stuff like blatantly missing features.
I think there still relatively easy to support in a generic way like this. The main cost relative to the specific way is performance, not the amount of effort it takes IMO.
Yeah, logins is in a lot of ways a better candidate for it than forms, which is unlikely to do much evolution. That said, both would work.