Live community hangout #3 - Jan 28th, 10AM PST

[mafintosh]

Let's do another community hangout and discuss the progress of Dat beta.
Recently we've written api docs for the upcoming JS api.

• Dataset/DB api
• Conflict/merge api
• blobs api

Talking points:

• go over APIs
• breaking changes (if any)
• talk about expected/supported use cases
• conflicts - should you merge before pushing?

raised questions:

• schema vs binary types

Implementation of this is under progress but it would be great to showcase/discuss this with the community.

Leave your questions in the thread below, in our gitter web chatroom or tweet at @dat_project

Agenda:

• Dat Beta API discussion. (JS and CLI?) (1 hour)

We'll be using Hangouts On Air.

Youtube link (to watch) https://www.youtube.com/watch?v=4J9CSoQ-4-0
Gitter link (chat/ask questions): gitter web chatroom

Note that only Dat team members and others on the agenda will be invited to join the Hangout so they can broadcast voice and video. Hangouts only supports about 10 people max, but the youtube stream can support an unlimited amount of viewers.

[max-mapper]

One bit TODO still is more fleshing out of what the command-line API is going to look like for these new merging/conflicts features.

E.g. when you dat pull and you enter conflict mode, what does it look like on the CLI? How much of git's merge resolution flow can we copy, but what new approaches are possible since everything in dat has a known schema?

[max-mapper]

feedback from @substack via IRC:

substack do you really need to have conflicts?
substack there are just sometimes multiple versions of a dataset
substack just like in real life
substack "Conflicts must be handled (or aborted) to complete the pull." :(
substack ogd: can you just make pulling separate from merging? when you pull you just fetch data and have multiple copies of a data set
substack and then as a separate step you can merge two documents together
mafintosh substack: and have dataset.get(key, cb) return multiple versions?
substack sure
substack but mostly, the ui should just be able to handle that
substack and then you can merge two forks as an explicit step with a diffing/merging tool
substack instead of everything being magic and implicit when you fetch data
substack conflicts are an extreme anti-feature
mafintosh substack: 2 sec - digesting that you are saying
mafintosh substack: so if i pull dataset d1 from you i would get a fork locally called d1.substack or something that contained your changes. if i want to i can now merge d1.substack into d1?
substack sure something like that
substack and then you can make edits directly to the d1.substack fork or to d1.prime (d1.mafintosh)
substack or merge them into a single fork
substack the problem I see with having pulls shift you into conflict state is that conflicts are impossible to solve in the general case
substack so it's usually going to be messy to merge them
substack and that will take time
substack and being in a conflict state means you've got to drop EVERYTHING to go and fix it
mafintosh true
substack and you can't really do anything else until the conflict is dealt with
substack that is how pretty much every historical database works and it's terrible
substack but the solution is simple: just decouple merging from pulling
mafintosh substack: so if you push changes to me its my responsibility to merge them (at some point in the future)?
substack that's one way
substack it could work like a pull request where you decide to merge a contribution as an explicit step
substack but I think it's very important, especially if non-experts are using this software, that when you push or pull software, you should always get the software
substack and then somebody who is "good at computers" can merge the forks
substack s/software/data/g
mafintosh substack: awesome feedback, thanks
mafintosh substack: the current draft is basically just trying to mirror how git does things

[max-mapper]

We'll be live in a few

[max-mapper]

Check out video here https://www.youtube.com/watch?v=4J9CSoQ-4-0