Data structures and tools for building a codepen-like project w/ Dat & Beaker

[flpvsk]

Hi, I'm considering building a project in a p2p-first way. Dat & Beaker seem like a good way to go, but I can't wrap my head around how to organize the data structures and which tools to use. We briefly talked about it with @mafintosh, @pfrazee and @taravancil at the Beaker meetup in Berlin. The recommendation was to write down my requirements in an issue, so here we are.

Project summary

I'm building a codepen (jsbin, glitch) -like project but for short pieces of music. I'll call those pieces of music "chops" for short.

The goal of the project is to help people practice, collaborate on, discover music.

People will be able to record, save and share their chops with bandmates or publicly. People who have access to a previously made chops can then fork / remix them.

My intention is to bring the project to production. I believe p2p has potential and I'm happy to try and do it with Dat and Beaker. If it doesn't work I'll just fall back to whatever mongo/firebase.

Core features

• A lot of it is about practicing by yourself, so

  • As a user I'd like to be able to record and store pieces of music and some meta data locally, so that I can come back to it at any time;
  • I'd like to have access to all of my chops on all of my devices, so I can both listen to and change them;
  • I'd like to be able to go back and forth in the history of the chop;
  • I'd like to be able to create new chops based on the existing ones aka remix / fork;

• Some of the features are about collaboration:

  • I'd like to share some of the chops with my bandmates / friends;
  • Chops that are shared with me, I'd like to be able to fork and remix;

• Some of it are social features:

  • I'd like to be able to browse, search and remix chops other people shared publicly;
  • I'd like to be able to comment and "like" other people's chops;
  • I'd like to be able to "follow" people, to get there chops in a chronologically ordered feed;

Supported devices

• Should work in all major browsers on desktop and mobile;
• In the future I might want to write native apps;
• I don't mind having a server to do replication / help with content discovery;

Questions

• I'm curious about how to organize the data structures to make this usecase work in a p2p world;
• What tools can I use to build this? @mafintosh mentioned that all the tools needed for this already exist in the Dat ecosystem, what are they?

Contribution

• If it is possible and it won't take forever to build, I'm happy to try build it and tell my experience afterwards in a form of docs/blog posts.
• I'll think about whatever umbrella-tools might have improved my workflow on the way and either build or propose them;
• I'm already working on an episode about p2p tech for code podcast, so I'll definitely share my experience there.

[pfrazee]

Broadly speaking, you can handle most of this using the DatArchive API. It'll be up to you to decide whether each Chop should have its own Dat, or if you want to give each user a "Profile Dat" which they put all their chops in.

For the social features, you might benefit from using WebDB, which is what we used in Fritter. You can see how that works by looking at libfritter.

The hard part is handling other browsers. I'll write up a longer response in a bit with your options there.

[pfrazee]

@flpvsk So @RangerMauve has been working on dat polyfills and gateways, and he just opened an issue on fritter talking about it. See #1018. I'd read/watch that issue to see how to support other environments than Beaker.

[flpvsk]

@pfrazee thanks for the recommendations! I'm trying them out in the next several days.

One thing that is still a mystery to me is how can a person use several devices to read/write their data?

[pfrazee]

@flpvsk that's a protocol feature we call "multiwriter" and it's being worked on now