Replicate foaf

[mixmix]

I've been seeing a intermittent bug:

• A connects to and friends B (pub)
• C connects to and friends B (pub)

expect:

1. ✔️ A's friend graph to now include C at hops: 2
2. ✔️ A to call replicate.request to B asking for C's history
3. ❓ A to receive C's history (via B)

I've got tests around (1), (2) and these happen reliably.
(3) fails intermittently 😢

---

In production we've seen that if A restarts, then they correctly pull C's history from pub (B)

[mixmix]

Debugging

I've followed this down into the stack:

• logging in ssb-replicate
  • I can see that even when failing replication is being correctly set up (A is requesting data about C from B)
  • I've not seen any errors thrown / logged anywhere
• reading into ssb-db
  • cannot see anything odd about createHistoryStream
  • perhaps could put logging around createWriteStream to see if there the steam is being closed?

[arj03]

Been looking at this module trying to convert it to work with db2. I was thinking that all of this replicate functionality doesn't really belong in this module. I would really like to rip out auth and replicate and maybe put in into separate files, preferably their own modules. Because right now tracking down the dependency between replicate, ebt, friends etc is like walking into the bermuda triangle.

[mixmix]

Yep, at the very least I reckon putting them in different files and giving control over them via config options...

It's tricky though because typically when I include ssb-friends it's actually just for the replicating functionality. After all what is this module about other than that?

Maybe it should be

ssb-friends = ssb-friends-graph + ssb-friends-replicate

Need to think about this for ssb-tribes too, as I have auto replication of all people you're in a private group with.... which is an abuse vector i need to close.... Sigh

[arj03]

The db2 PR does a bit of cleanup. My hunch is that once we start on the partial replication things will get a bit more tricky, so just calling replicate will not really be enough. We need a better machinery to figure out what to replicate and in this I see friends as just the graph component (I like that name). I'll circle back to this PR and help review once the other PR is in a good shape.

[mixmix]

The more p2p friendly features you add, the trickier it gets!

[mixmix]

hey @staltz I think you should move this test over to your schedular, then we can delete this PR