Sprint P

[jbenet]

(a bit late in consolidating this here, sorry)

Sprint Goals

Sprint Discussions

See https://github.com/ipfs/pm#sprint-discussion-schedule

Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

@jbenet

• lots of CR
• land signed objects and records
• chain record
• meetup venue
• json-ld / cbor-ld / ipfs-ld tooling
• cube feedback
• find designer for cube
• limits: bandwidth, connections, disk (discussion)
• dag path notation (discussion)
• docker blog post gif

@mappum

• cube frontend - initial UX mockups/previews
• cube frontend - React
• cube backend - file accounting
• cube backend - IPFS daemon integration (adding, pinning, unpinning)

@lgierth

• land monitoring Monitoring infra#44
• submit battlemesh talks
• discover peers from cjdns WIP: discover peers from cjdns kubo#1316
• spec out relevant cjdns internals Integration with cjdns notes#14
• migrate IRC bots to solarnet Migrate IRC bots to solarnet infra#33
• investigate goroutine spikes Spikes in the number of goroutines kubo#1470

@whyrusleeping

• fix too many open files issue with bitswap/flatfs respect open fd limit kubo#1456
• make ping its own protocol
• fix pinbot
• ipfs-ld parser
• faster daemon shutdown
• rebase dev0.4.0
• ping as its own protocol
• fix path parsing bug
• bitswap writeblock retries
• improve test reliability

@Heems

• programmatically set link settings between nodes in bssim based on real latency/bandwidths
• more/better graphs in bssim
• make pipeline configurable from 1 command/config file

@diasdavid

• abstract-stream-muxer
  • Finish heavy duty tests
  • Polish and Publish node-spdy-stream-muxer - https://github.com/diasdavid/node-spdy-stream-muxer
  • Make node-multiplex-stream-muxer - https://github.com/diasdavid/node-multiplex-stream-muxer
  • Report multiplex under 'mega' stress conditions (multiplex hangs with a lot of load max-mapper/multiplex#25)
  • ~Timing reports
• node-ipfs
  • swarm
    • attach Stream Muxer through interface
    • make Identify work with protobufs
    • send proper observed addrs
    • write swarm spec (Protocol Spec Update specs#18)
    • avoid multiaddr explosion (https://github.com/diasdavid/node-ipfs-swarm/issues/1)
  • dht
    • ping
    • attach swarm for pings
• Other
  • travis-ci all the things

@rht

• need 'Add log' for ipfs add kubo#1353 blocks versioning
• Collecting Process Metrics kubo#1418 stats
• crypto uses rand.Reader directly. kubo#143 ()

@wking

• support symlinks (discussion)
• open PRs
• example or blog post with runC (?)

@chriscool

• Fix error handling jbenet/ipfs-paste#3

Backlog

Anyone can pick these up:

• rabin fingerprinting
• limits: bandwidth, connections, disk (implementation)
• implementing dag path notation on bitswap
• jsonld protoc equivalent (with cbor and/or ubjson)
• dht records + ipns
• support symlinks (implementation)
• memleaks, goroutine spikes, gateway "lock-ups"

[jbenet]

I've spent a bunch of time looking for golang cbor encoders. this is the most promising: ugorji/go#86 -- but isn't perfect yet. i made this tool to test them: https://github.com/jbenet/go-json2cbor/

so we can try the ipfs-ld thing, but not sure cbor's going to work reliably this week. i think cbor is the right choice so far (it's the closest thing to json).

ubjson is worse-- there only seems to be one golang implementation, and no idea if it's complete: https://github.com/imipolexg/go-ubjson -- we could try it with {bson, bjson, msgpack} but those all have problems too.

[rht]

Is ipfs-ld meant to augment https://github.com/ipfs/go-ipfs/blob/master/merkledag/node.go? Regardless of current implementation detail, what is the recommended canonical reading here?

[rht]

One use case of json-ld's RDF is to add standardized contexts to the premises of a proposition in a proposition tree? From e.g. http://ethics.spinozism.org/graphs/js/ethica5.json.
Though the example doesn't require semversioning (or could be since there are versions of the manuscript) nor content-addressing (or could be for highly composite citations/footnotes, but this can be handled by linked data too).

(but content address is already a kind of semver, ipns is another, but for major versions and "manual" naming, e.g. "Diseased Newt")

[daviddias]

Sprint Update:

abstract-stream-muxer:

• Made heavy duties test, which lead to find some bugs in spdy-transport (now fixed) and a load problem in multiplex (still unresolved, but reported multiplex hangs with a lot of load max-mapper/multiplex#25)
• Make, polish and publish node-spdy-stream-muxer - https://github.com/diasdavid/node-spdy-stream-muxer
• Make, polish and publish node-multiplex-stream-muxer - https://github.com/diasdavid/node-multiplex-stream-muxer

ipfs-swarm:

• Make ipfs-swarm to use stream muxer interface (for easy change of stream muxer)
• Make Identify use protobufs
• Send the right observedAddr (Node.js IPv6 default representation workaround)
• Write swarm spec (complement the network spec - Protocol Spec Update specs#18)

ipfs-routing:

• good discussions around how the DHT is implemented (or will be implemented)

community:

• push for the Seattle IPFS meetup
• push for the Portland IPFS meetup

other updates

• Cory Benfield started an extension for HTTP/2 P2P - https://www.ietf.org/internet-drafts/draft-benfield-http2-p2p-00.txt
• dignifiedquire is interested in helping out with node-ipfs. This is cool because it forces me to be more verbose, creating issues for tasks, which might lead for other contributors

[jbenet]

• lots of CR.
• bad news is, i still haven't landed the records. (sorry whyrusleeping)
• but the good news is i have ipfsld working with cbor! \o/ which would cause the records to be rewritten - anyway, so that's good.
• i'm making a bit of tooling for making schemas, though the base one already works :)
• worked with mappum on cube things
• figured out the process limits discussion
• figured out more about dag path stuff (subsets of XPath, JSON Pointer, JSON Select). still need a decision here.
  worked through dht/routing/network interfaces with whyrusleeping and daviddias
  still need to finish CRing daviddias's network spec PR.
  i spent way too much time messing with asciinema, but the good part is we now have asciinemas of ipfs tooling that get rehosted to ipfs: http://gateway.ipfs.io/ipfs/QmTDAsNvNGAWmJ9Eu3NRFhQxW1Tc8eYBd2KG8UzDqeZWaP (will push out the tool i made later today), so i'll be making a lot of them this week.

[ghost]

• monitoring is finally ready for service. happy to answer any questions re: setting up access
• submitted battlemesh talks re: ipfs, cjdns -- also travel planning (bbq in vienna with @dch of ppsp)
• migrated pinbot to solarnet
• started rebuilding gateway hosts. pluto got resized and is rebuilt. plenty of little fixes for bootstrapping vanilla hosts (PR incoming)
• no news on the goroutine spikes yet, same for cjdns discovery

[jbenet]

etherpad for next week https://etherpad.mozilla.org/OsdMG3LCWe