go-ipfs high priority issue list

[whyrusleeping]

This is a list of issues that we have identified as being critical to solve in the short term. Our goal is to have a majority of these resolved by the end of the year. Each issue is tagged with one or more owners, the owners arent necessarily the ones who need to fix the issue, they are just responsible for making sure it gets done.

• connection closing (whyrusleeping)
  • integrate connection closing
  • ipfs works fine with a restricted connection count
• websocket FD leaks (stebalien)
  • don't leak file descriptors
• DHT response handler performance (vyzo)
  • bootstrap node dht handler response time never over 1 second
• cbor performance (whyrusleeping)
  • migrate to new, more performant and flexible cbor library (https://github.com/polydawn/refmt)
• gc performance (kubuxu, stebalien)
  • finish and merge @Kubuxus new GC PR
• don't necessarily provide content when you add it (magik6k)
  • provider strategies applied to initial adds
• bitswap session integrations for (cat, refs -r, get, ls) (whyrusleeping)
• flatfs successor recommended for use (magic6k, whyrusleeping)
  • Badger 1.0 issue: Provide stable Badger API dgraph-io/badger#161
• no memory leaks (whyrusleeping, stebalien)
• Swarm hotpaths optimized (stebalien)
• Profile all event handlers and fix slow ones (stebalien)
• Pin with ton of hashes won't start (stebalien)
• Avoid write corruption in go-multiplex (stebalien)
• Fix floodsub go routine leak (stebalien)
• Better Buffering (stebalien)
  • Comment: The motivating issue, metrics, was fixed. All of my attempts to add better buffering have lead to no improvement (so I haven't bothered getting them merged).
• Duplicate Directory Entries
• QmcTybF2LuVrm2W1sar1bqkaNjVmVrNdNpQdomz7sM8UTY (just a reminder for me)
• Share secio buffer pools between connections (stebalien)

[myleshorton]

Just a quick 👍 to this list -- we'll likely have more details soon, but we've been re-integrating IPFS into Lantern @whyrusleeping , and definitely performance and especially CPU issues are the main blockers for up deploying it on a much wider scale.

[whyrusleeping]

@myleshorton thats great to hear! What version are you guys working with now? 0.4.12/13 should be much better off than any of the previous versions, and the next release is going to be a good deal better as well. Please do let us know any issues you run into :)

[kvm2116]

Any estimate on when we expect Bitswap sessions to be implemented?
How can I contribute to speed it along?

[whyrusleeping]

@kvm2116 the initial bitswap sessions code has been implemented, and is in use by ipfs pin add. It improves transfers pretty nicely, but doesnt currently do scheduling of asks between multiple peers.

The next thing we need to do is to add a coalescing layer around the DHT. We did a test run of making the http gateway use bitswap sessions, and it pegged the DHT with requests (sessions 'more correctly' does providers lookups).

Last I checked, @magik6k had started working on that layer. I would ping him for a status update there.

[magik6k]

Part of this (#4333) is done and we wanted to merge it in 0.4.15, I'll rebase it after 0.4.14 releases.

[DonaldTsang]

Post 20 month review: Updates on the checklist?

[Stebalien]

We finished most of them. This list was mostly a todo list for that quarter.

[Stebalien]

We finished most of them. This list was mostly a todo list for that quarter.