Sprint January 19th

[RichardLitt]

Sprint January 19

Moved from the 18th to the 19th.

Sprint Goals

Sprint Discussions

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour	Lead	Time (PDT - UTC/Z - CET)	Pad
infrastructure	Lars Gierth	18:45Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-infrastructure
libp2p	David Dias	19:00Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-libp2p
js-ipfs	David Dias	19:15Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-js-ipfs
go-ipfs	Jeromy Johnson	19:30Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-go-ipfs
apps on ipfs	Juan Benet	19:45Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-apps-on-ipfs

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

• Add your tasks below in a comment, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

[whyrusleeping]

@whyrusleeping's TODOs:

• debug random multistream goroutines...
• fix ipfs files 'deadlocks'
• private networks
  • implement 'encrypted transport' thing
• thin out go-libp2p deps
  • go-net needs to die
• directory sharding
• node aliases (should be easy to add to config)
  • keep map in IpfsNode object, maybe in resolver too

Down the road:

• change importer to name file links
• merkledag insertInOrder

[daviddias]

David Dias TODO

• js-ipfs
  • js-ipfs (cli + http api)
    • block
      • put
      • del
      • stat
      • update node-libs-browser to ^1.0.0 webpack/webpack#1930 + update node-libs-browser to ^1.0.0 webpack/webpack#1931
    • init
    • CR+M Run core tests in the browser js-ipfs#55 (browser tests)
    • Debug, figure out why Buffers coming from local storage were bigger and then give support to binary data to local-storage-blob-store support binary data fbaiodias/local-storage-blob-store#1
  • js-ipfs-repo
    • Work with Francisco on getting IPFS repo to work with idb Running the tests in the browser js-ipfs-repo#26
  • js-ipfs-merkle-dag
    • Be able to successfully read (and write) MerkleDAG nodes that were stored into an IPFS Repo through go-ipfs, in JavaScript https://github.com/vijayee/js-ipfs-merkle-dag/pull/6
    • Documentation https://github.com/vijayee/js-ipfs-merkle-dag/pull/7
  • js-ipfs-data-importing
    • Create a protobuf version of the layouting (similar to what go-ipfs does) (a good test is creating it, adding to the repo and then let go-ipfs use it <- this will be so sweet)
    • Extrapolate learnings and merge them into the spec - DEX specs#57
• api documentation
  • Work on Docs with @RichardLitt
  • Finish with @RichardLitt Add Group add ipfs-inactive/http-api-spec#17
• js-ipfs-api
  • docs update Update README Docs + remove unmaintained examples ipfs-inactive/js-ipfs-http-client#191
  • CR+M fix: Always return the underlying request object ipfs-inactive/js-ipfs-http-client#177
  • Update README Docs + remove unmaintained examples ipfs-inactive/js-ipfs-http-client#191
  • add remaining version tests ipfs-inactive/js-ipfs-http-client#192
  • block stat + tests + readme update ipfs-inactive/js-ipfs-http-client#193
  • add object.new and a test for it + notes to the readme ipfs-inactive/js-ipfs-http-client#194
• libp2p
  • utp
    • ucat with utp-native
• npm on ipfs
  • Debug and solve with @whyrusleeping latests issues on files API to get it completely ready
• specs
• extra
  • R Added etherpad links #82
  • R Added note about usable prods awesome-ipfs#40
  • CR CR multiformats/js-mafmt#1
  • R Fix links awesome-ipfs#42
  • CR+M feat: Upgrade deps, including ipfs to 0.3.11 js-ipfsd-ctl#47
  • CR+M Chore/updates ipfs-inactive/js-ipfs-http-client#196
  • R Added repos tools community#96
  • R https://github.com/ipfs/webui/pull/217/files
  • check everyday https://github.com/issues?utf8=%E2%9C%93&q=is%3Aopen+org%3Aipfs+label%3A%22needs+review%22
  • CR+M Added the sync #85

[hackergrrl]

@noffle TODO

Keeping my list on the short side, since I expect ArcticJS (and my cold) to keep me pretty busy.

• go-ipfs
  • finish Removing a pin of a hash that does not exist should fail immediately kubo#2155
  • Mount only /ipfs/ kubo#2187
• ipget
  • create distributions entry

[whyrusleeping]

I think due to extenuating circumstances, we only really need 15 minutes for each of the sprint hangouts. Thoughts? @RichardLitt @diasdavid @lgierth @noffle

[dignifiedquire]

sounds good

[RichardLitt]

Works for me.

[dignifiedquire]

Can we start 15 minutes later than with the apps hangout please?

[RichardLitt]

Yes.

[RichardLitt]

How about:

(See above)

[ghost]

yeah shortening is cool for me

[RichardLitt]

Cool. Changed times. See first comment.

[RichardLitt]

To Dos

• Sprint Management
• Write the weekly roundup
• IPFS API
  • Add a link to the api from the specs repo
  • Add curl commands to all method descriptions
  • Check all responses have maximal entries
    • Log any errors in go-ipfs
  • Standardize formatting for ipfs --help entries, basing on 0.4
  • Open an issue about sharness testing.
  • Open a feature issue about having multiple language implementations for curl requests

[dignifiedquire]

@dignifiedquire To Dos

Work

Webui

• Extract the globe
• Finish and merge "remove jquery"
• Finish and merge "ES6/7 transition"
• Remove local storage
• New design (Photoshop)

Distributions

• Ship it
• Review blog post
• Fix download hierarchy
• Add better links

Learn

• Go

[jbenet]

@jbenet TODO

• finalize IPLD pathing issue
• finalize IPLD protobuf compat issue
• finalize IPLD spec
• finish transcribing seminar talk into blog post
• CR more things

[RichardLitt]

This Week

• Sprint Management
• [~] Write the weekly roundup
• [~] IPFS API
  • Split into current functions master branch and expected spec branch.
  • Add a link to the api from the specs repo. PR specs/pull/69
  • Add curl commands to all method descriptions
  • [~] Check all responses have maximal entries
    • Done for ipfs/add.
    • Log any errors in go-ipfs
  • [~] Standardize formatting for ipfs --help entries, basing on 0.4
    • Added docs-styleguide.md to ifps/community.
    • [~] PR docs-styleguide.md links where needed.
  • Open an issue about sharness testing. http-api-spec/issues/23
  • Open a feature issue about having multiple language implementations for curl requests

Extra

• CR+M project-repos/pull/36
• CR+M js-ipfs-api/pull/191
• CR+M project-repos/pull/37
• CR go-ipfs/pull/2231
• Added 'needs review', 'difficulty:*' labels to every repo. community/issues/94, PR community/pulls/95, community/pulls/96
• Created RichardLitt/gh-description to help with IPFS standardization, eventually.
• Added log to go-ipfs about C-c. PR go-ipfs/pull/2243
• Capitalized and period-appended logs in go-ipfs. PR go-ipfs/pull/2244

[hackergrrl]

This Week

As expected, ArcticJS and being sick kept me pretty busy! No blockers.

• go-ipfs
  • finish Removing a pin of a hash that does not exist should fail immediately kubo#2155
  • Mount only /ipfs/ kubo#2187
• ipget
  • create distributions entry

[whyrusleeping]

DONE (week of Jan 18):

• fix go-ipfs-api object patch code
• wrote a basic multicast package (https://github.com/whyrusleeping/multicast)
• distributions fixes
• ⭐ wrote 'just a dht' binary (https://github.com/whyrusleeping/ipfs-dht)
• wrote simple ipfs keygen program (https://github.com/whyrusleeping/ipfs-key)
• arcticJS
  • met richardlitt and noffle in person finally
  • hackathon with mafintosh feross wa7son and others
  • ⭐ gave a talk (https://www.youtube.com/watch?v=fVSu_t0CsWA)
• gx work
  • gx update was doing odd things, fixed that
  • added '.gx' dir for storing repo info
  • more tests (up to four test files now!)
• learned metric ass-tons about linux network routing
  • building a distributed networking sim tool
  • dude, wow, networking. hard shit.
• started a 'generic progress' lib, frustrated with how difficult progress bars are
• made progress on ipfs files deadlocks, registry mirror is running now

[ghost]

@lgierth sprint update

Really messy sprint on my end, but the dev040 work is largely done, and provsn works nicely. jbenet had to clean up after me today, and wrote a post mortem.

• dev040
  • deploy dist.ipfs.io
  • dev040 gateways/bootstrappers
    • ⭐ that means all of ipfs.io is now served by both v03x and v04x, whatever responds faster. v03x.ipfs.io and v04x.ipfs.io themselves are exempt from this.
  • dev040 pinbot
  • CR Make dns resolve paths under _dnslink.example.com kubo#2191
• infrastructure
  • use provsn for deploying ipfs
    • more about provsn in the coming days
  • ipv6
  • wildcard cert
  • wget benchmark

[daviddias]

David Dias Check In

• js-ipfs
  • js-ipfs (cli + http api)
    • block
      • put
      • del
      • stat
      • update node-libs-browser to ^1.0.0 webpack/webpack#1930 + update node-libs-browser to ^1.0.0 webpack/webpack#1931
    • CR+M Run core tests in the browser js-ipfs#55 (browser tests)
    • Debug, figure out why Buffers coming from local storage were bigger and then give support to binary data to local-storage-blob-store support binary data fbaiodias/local-storage-blob-store#1
  • js-ipfs-repo
    • Work with Francisco on getting IPFS repo to work with idb Running the tests in the browser js-ipfs-repo#26
    • Later we dropped IndexedDB in favor of localStorage, see https://github.com/xicombd/local-storage-blob-store, now fixing some issues when it comes about storing binary data, see IPFS Repo JavaScript Implementation js-ipfs#36 (comment)
  • js-ipfs-merkle-dag
    • Be able to successfully read (and write) MerkleDAG nodes that were stored into an IPFS Repo through go-ipfs, in JavaScript https://github.com/vijayee/js-ipfs-merkle-dag/pull/6 (🌟 MerkleDAG Impl interop)
    • Documentation https://github.com/vijayee/js-ipfs-merkle-dag/pull/7
• api documentation
  • Work on API Docs with @RichardLitt
  • Finish with @RichardLitt Add Group add ipfs-inactive/http-api-spec#17
• js-ipfs-api
  • docs update Update README Docs + remove unmaintained examples ipfs-inactive/js-ipfs-http-client#191
  • CR+M fix: Always return the underlying request object ipfs-inactive/js-ipfs-http-client#177
  • Update README Docs + remove unmaintained examples ipfs-inactive/js-ipfs-http-client#191
  • add remaining version tests ipfs-inactive/js-ipfs-http-client#192
  • block stat + tests + readme update ipfs-inactive/js-ipfs-http-client#193
  • add object.new and a test for it + notes to the readme ipfs-inactive/js-ipfs-http-client#194
  • CR+M Chore/updates ipfs-inactive/js-ipfs-http-client#196
• libp2p
  • utp
    • ucat with utp-native
• npm on ipfs
  • @whyrusleeping just fixed it :D Debug and solve with @whyrusleeping latests issues on files API to get it completely ready
• extra
  • R Added etherpad links #82
  • R Added note about usable prods awesome-ipfs#40
  • CR CR multiformats/js-mafmt#1
  • R Fix links awesome-ipfs#42
  • CR+M feat: Upgrade deps, including ipfs to 0.3.11 js-ipfsd-ctl#47
  • R Added repos tools community#96
  • R https://github.com/ipfs/webui/pull/217/files
  • check everyday https://github.com/issues?utf8=%E2%9C%93&q=is%3Aopen+org%3Aipfs+label%3A%22needs+review%22
  • CR+M Added the sync #85

[dignifiedquire]

@dignifiedquire Sync Update

Work

Webui

• [~] Extract the globe and https://github.com/ipfs/connections-globe
• Finish and merge "remove jquery" ⭐
• Finish and merge "ES6/7 transition"
• Remove local storage
• [~] New design (lots of bad drafts, nothing good yet)
• [~] 0.4.0 File Browser
• Update hjs-webpack to 7.0.0
• CR Polish localization ipfs-webui#219

Distributions

• Ship it ⭐
• Review blog post (waiting on @RichardLitt to write it)
• Fix download hierarchy 1 2
• Add better links
• Add about section
• Answering feedback Trying it out :) - Feedback distributions#45
• Review feedback issues from @jbenet

JS IPFS API

• Cleanup dependency PRs on ipfs/js-ipfs-api
• CR
  • add remaining version tests ipfs-inactive/js-ipfs-http-client#192
  • block stat + tests + readme update ipfs-inactive/js-ipfs-http-client#193
• Update to 0.3.11 ⭐
• Open Issue review and PR review
• Add tests for ls

JS IPFSD CTL

• Update to 0.3.11 ⭐

Misc

• CR up-for-grabs label community#94
• CR Added IPFS #3 newsletter#14
• CR Running the tests in the browser js-ipfs-repo#26

Learn

• Go
• [~] Learn about Redux

Teach

• introduce the dark magic of bluebird to @RichardLitt

[jbenet]

@jbenet TODO

• IPLD
  • finalize IPLD pathing issue -- didnt finish it but made examples repo which helps. i think i've settled on one pathing option.
  • finalize IPLD protobuf compat issue
  • Made https://github.com/ipfs/ipld-examples to see pathing uses
• finish transcribing seminar talk into blog post
• multihash base discussion: First byte used to represent the encoding or settle on base58bitcoin-multihash? multiformats/multihash#14
• surfaced notes for POST mekarpeles - jbenet meeting notes for archival ipfs-inactive/POST#6
• bugfix / solve gateway downtime
• Review
  • CR + M replace sha3 name with sha3-512, add other SHA-3 functions multiformats/multihash#20
  • CR + M Feat/modular multiformats/go-multiaddr#16
  • CR + M Add arguments to 'ipfs pin ls' kubo#2208
  • CR + M Fix SetAllowedOrigins kubo#2021
  • CR + M Add Server field in HTTP API response header  kubo#2141
  • CR + M Gateway: add support for HTTP OPTIONS request type kubo#2232
  • CR + M some website PRs
  • Review + comment on dist.ipfs.io things
  • Review + comment on firefox gateway things

[jbenet]

worth highlighting in weekly IMO:

• ⭐ the gateway stuff (now serving both 0.3.x and 0.4.x)
• ⭐ shipping dist.ipfs.io
• ⭐ standalone dht binary
• ⭐ articjs visit + talks

[ghost]

I forgot to post the notes from the infrastructure hangout on january 18th:

dmca

• denylists in go-ipfs
• double-hashing
• infra PR regarding blocking
• add notice to refs-denylists-dmca
• notice.md wording
• what to redact in a notice
  • what does github do?
• end-to-end dmca readme

ipv6

• neccessary change in bootstrap shipped
  • => go!

dev040

• a little go tool that sits between nginx and two ipfs daemons' gateways
  • proxies each request to both of the backends (v03x + v04x),
    simultaneously
  • we can't do this serially, because we can't reasonably tell when a
    timeout happens
  • the only thing we can reliably say is when a request is successful
  • requests and responses should not get buffered (RAM usage!)
  • first successful response is sent to the client
  • after successful response, request/response to other backend is
    aborted/discarded
  • first error is discarded
  • second error is sent to the client
• pinbot

fc00

• peer discovery e.g. via geoip or AS-number
• have a big great meeting in Q2
  • with cjd, ansuz, ehmry, (kubuxu, mildred)
  • maybe around squatconf
• routing schemes
• swarm should be called router or so