Release v0.8.0

[aschmahmann]

Release Issue Template

go-ipfs 0.8.0 Release

We're happy to announce go-ipfs 0.8.0. This is planned to be a fairly small release focused on integrating in the new pinning service/remote pinning API that makes the experience of managing pins across pinning services easier and more uniform.

🗺 What's left for release

• Remote Pinning
  • go-IPFS integration: Add support for remote Pinning Services #7559, add remote pinning to ipfs command #7661, etc.
    • Modify config to store pinning service information
    • Add commands for manually working with the pinning service API
    • Add basic policy support (e.g. automatically pin/unpin MFS root)
  • HTTP API integration (js: feat: support remote pinning services in ipfs-http-client js-ipfs#3293)
  • WebUI integration: epic: pinning services ipfs-webui#1615 Dropped from v0.8.0, although the latest deployed WebUI is always available at https://webui.ipfs.io/#/
• QUIC
  • update quic-go to support QUIC draft-32

🚢 Estimated shipping date

TBD

🔦 Highlights

🧷 Remote pinning services

There is now support for asking remote services to pin data for you. This means anyone can implement the spec (developed in this repo) and allow for pin management.

All of the CLI (and corresponding HTTP API) commands are available under ipfs pin remote.

This remote pinning service comes with a redesign of how we're thinking about pinning and includes some commonly requested features such as:

• Pins can have names (and coming soon metadata)
• The same content can be pinned multiple times, but of course stored only once
  • This allows applications using the same pinning service to manage their own pins without worrying about removing content important to another application
• Data can be pinned in either the foreground or background

Examples include:

ipfs pin remote service add myservice https://myservice.tld:1234/api/path myaccess key

ipfs pin remote add /ipfs/bafymydata --service=myservice --name=myfile 
ipfs pin remote ls --service=myservice --name=myfile
ipfs pin remote ls --service=myservice --cid=bafymydata
ipfs pin remote rm --serivce=myservice --name=myfile

A few notes:

Remote pinning services work with recursive pins. This means commands like ipfs pin remote ls will not list indirectly pinned CIDs.

While pinning service data is stored in the configuration file it cannot be edited directly via the ipfs config commands due to the sensitive nature of pinning service API keys. The ipfs pin remote service commands can be used for interacting with remote service settings.

📌 Faster local pinning and unpinning

The pinning subsystem has been redesigned to be much faster and more flexible in how it tracks pins. For users who are working with many pins this will lead to a big speed increase in listing and modifying the set of pinned items as well as decreased memory usage.

Part of the redesign was setup to account for being able to interact with local pins the same way we can now interact with remote pins (e.g. names, being allowed to pin the same CID multiple times, etc.). Keep posted for more improvements to pinning.

QUIC update

QUIC support has received a number of upgrades, including the ability to take advantage of larger UDP receive buffers for increased performance.

Linux users may notice a logged error on daemon startup if your system needs extra configuration to allow IPFS increase the buffer size. A helpful link for resolving this is in the log message as well as here.

👋 No more Darwin 386 builds

Go 1.15 (the latest version of Go) no longer supports Darwin 386 and so we are dropping support as well.

Changelog

< changelog generated by bin/mkreleaselog >

✅ Release Checklist

For each RC published in each stage:

• version string in version.go has been updated (in the release-vX.Y.Z branch).
• tag commit with vX.Y.Z-rcN
• upload to dist.ipfs.io
  1. Build: https://github.com/ipfs/distributions#usage.
  2. Pin the resulting release.
  3. Make a PR against ipfs/distributions with the updated versions, including the new hash in the PR comment.
  4. Ask the infra team to update the DNSLink record for dist.ipfs.io to point to the new distribution.
• cut a pre-release on github and upload the result of the ipfs/distributions build in the previous step.
• Announce the RC:
  • On IRC/Matrix (both #ipfs and #ipfs-dev)
  • To the early testers listed in docs/EARLY_TESTERS.md.

Checklist:

• Stage 0 - Automated Testing
  • Fork a new branch (release-vX.Y.Z) from master and make any further release related changes to this branch. If any "non-trivial" changes (see the footnotes of docs/releases.md for a definition) get added to the release, uncheck all the checkboxes and return to this stage.
    • Follow the RC release process to cut the first RC.
    • Bump the version in version.go in the master branch to vX.(Y+1).0-dev.
  • Automated Testing (already tested in CI) - Ensure that all tests are passing, this includes:
    • unit, sharness, cross-build, etc (make test)
    • lint (make test_go_lint)
    • interop
    • go-ipfs-api
    • go-ipfs-http-client
    • WebUI
• Stage 1 - Internal Testing
  • CHANGELOG.md has been updated
    • use ./bin/mkreleaselog to generate a nice starter list
  • Network Testing:
    • test lab things - TBD
  • Infrastructure Testing:
    • Deploy new version to a subset of Bootstrappers
    • Deploy new version to a subset of Gateways
    • Deploy new version to a subset of Preload nodes
    • Collect metrics every day. Work with the Infrastructure team to learn of any hiccup
  • IPFS Application Testing - Run the tests of the following applications:
    • IPFS Desktop - @hacdias
    • IPFS Companion - @lidel
    • NPM on IPFS - @achingbrain
• Stage 2 - Community Dev Testing
  • Reach out to the IPFS early testers listed in docs/EARLY_TESTERS.md for testing this release (check when no more problems have been reported). If you'd like to be added to this list, please file a PR.
  • Reach out to on IRC for beta testers.
  • Run tests available in the following repos with the latest beta (check when all tests pass):
    • orbit-db
• Stage 3 - Community Prod Testing
  • Documentation
    • Ensure that CHANGELOG.md is up to date
    • Ensure that README.md is up to date
    • Ensure that all the examples we have produced for go-ipfs run without problems
    • Update HTTP-API Documentation on the Website using https://github.com/ipfs/http-api-docs
  • Invite the IPFS early testers to deploy the release to part of their production infrastructure.
  • Invite the wider community through (link to the release issue):
    • discuss.ipfs.io
    • Twitter
    • IRC
• Stage 4 - Release
  • Final preparation
    • Verify that version string in version.go has been updated.
    • Merge release-vX.Y.Z into the release branch.
    • Tag this merge commit (on the release branch) with vX.Y.Z.
    • Release published
      • to dist.ipfs.io
      • to npm-go-ipfs
      • to chocolatey
      • to snap
      • to github
      • to arch (flag it out of date)
    • Cut a new ipfs-desktop release
  • Publish a Release Blog post (at minimum, a c&p of this release issue with all the highlights, API changes, link to changelog and thank yous)
  • Broadcasting (link to blog post)
    • Twitter
    • IRC
    • Reddit
    • discuss.ipfs.io
    • Announce it on the IPFS Users Mailing List
• Post-Release
  • Merge the release branch back into master, ignoring the changes to version.go (keep the -dev version from master).
  • Create an issue using this release issue template for the next release.
  • Make sure any last-minute changelog updates from the blog post make it back into the CHANGELOG.

❤️ Contributors

< list generated by bin/mkreleaselog >

Would you like to contribute to the IPFS project and don't know how? Well, there are a few places you can get started:

• Check the issues with the help wanted label in the go-ipfs repo
• Join an IPFS All Hands, introduce yourself and let us know where you would like to contribute - https://github.com/ipfs/team-mgmt/#weekly-ipfs-all-hands
• Hack with IPFS and show us what you made! The All Hands call is also the perfect venue for demos, join in and show us what you built
• Join the discussion at discuss.ipfs.io and help users finding their answers.
• Join the 🚀 IPFS Core Implementations Weekly Sync 🛰 and be part of the action!

⁉️ Do you have questions?

The best place to ask your questions about IPFS, how it works and what you can do with it is at discuss.ipfs.io. We are also available at the #ipfs channel on Freenode, which is also accessible through our Matrix bridge.

[MichaelMure]

I suppose this will be covered by "update quic-go to support QUIC draft-32", but please make sure this new release include quic-go/quic-go#2827

[marten-seemann]

@MichaelMure Yes, that PR is part of #7769.

[aschmahmann]

go-ipfs v0.8.0-rc1 is out, give it a spin and let us know how it goes 😄

CC Early Testers:

• pacman.store (@RubenKelevra)
• Infura (@MichaelMure)
• Textile (@sanderpick)
• Pinata (@obo20)
• RTrade (@postables)
• QRI (@b5)
• Siderus (@koalalorenzo)
• Charity Engine (@Rytiss, @tristanolive)
• Fission (@bmann)

[lidel]

Quick prompt for folks interested in remote pinning commands:

• ipfs pin remote service --help
• ipfs pin remote --help

Test services you can run on your own:

• https://github.com/ipfs-shipyard/rb-pinning-service-api (PoC with persistence, requires docker compose)
• https://github.com/ipfs-shipyard/js-mock-ipfs-pinning-service (light mock in Nodejs, in-memory storage, useful for CI and testing)
  • pin remote add will wait for pinning to finish by default, so either pass --background=true or use CID with data inlined in them (identity multihash) to instantly get 'pinned' status:

    $ echo "test @ $(date)" | ipfs add -q --inline --inline-limit 1000 --pin=false
    

In case your ipfs config show stops working, use workaround from #7818 (comment)

[RubenKelevra]

Never seen this issue again, looks like this might just have been a temporary connection issue dropping a lot of UDP traffic, while ICMP has worked fine. Dunno.

I'll create a ticket if this ever occurs again and will investigate further in this case. :)

[aschmahmann]

v0.8.0-rc2 is out with a number of bug fixes:

• fixed the ipfs config show bug related to the pinning services config Error in config show: case mismatch in config #7818
• fs-repo-migrations is now faster for this migration and has a locking issue fixed
• added a policy for remote pinning services to automatically pin your MFS root to the remote service
• support inlined DNSLink names with TLS (feat(gw): DNSLink names on https:// subdomains #7847)
• a libp2p update that included some QUIC fixes

One breaking change from RC1 is that instead of having pinning service APIs defined as Pinning.RemoteServices.{yourServiceName}.Api it is now Pinning.RemoteServices.{yourServiceName}.API to be more consistent with our other config options, sorry for any inconvenience here. I'm a bit out of 🔋 for today, but hope to have a script to make this quick to fix for folks (although if someone else wanted to write it I wouldn't complain 😄).

More docs around how to configure the MFS policy coming soon as well as any other info on the RC I may have forgotten tonight.

Give it a spin and let us know how it goes 🙏.

[lidel]

Good news regarding config API key rename: there is no breaking change, preexisting Api will work just fine because if exact match is not found, encoding/json will do case-insensitive match as a fallback:

https://golang.org/pkg/encoding/json/#Unmarshal

To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal (either the struct field name or its tag), preferring an exact match but also accepting a case-insensitive match.

[aschmahmann]

reopening while some of the post-release process is still ongoing (e.g. package managers, blog posts, etc.)

[fiott248]

where is the version of python library to match with this agent "0.8.0" i can only install up to 0.7.0.