Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release v0.4.23 #6837

Closed
49 of 51 tasks
Stebalien opened this issue Jan 17, 2020 · 28 comments
Closed
49 of 51 tasks

Release v0.4.23 #6837

Stebalien opened this issue Jan 17, 2020 · 28 comments
Labels
topic/release Topic release

Comments

@Stebalien
Copy link
Member

Stebalien commented Jan 17, 2020 •
edited

go-ipfs 0.4.23 Release

PR: #6836
Download: https://dist.ipfs.io/go-ipfs/v0.4.23-rc2 (RC2)

Unfortunately, go-ipfs 0.5.0 is taking longer than expected. Given the large number of fixes merged since 0.4.22, we've decided to cut another patch release.

This release contains critical fixes. Please upgrade ASAP. Importantly, we're strongly considering switching to TLS by default in go-ipfs 0.5.0 and dropping SECIO support. However, our the current TLS transport in go-ipfs 0.4.22 has a bug that can cause connections to spontaneously disconnect during the handshake.

This release fixes that bug, among many other issues. Users that don't upgrade may experience connectivity issues when the network upgrades to go-ipfs 0.5.0.

🗺 What's left for release

  • Get the tests passing.
  • Release Notes.

🚢 Estimated shipping date

January 27th, at the latest (likely earlier).

🔦 Highlights

  • Fixes build on go 1.13
  • Fixes an issue where we may not connect to providers in bitswap.
  • Fixes an issue on our TLS transport where we may abort a handshake unintentionally.
  • Fixes a common panic in the websocket transport.
  • Adds support for recursively resolving dnsaddrs (makes go-ipfs compatible with the new bootstrappers).
  • Fixes several potential panics/crashes.
  • Switches to using pre-defined autorelays instead of trying to find them in the DHT:
    • Avoids selecting random, potentially poor, relays.
    • Avoids spamming the DHT with requests trying to find relays.
    • Reduces the impact of accidentally enabling AutoRelay + RelayHop. I.e., the network won't try to DoS you.
  • Modifies the connection manager to not count connections in the grace period towards the connection limit.
    • Pro: New connections don't cause us to close useful, existing connections.
    • Con: Libp2p will keep more connections. Consider reducing your HighWater after applying this patch.
  • Improved peer usefulness tracking in bitswap. Frequently used peers will be marked as "important" and the connection manager will avoid closing connections to these peers.
  • Includes a new version of the WebUI to fix some issues with the peers map.

Changelog

Other:

Update bloom filter libraries to remove unsound usage of the unsafe package.

✅ Release Checklist

For each RC published in each stage:

  • version string in version.go has been updated
  • tag commit with vX.Y.Z-rcN
  • upload to dist.ipfs.io
    1. Build: https://github.com/ipfs/distributions#usage.
    2. Pin the resulting release.
    3. Make a PR against ipfs/distributions with the updated versions, including the new hash in the PR comment.
    4. Ask the infra team to update the DNSLink record for dist.ipfs.io to point to the new distribution.
  • cut a pre-release on github and upload the result of the ipfs/distributions build in the previous step.

Checklist:

  • Stage 0 - Automated Testing
    • Feature freeze. If any "non-trivial" changes (see the footnotes of docs/releases.md for a definition) get added to the release, uncheck all the checkboxes and return to this stage.
    • Automated Testing (already tested in CI) - Ensure that all tests are passing, this includes:
  • Stage 1 - Internal Testing
    • CHANGELOG.md has been updated
    • Infrastructure Testing:
      • Deploy new version to a subset of Bootstrappers
        • Deploying to mars.
      • Deploy new version to a subset of Gateways
      • Deploy new version to a subset of Preload nodes
      • Collect metrics every day. Work with the Infrastructure team to learn of any hiccup
    • IPFS Application Testing - Run the tests of the following applications:
  • Stage 2 - Community Dev Testing
    • skipping because this is a patch release
  • Stage 3 - Community Prod Testing
    • Documentation
    • Invite the IPFS early testers to deploy the release to part of their production infrastructure.
    • Invite the wider community through (link to the release issue):
  • Stage 4 - Release
    • Final preparation
    • Publish a Release Blog post (at minimum, a c&p of this release issue with all the highlights, API changes, link to changelog and thank yous)
    • Broadcasting (link to blog post)
  • Post-Release
    • Bump the version in version.go to vX.(Y+1).0-dev.
    • Create an issue using this release issue template for the next release.
    • Make sure any last-minute changelog updates from the blog post make it back into the CHANGELOG.

❤️ Contributors

Contributor Commits Lines ± Files Changed
Steven Allen 52 +1866/-578 102
vyzo 12 +167/-90 22
whyrusleeping 5 +136/-52 7
Roman Proskuryakov 7 +94/-7 10
Jakub Sztandera 3 +58/-13 7
hucg 2 +31/-11 2
Raúl Kripalani 2 +7/-33 6
Marten Seemann 3 +27/-10 5
Marcin Rataj 2 +26/-0 5
b5 1 +2/-22 1
Hector Sanjuan 1 +11/-0 1
Yusef Napora 1 +4/-0 1

Would you like to contribute to the IPFS project and don't know how? Well, there are a few places you can get started:

⁉️ Do you have questions?

The best place to ask your questions about IPFS, how it works and what you can do with it is at discuss.ipfs.io. We are also available at the #ipfs channel on Freenode, which is also accessible through our Matrix bridge.
@Stebalien Stebalien added the topic/release Topic release label Jan 17, 2020
@Stebalien Stebalien mentioned this issue Jan 17, 2020
Closed
@Stebalien
Copy link
Member Author

Stebalien commented Jan 18, 2020

@alanshaw could I borrow your expertise at writing wonderful, engaging release notes for this release?

(I understand that might be a bit tricky given that you weren't directly involved in most of the changes; we can discuss this on Monday and/or I can try writing a soulless skeleton).

@Stebalien
Copy link
Member Author

Stebalien commented Jan 18, 2020

RC1 at /ipfs/QmRxq967po6BcGW68wbUvwin758dkRiejAKmyHDm9wqaRB/go-ipfs/v0.4.23-rc1/. I'll update dist.ipfs.io later.

@alanshaw
Copy link
Member

alanshaw commented Jan 20, 2020

@alanshaw could I borrow your expertise at writing wonderful, engaging release notes for this release?

I'll give it a go!

@Stebalien Stebalien pinned this issue Jan 20, 2020
@lidel
Copy link
Member

lidel commented Jan 21, 2020

@Stebalien is it too late to include WebUI fixes from #6778 (comment)?
Commit to cherrypick is 5ab7a70

@Stebalien
Copy link
Member Author

Stebalien commented Jan 21, 2020

No, we can include that. Could you send a patch against the current RC branch to use the new version?

This was referenced Jan 21, 2020
Merged
Merged
@Stebalien
Copy link
Member Author

Stebalien commented Jan 21, 2020

I'm releasing an RC2 with two changes:

  1. The backported WebUI update. This should probably count as a "feature" but it's not likely to cause wide-scale regressions and it includes a bunch of important bug fixes.
  2. A patch to fix a memory leak from the DHT that I intended to include but didn't. Luckily, we have our handy release checklist and this was caught when profiling a bootstrapper.

@Stebalien
Copy link
Member Author

Stebalien commented Jan 22, 2020 •
edited

Early testers: We're entering stage 3 of this patch release. As with the previous release, this release only includes cherry-picked bug fixes (quite a few of them) so we don't expect any regressions or significant performance changes. However, we're still inviting you to do a quick sanity check before the final release.

Note: We have cherry-picked a few patches that could cause interesting behavioral changes:

If you're relying on AutoRelay, we're no longer picking random relays from the DHT. Instead, we've hard-coded a list of known-good relays. This is a stop-gap to deal with the fact that many of the public relays were unreliable, causing nodes using AutoRelay to continuously crawl the network looking for a good relay.

The connection manager no longer counts connections in the "grace period" (default is 30s) against the limit. Before this change, nodes with many inbound connections would disconnect from all long-term peers as connections in the grace period would take the entire quota. You may need to adjust your connection limits to account for this change.

Finally, bitswap now tracks which peers we frequently exchange data with and asks the connection manager to avoid closing connections to these peers.

@MichaelMure
Copy link
Contributor

MichaelMure commented Jan 22, 2020

Thanks for the release :)

Just a note: consider pinging the early testers when you make the release (aka 5 days ago) instead of at stage 3 so we can do the testing with a bit more room.

@Stebalien
Copy link
Member Author

Stebalien commented Jan 22, 2020

@MichaelMure you're right, I should have pinged the early testers when creating the release issue. I'll add that to the release process.

@MichaelMure
Copy link
Contributor

MichaelMure commented Jan 22, 2020

What about fixing #6795 as well ?

@Stebalien
Copy link
Member Author

Stebalien commented Jan 22, 2020

go get github.com/ipfs/go-ipfs/cmd/ipfs@v0.4.23-rc2 works for me.

@bonedaddy
Copy link

bonedaddy commented Jan 22, 2020

I'm currently testing rc2 in dev. I wont be able to switch over any production nodes until friday during maintenance however.

@Stebalien
Copy link
Member Author

Stebalien commented Jan 23, 2020

@bonedaddy SGTM. We're planning on cutting a release either late friday or early next monday but I'm willing to wait to get this right.

@alanshaw
Copy link
Member

alanshaw commented Jan 23, 2020

I'm on this now - sorry for the delay. Here's my intro:

Yes, that's right the next feature release of go-ipfs (0.5.0) is, well, running a tiny bit behind schedule. In the mean time though we have patches, and I'm not talking pirate eye patches, I'm talking bug fixes. We're hunting these bugs like they're pokemon, and jeez, do we come across some rare and difficult to fix ones? - you betcha.

@b5
Copy link
Contributor

b5 commented Jan 23, 2020

@alanshaw playing to both pirate and pokemon crowds in a single paragraph? Deep.

I'll upgrade our deps & run tests ASAP & try to report back before the end of the week. @Stebalien I'm hoping you're still a few timezones behind me 😄

@RubenKelevra
Copy link
Contributor

RubenKelevra commented Jan 23, 2020

Update bloom filter libraries to remove unsafe usage of unsafe.

You're sure this sentence should be that way? 🤔

@Stebalien
Copy link
Member Author

Stebalien commented Jan 23, 2020

I guess "unsound" usage of unsafe is more accurate.

@RubenKelevra
Copy link
Contributor

RubenKelevra commented Jan 24, 2020 •
edited

I guess "unsound" usage of unsafe is more accurate.

I thought you accidentally wrote 'unsafe' the second time, but actually meant something else :)

Btw: I really appreciate your hard work, this wasn't meant to be picky. The size of this changes is unreal :)

@rklaehn
Copy link

rklaehn commented Jan 24, 2020

we're strongly considering switching to TLS by default in go-ipfs 0.5.0 and dropping SECIO support.

I hope there will be a long grace period where both TLS and SECIO are supported. We got a lot of ipfs nodes in production, and given the trouble we had upgrading last time we are not going to immediately upgrade all nodes.

Also, it is very important for us that interop between rust-libp2p and js-ipfs via TLS is confirmed to work before disabling SECIO.

@RubenKelevra
Copy link
Contributor

RubenKelevra commented Jan 24, 2020 •
edited

@Stebalien
Copy link
Member Author

Stebalien commented Jan 24, 2020

I hope there will be a long grace period where both TLS and SECIO are supported.

TLS has been supported for quite some time (since go-ipfs 0.4.21, 30 May 2019). The issue fixed in this release was just a bug that could cause the handshake to fail in some rare cases.

The current plan is to drop SECIO support in go-ipfs 0.5.0, assuming we have full interoperability with js-ipfs and rust-libp2p.

Note: go-ipfs 0.5.0 will not work with go-ipfs nodes less than 0.4.21 anyways as it introduces a feature that triggers a bug in 0.4.20 (and before).

We got a lot of ipfs nodes in production, and given the trouble we had upgrading last time we are not going to immediately upgrade all nodes.

0.4.23 is a patch release on 0.4.22 which is a patch release on 0.4.21. If you're running 0.4.22 or 0.4.21, the upgrade should be very smooth.

Also, it is very important for us that interop between rust-libp2p and js-ipfs via TLS is confirmed to work before disabling SECIO.

The plan is to introduce a Noise transport in all languages as it's significantly easier to implement correctly. There are ongoing efforts in rust, js, and go.

@alanshaw
Copy link
Member

alanshaw commented Jan 24, 2020 •
edited

Done, here are the release notes for your reading pleasure:

go-ipfs 0.4.23

Would sir/madam care for another patch release while they wait?

Yes that's right, the next feature release of go-ipfs (0.5.0) is, well, running a tiny bit behind schedule. In the mean time though we have patches, and I'm not talking pirate eye patches, I'm talking bug fixes. We're hunting these bugs like they're Pokemon, and jeez, do we come across some rare and difficult to fix ones? - you betcha.

Alright, enough funny business, what's the deal? Ok so, I don't want to alarm anyone but this release has some critical fixes and if you're using go-ipfs or know someone who is then you and your friends need to slide into your upgrade pants and give those IPFS nodes a good wipe down ASAP.

🤝 Fixed Spontaneous TLS Disconnects

If this isn't reason enough to upgrade I don't know what is. Turns out, a TLS handshake may have accidentially been unintentionally aborted for no good reason 😱. I shouldn't need to tell you this but *whispers* that's really bad for connectivity! Excellent news though, it's fixed, and if you want to stay well connected in the glorious future of p2p communications then you should definitely upgrade.

😱 Fixed Panics and Crashes

Panicing won't help, in life, and also in golang. Stay calm and breathe slowly. We patched a number of panics and crashes that were uncovered, including a panic seen commonly in the websocket transport. High ten 🙌?

🔁 Fixed Resursive Resolving of dnsaddr Multiaddrs

Imagine this: rolling out a brand spanking new set of bootstrap nodes only to discover their new addresses are not resolvable. I know right - bad news bears 🐻!?

Ok, "bear" this in mind: you want to keep all your DNS TXT records below 512 bytes to avoid UDP fragmentation, otherwise you'll get a truncated reply and have to connect with TCP to get all the records. If you have lots of dnsaddr TXT records then it can be more efficient to use recursive resolving than to get a truncated reply and go through the famous 18-way SYN, SYN-ACK ACK, ACK-SYN, ACK-ACK (...etc, etc) TCP handshake, not to mention the fact that go-ipfs will not even try to fallback to TCP 😅.

Anyway, long story short. We fixed that. You're welcome.

📻 Retuned Connection Manager

The Connection Manager has been tuned to better prioritise existing connections by not counting new connections in the "grace" period towards connection limits. New connections are like new friends. You can't hang out with everyone all the time, I mean, it just gets difficult to book a resturant after a while.

You also wouldn't stop being friends with Jane just because you met Sarah once on the train. You and Jane have history, think of everything you've been through. Remember that time when Jane's dog, Dave, ran away? I know, it's a weird name for a dog, I mean who gives a human name to a dog anyway, but I guess that's one of the reasons you like Jane. Anyway, she lost her dog and you both looked all around town for it, you were about to give up but then you heared faint wimpering as you were walking back to the house. Dave had somehow managed to fall into the old abandoned well!

You see?! History! ...and, erh, what was I saying? Oh yeah, Connection Manager - new connections don't cause us to close useful, existing connections (like Jane), but it does mean you'll keep more connections in total. Maybe consider reducing the HighWater setting in your config.

🍖 Reduced Relay Related DHT Spam

When AutoRelay was enabled, your IPFS node basically spent all it's time searching the DHT for relays, like a fly to a honey trap. I say that because every other IPFS node was doing the same and if you had AutoRelay and RelayHop enabled then, erm, DoS happens. Bad times.

So, instead of spamming the DHT the whole time trying to find random, potentially poor relays, IPFS is now using a pre-defined set of autorelays. I mean, try to tell me that doesn't make sense.

🐾 Better Bitswap

Joe has the rare shiny collectable card but your shoes have holes in them and he's standing in the rain on the other side of the playground 💦🕺🏻. This is exactly what bitswap is like when you have a bug that stops you from connecting to providers. Ok, I mean, there's a few differences, but it's basically the same.

We're also now marking frequently used peers as "important" in the connection manager so those connections do not get dropped. This is like, erm, you and Joe being besties. Joe has all the good cards and is surprisingly willing to part with them. Ok, I'll admit, card trading is probably not a great analogy to bitswap 😛

🦄 And More!

  • Fixed build on go 1.13
  • New version of the WebUI to fix some issues with the peers map

@Stebalien
Copy link
Member Author

Stebalien commented Jan 27, 2020

I'm planning on cutting the release by EOD today (PST) or tomorrow morning at the latest.

Does anyone have any regressions to report?

@bonedaddy
Copy link

bonedaddy commented Jan 28, 2020

Maintenance was postponed so I didn't get a chance to deploy one node in production, but the node in dev seems fine.

Output from pmap seems to be slightly improved with 922744152K vs the usual 999xxxxxxxK

@Stebalien
Copy link
Member Author

Stebalien commented Jan 28, 2020

Output from pmap seems to be slightly improved with 922744152K vs the usual 999xxxxxxxK

Good to hear. We fixed at least one memory leak.

@Stebalien
Copy link
Member Author

Stebalien commented Jan 28, 2020

I'm going to include one more last minute fix in this release: #6418 (not sure how I missed this). The fix is trivial (libp2p/go-libp2p-swarm@50cbe2b) so I'm not going to bother with another RC.

@b5
Copy link
Contributor

b5 commented Jan 28, 2020

LGTM

@Stebalien
Copy link
Member Author

Stebalien commented Jan 30, 2020

The release is up and announced. Thanks everyone!

@meeDamian meeDamian mentioned this issue Jan 31, 2020
Open
@Stebalien Stebalien unpinned this issue Feb 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic/release Topic release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@alanshaw @lidel @rklaehn @MichaelMure @Stebalien @RubenKelevra @b5 @bonedaddy