Handle incoming conns in their own goroutines #35

whyrusleeping · 2016-04-10T06:12:27Z

Doing the multistream negotiation in sync causes hanging issues.
This commit accepts transport connections and starts the negotiation
in a separate goroutine, sending it down a channel when its ready.

Doing the multistream negotiation in sync causes hanging issues. This commit accepts transport connections and starts the negotiation in a separate goroutine, sending it down a channel when its ready.

jbenet · 2016-04-10T06:55:13Z

p2p/net/conn/dial_test.go

+	}
+
+	c.Close()
+	<-done


good basic test case. but we need a stronger stress tests. something like:

start off N (=1000) goroutines, all hang for ~20ms, then continue. measure that they all complete async (within < 2s, instead of serially N * 20 = 20s)

start off N (=1000) goroutines, hang half of them until timeout. the others should complete shortly after kicking off.

start off N (=1000) goroutines, dial, hang for some (1-10)ms , complete them. check listener goroutines are all closed properly (i.e. total goroutines << N)

hanging to timeout is important. can decrease the tiemeout for test, or mock it out with a channel to verify things make forward progress correctly

jbenet · 2016-04-11T03:50:34Z

p2p/net/conn/dial_test.go

+	if took > limit {
+		t.Fatal("took too long!")
+	}
+	log.Errorf("took: %s (less than %s)", took, limit)


need to call wg.Wait() before exiting this function.

jbenet · 2016-04-11T04:19:12Z

Thanks for handling the changes. This is pretty close. couple things left above o/

jbenet · 2016-04-11T04:41:27Z

package.json

+      "author": "whyrusleeping",
+      "name": "gorocheck",
+      "hash": "QmWBTBj2ceC8gk2pS5scr4WwuHGrzsTkUnPpv1KPhYET57",
+      "version": "0.0.0"


would be great to have the "pkg repo" (individual pkg repo where you can report issues) listed in gx to get it from stored here. this could be a link, http for now and ipns name in the future.

I could do that, thats a good idea.

jbenet · 2016-04-11T05:05:41Z

I tried running tests locally on go 1.6 and:

--- FAIL: TestConcurrentAccept (0.15s)
panic: had 1 goroutines still running. First on list: [os/signal.signal_recv(0x0)   /usr/local/go/src/runtime/sigqueue.go:116 +0x132 os/signal.loop()   /usr/local/go/src/os/signal/signal_unix.go:22 +0x18 created by os/signal.init.1     /usr/local/go/src/os/signal/signal_unix.go:28 +0x37] [recovered]
    panic: had 1 goroutines still running. First on list: [os/signal.signal_recv(0x0)   /usr/local/go/src/runtime/sigqueue.go:116 +0x132 os/signal.loop()   /usr/local/go/src/os/signal/signal_unix.go:22 +0x18 created by os/signal.init.1     /usr/local/go/src/os/signal/signal_unix.go:28 +0x37]

goroutine 80 [running]:
panic(0x44b200, 0xc8203832e0)
    /usr/local/go/src/runtime/panic.go:464 +0x3e6
testing.tRunner.func1(0xc820388240)
    /usr/local/go/src/testing/testing.go:467 +0x192
panic(0x44b200, 0xc8203832e0)
    /usr/local/go/src/runtime/panic.go:426 +0x4e9
github.com/ipfs/go-libp2p/p2p/net/conn.TestConcurrentAccept(0xc820388240)
    /Users/jbenet/go/src/github.com/ipfs/go-libp2p/p2p/net/conn/dial_test.go:530 +0x9a5
testing.tRunner(0xc820388240, 0x75b5a0)
    /usr/local/go/src/testing/testing.go:473 +0x98
created by testing.RunTests
    /usr/local/go/src/testing/testing.go:582 +0x892
exit status 2
FAIL    github.com/ipfs/go-libp2p/p2p/net/conn  0.686s

jbenet · 2016-04-11T05:19:35Z

passes fine for me in 1.5.3. Looks like gorocheck doesn't account something new in 1.6?

whyrusleeping · 2016-04-11T16:27:27Z

alright, this LGTM. gonna ship and get it into go-ipfs

jbenet · 2016-04-11T17:42:32Z

> go-use 1.6
using: go version go1.6 darwin/amd64
> go test
13:41:58.883 ERROR       conn: took: 34.695497ms (less than 2s) dial_test.go:521
skipped  &{gorocheck.parseStack 130  [gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck.parseStack(0x0, 0x0, 0x0)    /Users/jbenet/go/src/gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck/gorochecker.go:37 +0xa8 gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck.CheckForLeaks(0x5cdf90, 0x0, 0x0)    /Users/jbenet/go/src/gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck/gorochecker.go:91 +0x36 github.com/ipfs/go-libp2p/p2p/net/conn.TestConcurrentAccept(0xc82016c090)     /Users/jbenet/go/src/github.com/ipfs/go-libp2p/p2p/net/conn/dial_test.go:528 +0x95d testing.tRunner(0xc82016c090, 0x75c5a0)     /usr/local/go/src/testing/testing.go:473 +0x98 created by testing.RunTests  /usr/local/go/src/testing/testing.go:582 +0x892 ]}
skipped  &{testing.RunTests 1  [testing.RunTests(0x5ce4c0, 0x75c4e0, 0xf, 0xf, 0x4fff01)    /usr/local/go/src/testing/testing.go:583 +0x8d2 testing.(*M).Run(0xc820476f08, 0x4db160)    /usr/local/go/src/testing/testing.go:515 +0x81 main.main()  github.com/ipfs/go-libp2p/p2p/net/conn/_test/_testmain.go:82 +0x117 ]}
skipped  &{runtime.goexit 17  [runtime.goexit()     /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 ]}
skipped  &{signal.signal_recv 6  [os/signal.signal_recv(0x0)    /usr/local/go/src/runtime/sigqueue.go:116 +0x132 os/signal.loop()   /usr/local/go/src/os/signal/signal_unix.go:22 +0x18 created by os/signal.init.1     /usr/local/go/src/os/signal/signal_unix.go:28 +0x37]}
skipped  &{gorocheck.parseStack 338  [gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck.parseStack(0x0, 0x0, 0x0)    /Users/jbenet/go/src/gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck/gorochecker.go:37 +0xa8 gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck.CheckForLeaks(0x5cdf90, 0x0, 0x0)    /Users/jbenet/go/src/gx/ipfs/QmWdkzYigQqvWn7DhthjhEmJ4gVQc3R7VNrjXTwMmmLjMf/gorocheck/gorochecker.go:91 +0x36 github.com/ipfs/go-libp2p/p2p/net/conn.TestConnectionTimeouts(0xc82016c120)   /Users/jbenet/go/src/github.com/ipfs/go-libp2p/p2p/net/conn/dial_test.go:645 +0xa2d testing.tRunner(0xc82016c120, 0x75c5b8)     /usr/local/go/src/testing/testing.go:473 +0x98 created by testing.RunTests  /usr/local/go/src/testing/testing.go:582 +0x892 ]}
skipped  &{testing.RunTests 1  [testing.RunTests(0x5ce4c0, 0x75c4e0, 0xf, 0xf, 0x4fff01)    /usr/local/go/src/testing/testing.go:583 +0x8d2 testing.(*M).Run(0xc820256f08, 0x4db160)    /usr/local/go/src/testing/testing.go:515 +0x81 main.main()  github.com/ipfs/go-libp2p/p2p/net/conn/_test/_testmain.go:82 +0x117 ]}
skipped  &{runtime.goexit 17  [runtime.goexit()     /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 ]}
skipped  &{signal.signal_recv 6  [os/signal.signal_recv(0x0)    /usr/local/go/src/runtime/sigqueue.go:116 +0x132 os/signal.loop()   /usr/local/go/src/os/signal/signal_unix.go:22 +0x18 created by os/signal.init.1     /usr/local/go/src/os/signal/signal_unix.go:28 +0x37]}
PASS
ok      github.com/ipfs/go-libp2p/p2p/net/conn  6.017s

* Single goroutine managing autonat-relevent events. * Watching incoming connections and local interface changes as signals. * Emit a single 'rechabilitychanged' persistent event fix #40 fix #36 fix #35 fix #34 fix #11 obsolete #28 obsolete #9 Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>

Update stream muxer.

chore: update go-log to v2

fix deadlock in the transport's serve function

Handle incoming conns in their own goroutines

b938ab9

Doing the multistream negotiation in sync causes hanging issues. This commit accepts transport connections and starts the negotiation in a separate goroutine, sending it down a channel when its ready.

jbenet reviewed Apr 10, 2016
View reviewed changes

whyrusleeping added 3 commits April 10, 2016 10:21

address feedback from CR

9d44390

add test to show concurrency of dials

d3a9043

make test smaller for osx

e123a79

jbenet reviewed Apr 11, 2016
View reviewed changes

add a test with connections timing out

261b113

whyrusleeping force-pushed the fix/listener-hang branch from 55722da to 261b113 Compare April 11, 2016 18:06

whyrusleeping merged commit b12b73d into master Apr 11, 2016

whyrusleeping deleted the fix/listener-hang branch April 11, 2016 18:16

marten-seemann pushed a commit that referenced this pull request Apr 21, 2022

Merge pull request #35 from libp2p/feat/update-stream-muxer

1dfbb4e

Update stream muxer.

marten-seemann added a commit that referenced this pull request Apr 26, 2022

Merge pull request #35 from libp2p/update-go-log

ef6c8c9

chore: update go-log to v2

marten-seemann added a commit that referenced this pull request May 18, 2022

Merge pull request #35 from libp2p/fix-serve-deadlock

bfd7dbd

fix deadlock in the transport's serve function

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle incoming conns in their own goroutines #35

Handle incoming conns in their own goroutines #35

whyrusleeping commented Apr 10, 2016

jbenet Apr 10, 2016

jbenet Apr 10, 2016

jbenet Apr 11, 2016

jbenet commented Apr 11, 2016

jbenet Apr 11, 2016

whyrusleeping Apr 11, 2016

jbenet commented Apr 11, 2016

jbenet commented Apr 11, 2016

whyrusleeping commented Apr 11, 2016

jbenet commented Apr 11, 2016

Handle incoming conns in their own goroutines #35

Handle incoming conns in their own goroutines #35

Conversation

whyrusleeping commented Apr 10, 2016

jbenet Apr 10, 2016

Choose a reason for hiding this comment

jbenet Apr 10, 2016

Choose a reason for hiding this comment

jbenet Apr 11, 2016

Choose a reason for hiding this comment

jbenet commented Apr 11, 2016

jbenet Apr 11, 2016

Choose a reason for hiding this comment

whyrusleeping Apr 11, 2016

Choose a reason for hiding this comment

jbenet commented Apr 11, 2016

jbenet commented Apr 11, 2016

whyrusleeping commented Apr 11, 2016

jbenet commented Apr 11, 2016