Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fuse: panic in handler for Read #6765

Open
jaller94 opened this issue Nov 15, 2019 · 2 comments
Open

fuse: panic in handler for Read #6765

jaller94 opened this issue Nov 15, 2019 · 2 comments
Labels
kind/bug A bug in existing code (including security flaws)

Comments

@jaller94
Copy link

Version information:

ipfs version 0.4.22 from the Manjaro Linux repo.

go-ipfs version: 0.4.22-4e981576b-dirty
Repo version: 7
System version: amd64/linux
Golang version: go1.12.8

Description:

  1. I ran ipfs daemon.
  2. I ran ipfs mount.
  3. I pointed my music player Clementine to /ipfs/Qm…
  4. Indexing worked and I was able to listen to an entire album.
  5. After about 60 minutes, when starting a new album, the IPFS daemon crashed. This left the mount points in the status "/ipns: Transport endpoint is not connected".

Stacktrace:

2019/11/12 19:39:33 fuse: panic in handler for Read [ID=0x8348 Node=0x2d Uid=1000 Gid=1000 Pid=12152] 0x1 4096 @0x6cb000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2451559 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc002506070, 0xc004a9fef8, 0xc004a9fe67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc00fd9d320, 0x559001993d80, 0xc002f1b380, 0xc002506070, 0xc0041b7000, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc002f1b380, 0x5590019736c0, 0xc00fd9d320, 0xc008702d80, 0x55900198c140, 0xc002506070, 0xc004a9fef8, 0xc000462768, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc002506070)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc002506070)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:33 fuse: panic in handler for Read [ID=0x834a Node=0x2d Uid=1000 Gid=1000 Pid=12152] 0x1 4096 @0x6cb000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2451561 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc0025060e0, 0xc01152def8, 0xc01152de67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc00fd9d320, 0x559001993d80, 0xc002f1b480, 0xc0025060e0, 0xc0041b7f40, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc002f1b480, 0x5590019736c0, 0xc00fd9d320, 0xc008702d80, 0x55900198c140, 0xc0025060e0, 0xc01152def8, 0xa, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc0025060e0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc0025060e0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:33 fuse: panic in handler for Read [ID=0x834c Node=0x2d Uid=1000 Gid=1000 Pid=12152] 0x1 4096 @0x6cb000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2451563 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc0025061c0, 0xc01152def8, 0xc01152de67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc00fd9d320, 0x559001993d80, 0xc002f1b540, 0xc0025061c0, 0xc0025dcac0, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc002f1b540, 0x5590019736c0, 0xc00fd9d320, 0xc008702d80, 0x55900198c140, 0xc0025061c0, 0xc01152def8, 0xc0129e9f68, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc0025061c0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc0025061c0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:33 fuse: panic in handler for Read [ID=0x834e Node=0x2d Uid=1000 Gid=1000 Pid=12152] 0x1 4096 @0x6cb000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2451565 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc0025062a0, 0xc004a9fef8, 0xc004a9fe67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc00fd9d320, 0x559001993d80, 0xc002f1b600, 0xc0025062a0, 0xc0023f0b20, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc002f1b600, 0x5590019736c0, 0xc00fd9d320, 0xc008702d80, 0x55900198c140, 0xc0025062a0, 0xc004a9fef8, 0xc012772768, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc0025062a0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc0025062a0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:33 fuse: panic in handler for Read [ID=0x8350 Node=0x2d Uid=1000 Gid=1000 Pid=12152] 0x1 4096 @0x6cb000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2451567 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc002506380, 0xc004a9fef8, 0xc004a9fe67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc00fd9d320, 0x559001993d80, 0xc002f1b6c0, 0xc002506380, 0xc0083ce1a0, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc002f1b6c0, 0x5590019736c0, 0xc00fd9d320, 0xc008702d80, 0x55900198c140, 0xc002506380, 0xc004a9fef8, 0x2, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc002506380)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc002506380)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:33 fuse: panic in handler for Read [ID=0x8352 Node=0x2d Uid=1000 Gid=1000 Pid=12152] 0x1 4096 @0x6cb000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2451569 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc0025063f0, 0xc01152def8, 0xc01152de67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc00fd9d320, 0x559001993d80, 0xc002f1b780, 0xc0025063f0, 0xc0083cea00, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc002f1b780, 0x5590019736c0, 0xc00fd9d320, 0xc008702d80, 0x55900198c140, 0xc0025063f0, 0xc01152def8, 0xa, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc0025063f0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc0025063f0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:34 fuse: panic in handler for Read [ID=0x85ec Node=0xaa Uid=1000 Gid=1000 Pid=12146] 0x1 4096 @0x5ed000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2453166 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc00325b7a0, 0xc0111d3ef8, 0xc0111d3e67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc005f201e0, 0x559001993d80, 0xc007dbe200, 0xc00325b7a0, 0xc0134b28e0, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc007dbe200, 0x5590019736c0, 0xc005f201e0, 0xc013ffa540, 0x55900198c140, 0xc00325b7a0, 0xc0111d3ef8, 0xc00f8e4101, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc00325b7a0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc00325b7a0)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:34 fuse: panic in handler for Read [ID=0x85ee Node=0xaa Uid=1000 Gid=1000 Pid=12146] 0x1 4096 @0x5ed000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2453168 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc00325b810, 0xc011d89ef8, 0xc011d89e67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc005f201e0, 0x559001993d80, 0xc013ca2100, 0xc00325b810, 0xc001d90620, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc013ca2100, 0x5590019736c0, 0xc005f201e0, 0xc013ffa540, 0x55900198c140, 0xc00325b810, 0xc011d89ef8, 0xc00f440b01, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc00325b810)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc00325b810)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
2019/11/12 19:39:34 fuse: panic in handler for Read [ID=0x85f0 Node=0xaa Uid=1000 Gid=1000 Pid=12146] 0x1 4096 @0x5ed000 dir=false fl=0 lock=0 ffl=OpenReadWrite: runtime error: slice bounds out of range
goroutine 2453170 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x55900198c140, 0xc00325b880, 0xc003185ef8, 0xc003185e67)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:857 +0x1a4
panic(0x5590017c9320, 0x55900224cc70)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b9
github.com/ipfs/go-ipfs/fuse/readonly.(*Node).Read(0xc005f201e0, 0x559001993d80, 0xc013ca2200, 0xc00325b880, 0xc001d90e20, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/fuse/readonly/readonly_unix.go:261 +0x7fd
bazil.org/fuse/fs.(*Server).handleRequest(0xc0004f4240, 0x559001993d80, 0xc013ca2200, 0x5590019736c0, 0xc005f201e0, 0xc013ffa540, 0x55900198c140, 0xc00325b880, 0xc003185ef8, 0xa, ...)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:1249 +0x8d2
bazil.org/fuse/fs.(*Server).serve(0xc0004f4240, 0x55900198c140, 0xc00325b880)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:878 +0x411
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0004f4240, 0x55900198c140, 0xc00325b880)
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:425 +0x70
created by bazil.org/fuse/fs.(*Server).Serve
	/build/go-ipfs/src/go-ipfs/vendor/bazil.org/fuse/fs/serve.go:423 +0x39b
panic: concurrent write to websocket connection

goroutine 3713520 [running]:
github.com/gorilla/websocket.(*messageWriter).flushFrame(0xc003423140, 0x5c4a989ca563be01, 0x0, 0x0, 0x0, 0x0, 0x0)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/gorilla/websocket/conn.go:591 +0x77c
github.com/gorilla/websocket.(*messageWriter).Close(0xc003423140, 0xc012d72d18, 0x5590006ae958)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/gorilla/websocket/conn.go:709 +0x58
github.com/gorilla/websocket.(*Conn).prepWrite(0xc00f38d340, 0x2, 0x559000a1bd04, 0xc0015f8c60)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/gorilla/websocket/conn.go:459 +0x20f
github.com/gorilla/websocket.(*Conn).NextWriter(0xc00f38d340, 0x2, 0x7c, 0x46, 0x80, 0x10000c011b93604)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/gorilla/websocket/conn.go:494 +0x3b
github.com/gorilla/websocket.(*Conn).WriteMessage(0xc00f38d340, 0x2, 0xc0126d9bc0, 0x22, 0x40, 0x10, 0xc001bd77b0)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/gorilla/websocket/conn.go:755 +0x75
github.com/libp2p/go-ws-transport.(*Conn).Write(0xc003422f30, 0xc0126d9bc0, 0x22, 0x40, 0x5590017d12c0, 0x559001929200, 0x7f723afc6230)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/libp2p/go-ws-transport/conn.go:74 +0x5b
bytes.(*Buffer).WriteTo(0xc004554e10, 0x7f723afcca38, 0xc0034230b0, 0x7f723afc6230, 0xc004554e10, 0x5590009e4b01)
	/usr/lib/go/src/bytes/buffer.go:242 +0xba
io.copyBuffer(0x7f723afcca38, 0xc0034230b0, 0x559001972a20, 0xc004554e10, 0x0, 0x0, 0x0, 0xc01552a704, 0x559000560318, 0x559001952ab8)
	/usr/lib/go/src/io/io.go:384 +0x350
io.Copy(...)
	/usr/lib/go/src/io/io.go:364
github.com/multiformats/go-multistream.SelectProtoOrFail.func1(0x5590010a17fd, 0xc, 0x7f723afcc980, 0xc0034230b0, 0xc003291200)
	/build/go-ipfs/src/go-ipfs/vendor/github.com/multiformats/go-multistream/client.go:27 +0x166
created by github.com/multiformats/go-multistream.SelectProtoOrFail
	/build/go-ipfs/src/go-ipfs/vendor/github.com/multiformats/go-multistream/client.go:23 +0x97
@jaller94 jaller94 added the kind/bug A bug in existing code (including security flaws) label Nov 15, 2019
@aschmahmann
Copy link
Contributor

Looks like we may not be checking for non-empty responses then getting an out-of-bounds exception from calling https://github.com/ipfs/go-ipfs/blob/410e7e87875c6b0d8ac02885b1331549f770dc86/fuse/readonly/readonly_unix.go#L261

As @jaller94 already noted this seems to be getting triggered by #6197

@Stebalien
Copy link
Member

Hm. I'm not entirely sure how this is happening. We might:

  1. Have some other memory corruption (e.g., the websocket one).
  2. Have a file reporting the wrong size? That would do it, probably.

Stebalien added a commit that referenced this issue Jan 28, 2020
We likely encountered a file that misreported its size. That or there was no bug
here and we hit an issue somewhere else. Regardless, there's no reason not to
simplify this code and this should fix the issue.

fixes #6765
Stebalien added a commit that referenced this issue Jan 28, 2020
We likely encountered a file that misreported its size. That or there was no bug
here and we hit an issue somewhere else. Regardless, there's no reason not to
simplify this code and this should fix the issue.

fixes #6765
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws)
Projects
None yet
Development

No branches or pull requests

3 participants