Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't start - out of memory? #26

Closed
hrdwdmrbl opened this issue Oct 3, 2016 · 21 comments
Closed

Can't start - out of memory? #26

hrdwdmrbl opened this issue Oct 3, 2016 · 21 comments

Comments

@hrdwdmrbl
Copy link

Simply trying to run factomd

$ factomd
2016/10/03 01:23:37 read factom config file:  /home/chainhead/.factom/factomd.conf
2016/10/03 01:23:37 ERROR Reading config file!
Server starting with default settings...
 open /home/chainhead/.factom/factomd.conf: no such file or directory
Go compiler version: go1.6.2
boltDBpath: /home/chainhead/.factom/
2016/10/03 01:23:37 web.go serving :8088

'factomd initializeonly' will do just that.  Initialize and stop.


>>>>>>>>>>>>>>>>>  CLIENT MODE <<<<<<<<<<<<<<<<<<<<<<<


01:23:39 2016-10-03 [WRN] BTCD: open /home/chainhead/.factom/NONE.conf: no such file or directory
01:23:39 2016-10-03 [INF] BTCD: Version 0.9.0-beta
peers.json is disabled for Milestone1
01:23:39 2016-10-03 [INF] RPCS: RPC server listening on 127.0.0.1:8384
01:23:39 2016-10-03 [INF] SRVR: Server listening on 0.0.0.0:8108 ; MaxPeers= 125
01:23:39 2016-10-03 [INF] SRVR: Server listening on [::]:8108 ; MaxPeers= 125
01:23:39 2016-10-03 [INF] DISC: 1 addresses found from DNS seed factom.network
Killed

But it always crashes nearly right-away. I believe the issue is probably that the machine runs out of memory since it only has 512mb and I notice that it's using ~83% of it right before crashing (polling is slow so that might not be peak).

Any way that I can reduce memory? I noticed from another issue that changing the loglevel could help, but I tried setting it to 'critical' and that didn't work...

@hrdwdmrbl
Copy link
Author

The current database size is 1.2GB

@ghost
Copy link

ghost commented Oct 3, 2016

I fixed this by going into C;/Users/youruser/.factom and taking a copy of factoid_wallet_bolt & factoid_bolt (Bolt is the important one) and deleting the entire .factom file from the user directory. Re running factomd and waiting for it to re sync (.factom will be approx 4gb when synced) - put your factoid_bolt and wallet .dat back into the directory and open factom walletapp and should be all good - Out of memory issues can be a number of things.

dont forget to backup the entire .factom directory just in case!!!

@carryforward
Copy link
Contributor

is this on windows or linux?

Here is another issue that might have been a similar problems.

#23

We haven't gotten a lot of testing on these new binaries and would like to know if these work for you.

512mb might not be enough for factomd.

@hrdwdmrbl
Copy link
Author

@carryforward Linux. I'm in the process of testing out 12 different blockchain products, mostly the biggest-name ones, and so far haven't found one where memory was a problem. But none of those other ones are based on Go (AFAIK), so maybe that makes all the difference

@hrdwdmrbl
Copy link
Author

@tdkmatt I will try this, thanks

@carryforward
Copy link
Contributor

carryforward commented Oct 3, 2016

Here are some experimental linux 64 bit + golang v1.7 binaries:

36d31827b552412e925f035ee04b6aa8e506e5a41bd00bada8789b8757736919 factom-cli
8a55d59187a84b0fada61c772f6282cbd39e19e2d3b794bee622b53f57a0535f factomd
66b7e86ae53ca95a2694f3850849d6fdb8029cf3854edc4d0b28c8aa7df5bbf0 fctwallet
d611adc179e23a5d7d38c3fe9666d6dc1ed99c6b2d22fb82afab0da9bf5e9f30 walletapp

fa100ff6c569f77dffa5be82978b090a6fb1d4b33ff355abef5c71fd443b8a6b linux64_1.7.zip
http://dropjar.com/#1h9egxbq

We have been distributing with i386 compiled binaries. Also, golang 1.7 is advertised to have fixed some memory errors.

To do development and evaluation, you might not want to start with the full blockchain (4.4 GB). Here are directions on how to setup a local testnet.
https://github.com/FactomProject/FactomDocs/blob/master/developerSandboxSetup.md

Keep in mind that factom will be going through a hard fork in the near future (Factom Genesis -> Factom Federation AKA m1->m2). It also involves a complete redesign of how the clients work. It will have instant (round trip latency) acknowledgements from the system authorities. Also, the memory overhead is significantly lower with the next generation of code. 512 MB of system RAM may not be enough though, even with the next version.

@mberry
Copy link

mberry commented Oct 7, 2016

I'm quite sure the Go 1.6 memory allocation issue is limited to Windows, possibly just newer versions of windows. In my experience Linux is able to handle the stack, even on restricted (1GB RAM) virtual machines running factomd.

That said, 512MB is tiny, current RaspberryPi's ship with more RAM.

@hrdwdmrbl
Copy link
Author

Hahaha, Digital Ocean $5/month is 512 😌

On Friday, 7 October 2016, Mitchell notifications@github.com wrote:

I'm quite sure the Go 1.6 memory allocation issue is limited to Windows,
possibly just newer versions of windows. In my experience Linux is able to
handle the stack, even on restricted (1GB RAM) virtual machines running
factomd.

That said, 512MB is tiny, current RaspberryPi's ship with more RAM.


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#26 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAh3gcx7hdyaf2cRlgsdN4hER1lAf4Zgks5qxrOwgaJpZM4KMLVH
.

Marc Beaupré-Pham
Kawaii Ambassador kigurumi.com
Tel: 514-943-8996
github.com/jackquack

@mberry
Copy link

mberry commented Oct 7, 2016

Sorry I didn't realise it was a cloud server. Consider running your own if possible, it's usually much cheaper!

@hironohata
Copy link

I have a similar problem with ubuntu.

factomd worked well three days ago. It, however, generates "out of memory" trouble since yesterday.

I updated golang to 1.7, but the result was same as before.

Should I modify some factom configuration files, too?

hiron


a@s:~$ factomd
2016/10/21 10:41:03 read factom config file: /home/USER/.factom/factomd.conf
2016/10/21 10:41:03 ERROR Reading config file!
Server starting with default settings...
open /home/USER/.factom/factomd.conf: no such file or directory
Go compiler version: go1.6.2
boltDBpath: /home/USER/.factom/
2016/10/21 10:41:03 web.go serving :8088

'factomd initializeonly' will do just that. Initialize and stop.

CLIENT MODE <<<<<<<<<<<<<<<<<<<<<<<

10:41:07 2016-10-21 [WRN] BTCD: open /home/USER/.factom/NONE.conf: no such file or directory
peers.json is disabled for Milestone1
10:41:07 2016-10-21 [INF] BTCD: Version 0.9.0-beta
10:41:07 2016-10-21 [INF] SRVR: Server listening on 0.0.0.0:8108 ; MaxPeers= 125
10:41:07 2016-10-21 [INF] SRVR: Server listening on [::]:8108 ; MaxPeers= 125
10:41:07 2016-10-21 [INF] RPCS: RPC server listening on 127.0.0.1:8384
10:41:08 2016-10-21 [INF] DISC: 1 addresses found from DNS seed factom.network
10:41:21 2016-10-21 [INF] PEER: Starting peer 52.19.117.149:8108 (outbound)
10:41:21 2016-10-21 [INF] BMGR: New valid peer 52.19.117.149:8108 (outbound) (/btcwire:0.2.0/btcd:0.9.0/)
10:41:21 2016-10-21 [INF] BMGR: Latest DirBlock Height: 59457
10:41:21 2016-10-21 [INF] BMGR: LatestDirBlockLocator: (blockchain.BlockLocator) (len=26 cap=500) {
(*wire.ShaHash)(0x21b27f40)((len=32 cap=32) 2cc7c7524bccc34388247189518586dcb6d2444220a9408738d8857e4449eb8a),
...

(*wire.ShaHash)(0x41cd37a0)((len=32 cap=32) cbd3d09db6defdc25dfc7d57f3479b339a077183cd67022e6d1ef6c041522b40)
}

10:41:21 2016-10-21 [INF] BMGR: At 59457: syncing to block height 59667 from peer 52.19.117.149:8108
runtime: memory allocated by OS (0xf60df000) not in usable range [0x18f00000,0x98f00000)
fatal error: out of memory

untime stack:
runtime.throw(0x886d070, 0xd)
/usr/local/go/src/runtime/panic.go:547 +0x7f
runtime.(*mcache).refill(0xf776f4b0, 0x3, 0xf6212b48)
/usr/local/go/src/runtime/mcache.go:121 +0xce
runtime.mallocgc.func2()
/usr/local/go/src/runtime/malloc.go:642 +0x2b
runtime.systemstack(0x1935cd20)
/usr/local/go/src/runtime/asm_386.s:313 +0x5e
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1051

goroutine 45 [running]:

...

@carryforward
Copy link
Contributor

are you building from source? it looks like you are still running the code compiled in 1.6. See the line:

Go compiler version: go1.6.2

you might want to try ./factomd in the directory where you saved the experimental versions.
It is not enough to just update golang on your computer. the binaries that are run are compiled, so they ignore the currently installed version of golang unless you recompile.

@hrdwdmrbl
Copy link
Author

I tried running it again on a machine with 2GB of memory and that still isn't enough

@hrdwdmrbl
Copy link
Author

If I want to try building from source, but still have it work with the existing M1 network, how do I do that?

@mberry
Copy link

mberry commented Oct 26, 2016

Can you confirm that you are running Go1.7?

Second line should tell you the compiled version.

On Wed, Oct 26, 2016 at 11:44 AM, Marc Beaupré-Pham <
notifications@github.com> wrote:

If I want to try building from source, but still have it work with the
existing M1 network, how do I do that?


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#26 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQljMzr5W_d2VqQg5JkarvqURcfiOnhKks5q3qJhgaJpZM4KMLVH
.

@hrdwdmrbl
Copy link
Author

I haven't yet tried compiling from source. I can make sure to try 1.7 when/if I do.

I actually did get it working though, using @tdkmatt 's method

@carryforward
Copy link
Contributor

please try the new installer for v 0.3.8.0
https://github.com/FactomProject/distribution

@vbaranov
Copy link

Hi,

I faced with the same issue with new binaries (0.3.8.0).
We use factom-0.3.8.0-i386.deb at Ubuntu 14.04.3 (1GB RAM).

Log at the moment of factomd crashing:
runtime: out of memory: cannot allocate 65536-byte block (3489660928 in use)

It was syncing from 67775 to 67802 block

Memory usage before crashing:

             total       used       free     shared    buffers     cached
Mem:           994        929         64          0          0          3
-/+ buffers/cache:        926         67
Swap:         4095       2886       1209

Any ideas?

@carryforward
Copy link
Contributor

carryforward commented Dec 16, 2016

I don't think that the current m1 code works well with only 1 gb of ram. Try bumping that up until we get m2 launched.

@igorbarinov
Copy link

@carryforward technically the host has 1Gb of RAM + 4Gb of swap == 5Gb of RAM. Any ideas why the process didn't use the swap?

@igorbarinov
Copy link

@carryforward any updates on this issue?
I've added 4Gb more swap. Now its 8Gb
can't start the factomd

root@factom:/# factomd
2017/01/03 22:49:17 read factom config file:  /root/.factom/factomd.conf
Go compiler version: go1.7.3
boltDBpath: /root/.factom/
2017/01/03 22:49:17 web.go serving :8088

'factomd initializeonly' will do just that.  Initialize and stop.


>>>>>>>>>>>>>>>>>  CLIENT MODE <<<<<<<<<<<<<<<<<<<<<<<


22:49:29 2017-01-03 [WRN] BTCD: open /root/.factom/NONE.conf: no such file or directory
22:49:29 2017-01-03 [INF] BTCD: Version 0.9.0-beta
peers.json is disabled for Milestone1
22:49:29 2017-01-03 [INF] SRVR: Server listening on 0.0.0.0:8108 ; MaxPeers= 125
22:49:29 2017-01-03 [INF] SRVR: Server listening on [::]:8108 ; MaxPeers= 125
22:49:29 2017-01-03 [INF] RPCS: RPC server listening on 127.0.0.1:8384
22:49:29 2017-01-03 [INF] DISC: 1 addresses found from DNS seed factom.network
runtime: out of memory: cannot allocate 65536-byte block (3489660928 in use)
fatal error: out of memory

runtime stack:
runtime.throw(0x875cc77, 0xd)
	/usr/local/go/src/runtime/panic.go:566 +0x7f
runtime.(*mcache).refill(0xf7747000, 0x3, 0x18fafbbc)
	/usr/local/go/src/runtime/mcache.go:123 +0xd2
runtime.(*mcache).nextFree.func1()
	/usr/local/go/src/runtime/malloc.go:505 +0x2b
runtime.systemstack(0x18e1a000)
	/usr/local/go/src/runtime/asm_386.s:323 +0x5e
runtime.mstart()
	/usr/local/go/src/runtime/proc.go:1079

goroutine 20 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_386.s:277 fp=0x18fafb5c sp=0x18fafb58
runtime.(*mcache).nextFree(0xf7747000, 0x3, 0x20, 0xf31569c0, 0xecf8d000)
	/usr/local/go/src/runtime/malloc.go:506 +0x129 fp=0x18fafb80 sp=0x18fafb5c
runtime.mallocgc(0x20, 0x0, 0xe8dfdf00, 0x1d)
	/usr/local/go/src/runtime/malloc.go:658 +0x6e3 fp=0x18fafbe8 sp=0x18fafb80
runtime.rawbyteslice(0x1d, 0x0, 0x0, 0x0)
	/usr/local/go/src/runtime/string.go:307 +0x9d fp=0x18fafc10 sp=0x18fafbe8
runtime.stringtoslicebyte(0x0, 0x8769444, 0x1d, 0x0, 0x0, 0x0)
	/usr/local/go/src/runtime/string.go:145 +0xd9 fp=0x18fafc44 sp=0x18fafc10
github.com/FactomProject/factoid/state.(*FactoidState).UpdateECBalance(0x19223230, 0x8b797a0, 0xe8dfdfa0, 0x1, 0x0, 0x0, 0x0)
	/home/brian/go/src/github.com/FactomProject/factoid/state/factoidstate.go:504 +0xb7 fp=0x18fafc9c sp=0x18fafc44
github.com/FactomProject/FactomCode/process.initializeECreditMap(0x18fafd58)
	/home/brian/go/src/github.com/FactomProject/FactomCode/process/init.go:251 +0x46f fp=0x18fafd30 sp=0x18fafc9c
github.com/FactomProject/FactomCode/process.initECChain()
	/home/brian/go/src/github.com/FactomProject/FactomCode/process/init.go:93 +0x162 fp=0x18fafdbc sp=0x18fafd30
github.com/FactomProject/FactomCode/process.initProcessor()
	/home/brian/go/src/github.com/FactomProject/FactomCode/process/processor.go:133 +0x337 fp=0x18fafec4 sp=0x18fafdbc
github.com/FactomProject/FactomCode/process.Start_Processor(0x8b7b700, 0x192214c0, 0x19220fc0, 0x19221000, 0x19221040, 0x19221080)
	/home/brian/go/src/github.com/FactomProject/FactomCode/process/processor.go:194 +0xd6 fp=0x18faffc0 sp=0x18fafec4
runtime.goexit()
	/usr/local/go/src/runtime/asm_386.s:1612 +0x1 fp=0x18faffc4 sp=0x18faffc0
created by main.factomdMain
	/home/brian/go/src/github.com/FactomProject/FactomCode/factomd/factomd.go:100 +0x87

goroutine 1 [chan receive, 2 minutes]:
github.com/FactomProject/btcd.btcdMain(0x0, 0x0, 0x0)
	/home/brian/go/src/github.com/FactomProject/btcd/btcd.go:158 +0x58f
github.com/FactomProject/btcd.Start_btcd(0x8b7b700, 0x192214c0, 0x19220fc0, 0x19221000, 0x19221040, 0x19221080, 0x192244e0, 0x8, 0x19224600, 0xb, ...)
	/home/brian/go/src/github.com/FactomProject/btcd/factom.go:197 +0x1d5
main.factomdMain(0x0, 0x0)
	/home/brian/go/src/github.com/FactomProject/FactomCode/factomd/factomd.go:130 +0x346
main.main()
	/home/brian/go/src/github.com/FactomProject/FactomCode/factomd/factomd.go:91 +0x6aa

goroutine 5 [semacquire, 2 minutes]:
sync.runtime_notifyListWait(0x192088e8, 0x0)
	/usr/local/go/src/runtime/sema.go:267 +0x17b
sync.(*Cond).Wait(0x192088e0)
	/usr/local/go/src/sync/cond.go:57 +0x8d
github.com/btcsuitereleases/seelog.(*asyncLoopLogger).processItem(0x18e47ac0, 0x0)
	/home/brian/go/src/github.com/btcsuitereleases/seelog/behavior_asynclooplogger.go:50 +0xaf
github.com/btcsuitereleases/seelog.(*asyncLoopLogger).processQueue(0x18e47ac0)
	/home/brian/go/src/github.com/btcsuitereleases/seelog/behavior_asynclooplogger.go:63 +0x2a
created by github.com/btcsuitereleases/seelog.newAsyncLoopLogger
	/home/brian/go/src/github.com/btcsuitereleases/seelog/behavior_asynclooplogger.go:40 +0x79

goroutine 6 [semacquire, 2 minutes]:
sync.runtime_notifyListWait(0x19208aa8, 0x0)
	/usr/local/go/src/runtime/sema.go:267 +0x17b
sync.(*Cond).Wait(0x19208aa0)
	/usr/local/go/src/sync/cond.go:57 +0x8d
github.com/btcsuitereleases/seelog.(*asyncLoopLogger).processItem(0x18e47b40, 0x0)
	/home/brian/go/src/github.com/btcsuitereleases/seelog/behavior_asynclooplogger.go:50 +0xaf
github.com/btcsuitereleases/seelog.(*asyncLoopLogger).processQueue(0x18e47b40)
	/home/brian/go/src/github.com/btcsuitereleases/seelog/behavior_asynclooplogger.go:63 +0x2a
created by github.com/btcsuitereleases/seelog.newAsyncLoopLogger
	/home/brian/go/src/github.com/btcsuitereleases/seelog/behavior_asynclooplogger.go:40 +0x79

goroutine 15 [select, 2 minutes]:
github.com/FactomProject/goleveldb/leveldb.(*DB).compactionError(0x18e640d0)
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db_compaction.go:120 +0x3fd
created by github.com/FactomProject/goleveldb/leveldb.openDB
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db.go:126 +0x6dc

goroutine 9 [semacquire, 2 minutes]:
sync.runtime_notifyListWait(0x19227b28, 0x0)
	/usr/local/go/src/runtime/sema.go:267 +0x17b
sync.(*Cond).Wait(0x19227b20)
	/usr/local/go/src/sync/cond.go:57 +0x8d
github.com/FactomProject/seelog.(*asyncLoopLogger).processItem(0x19220a40, 0x0)
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_asynclooplogger.go:50 +0xaf
github.com/FactomProject/seelog.(*asyncLoopLogger).processQueue(0x19220a40)
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_asynclooplogger.go:63 +0x2a
created by github.com/FactomProject/seelog.newAsyncLoopLogger
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_asynclooplogger.go:40 +0x79

goroutine 10 [semacquire, 2 minutes]:
sync.runtime_notifyListWait(0x19227ca8, 0x0)
	/usr/local/go/src/runtime/sema.go:267 +0x17b
sync.(*Cond).Wait(0x19227ca0)
	/usr/local/go/src/sync/cond.go:57 +0x8d
github.com/FactomProject/seelog.(*asyncLoopLogger).processItem(0x19220b40, 0x0)
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_asynclooplogger.go:50 +0xaf
github.com/FactomProject/seelog.(*asyncLoopLogger).processQueue(0x19220b40)
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_asynclooplogger.go:63 +0x2a
created by github.com/FactomProject/seelog.newAsyncLoopLogger
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_asynclooplogger.go:40 +0x79

goroutine 11 [syscall, 2 minutes]:
os/signal.signal_recv(0x0)
	/usr/local/go/src/runtime/sigqueue.go:116 +0x11e
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x1a
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x36

goroutine 12 [IO wait, 2 minutes]:
net.runtime_pollWait(0xf76f22e8, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:160 +0x55
net.(*pollDesc).wait(0x19221978, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x35
net.(*pollDesc).waitRead(0x19221978, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x33
net.(*netFD).accept(0x19221940, 0x0, 0x8b701b0, 0x19218bf0)
	/usr/local/go/src/net/fd_unix.go:419 +0x20b
net.(*TCPListener).accept(0x18e0adb8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:132 +0x2d
net.(*TCPListener).AcceptTCP(0x18e0adb8, 0x8b95588, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock.go:209 +0x8f
net/http.tcpKeepAliveListener.Accept(0x18e0adb8, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2608 +0x3d
net/http.(*Server).Serve(0x18e72870, 0x8b72da0, 0x18e0adb8, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2273 +0x216
net/http.(*Server).ListenAndServe(0x18e72870, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2219 +0xd9
net/http.ListenAndServe(0x19218bb0, 0x5, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2351 +0x7a
github.com/FactomProject/FactomCode/controlpanel.runPanel()
	/home/brian/go/src/github.com/FactomProject/FactomCode/controlpanel/webserver.go:162 +0x135
created by github.com/FactomProject/FactomCode/controlpanel.(*ControlPanel).Run
	/home/brian/go/src/github.com/FactomProject/FactomCode/controlpanel/controlpanel.go:100 +0x83

goroutine 13 [select]:
github.com/FactomProject/goleveldb/leveldb/util.(*BufferPool).drain(0x18e0c8f0)
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/util/buffer_pool.go:205 +0x207
created by github.com/FactomProject/goleveldb/leveldb/util.NewBufferPool
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/util/buffer_pool.go:236 +0x1da

goroutine 16 [select]:
github.com/FactomProject/goleveldb/leveldb.(*DB).mpoolDrain(0x18e640d0)
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db_state.go:73 +0x11a
created by github.com/FactomProject/goleveldb/leveldb.openDB
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db.go:127 +0x6f9

goroutine 17 [select, 2 minutes]:
github.com/FactomProject/goleveldb/leveldb.(*DB).tCompaction(0x18e640d0)
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db_compaction.go:812 +0x602
created by github.com/FactomProject/goleveldb/leveldb.openDB
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db.go:130 +0x73b

goroutine 18 [select, 2 minutes]:
github.com/FactomProject/goleveldb/leveldb.(*DB).mCompaction(0x18e640d0)
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db_compaction.go:759 +0x1f6
created by github.com/FactomProject/goleveldb/leveldb.openDB
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db.go:131 +0x758

goroutine 19 [select, 2 minutes]:
github.com/FactomProject/goleveldb/leveldb.(*DB).jWriter(0x18e640d0)
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db_write.go:37 +0x158
created by github.com/FactomProject/goleveldb/leveldb.openDB
	/home/brian/go/src/github.com/FactomProject/goleveldb/leveldb/db.go:132 +0x775

goroutine 21 [IO wait, 2 minutes]:
net.runtime_pollWait(0xf76f2270, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:160 +0x55
net.(*pollDesc).wait(0x18e46878, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x35
net.(*pollDesc).waitRead(0x18e46878, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x33
net.(*netFD).accept(0x18e46840, 0x0, 0x8b701b0, 0x18f9c320)
	/usr/local/go/src/net/fd_unix.go:419 +0x20b
net.(*TCPListener).accept(0x18f8ecb0, 0x86c1de0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:132 +0x2d
net.(*TCPListener).Accept(0x18f8ecb0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock.go:222 +0x9b
net/http.(*Server).Serve(0x18e720a0, 0x8b72940, 0x18f8ecb0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2273 +0x216
net/http.Serve(0x8b72940, 0x18f8ecb0, 0x8b6fd90, 0x18fa2460, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2091 +0x74
github.com/FactomProject/web.(*Server).Run(0x18ebec00, 0x18f9c1d0, 0x5)
	/home/brian/go/src/github.com/FactomProject/web/server.go:155 +0x4fc
created by github.com/FactomProject/FactomCode/wsapi.Start
	/home/brian/go/src/github.com/FactomProject/FactomCode/wsapi/wsapi.go:69 +0x5d5

goroutine 23 [semacquire, 2 minutes]:
sync.runtime_notifyListWait(0x3180da48, 0x0)
	/usr/local/go/src/runtime/sema.go:267 +0x17b
sync.(*Cond).Wait(0x3180da40)
	/usr/local/go/src/sync/cond.go:57 +0x8d
github.com/FactomProject/seelog.(*asyncAdaptiveLogger).processItem(0x27760d20, 0x32d61300, 0x0)
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_adaptivelogger.go:94 +0xc3
github.com/FactomProject/seelog.(*asyncAdaptiveLogger).processQueue(0x27760d20)
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_adaptivelogger.go:119 +0x2a
created by github.com/FactomProject/seelog.newAsyncAdaptiveLogger
	/home/brian/go/src/github.com/FactomProject/seelog/behavior_adaptivelogger.go:84 +0x30c

goroutine 24 [select, 2 minutes, locked to thread]:
runtime.gopark(0x8921fb8, 0x0, 0x8756858, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:259 +0x130
runtime.selectgoImpl(0x18f94f90, 0x0, 0xc)
	/usr/local/go/src/runtime/select.go:423 +0xbf0
runtime.selectgo(0x18f94f90)
	/usr/local/go/src/runtime/select.go:238 +0xf
runtime.ensureSigM.func1()
	/usr/local/go/src/runtime/signal1_unix.go:304 +0x2fd
runtime.goexit()
	/usr/local/go/src/runtime/asm_386.s:1612 +0x1

goroutine 25 [select, 2 minutes]:
github.com/FactomProject/btcd.mainInterruptHandler()
	/home/brian/go/src/github.com/FactomProject/btcd/signal.go:35 +0x23b
created by github.com/FactomProject/btcd.addInterruptHandler
	/home/brian/go/src/github.com/FactomProject/btcd/signal.go:80 +0xc0

goroutine 26 [IO wait, 2 minutes]:
net.runtime_pollWait(0xf76f21f8, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:160 +0x55
net.(*pollDesc).wait(0x31816fb8, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x35
net.(*pollDesc).waitRead(0x31816fb8, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x33
net.(*netFD).accept(0x31816f80, 0x0, 0x8b701b0, 0x321deb60)
	/usr/local/go/src/net/fd_unix.go:419 +0x20b
net.(*TCPListener).accept(0x21ef5bc8, 0x320b1560, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:132 +0x2d
net.(*TCPListener).Accept(0x21ef5bc8, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock.go:222 +0x9b
github.com/FactomProject/btcd.(*server).listenHandler(0x3181e480, 0x8b72940, 0x21ef5bc8)
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:508 +0x179
created by github.com/FactomProject/btcd.(*server).Start
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:923 +0x1a2

goroutine 27 [IO wait, 2 minutes]:
net.runtime_pollWait(0xf76f2180, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:160 +0x55
net.(*pollDesc).wait(0x31816ff8, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x35
net.(*pollDesc).waitRead(0x31816ff8, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x33
net.(*netFD).accept(0x31816fc0, 0x0, 0x8b701b0, 0x321deb90)
	/usr/local/go/src/net/fd_unix.go:419 +0x20b
net.(*TCPListener).accept(0x21ef5bd8, 0x320b1590, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:132 +0x2d
net.(*TCPListener).Accept(0x21ef5bd8, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock.go:222 +0x9b
github.com/FactomProject/btcd.(*server).listenHandler(0x3181e480, 0x8b72940, 0x21ef5bd8)
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:508 +0x179
created by github.com/FactomProject/btcd.(*server).Start
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:923 +0x1a2

goroutine 28 [sleep]:
time.Sleep(0x5f5e100, 0x0)
	/usr/local/go/src/runtime/time.go:59 +0xd4
github.com/FactomProject/btcd.(*server).peerHandler(0x3181e480)
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:670 +0x693
created by github.com/FactomProject/btcd.(*server).Start
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:929 +0x1f9

goroutine 29 [select, 2 minutes]:
github.com/FactomProject/btcd.(*server).rebroadcastHandler(0x3181e480)
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:862 +0x4fc
created by github.com/FactomProject/btcd.(*server).Start
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:941 +0x28a

goroutine 30 [IO wait, 2 minutes]:
net.runtime_pollWait(0xf76f2108, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:160 +0x55
net.(*pollDesc).wait(0x320b3f38, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x35
net.(*pollDesc).waitRead(0x320b3f38, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x33
net.(*netFD).accept(0x320b3f00, 0x0, 0x8b701b0, 0x321dec50)
	/usr/local/go/src/net/fd_unix.go:419 +0x20b
net.(*TCPListener).accept(0x1e026dd8, 0x321efdb4, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:132 +0x2d
net.(*TCPListener).Accept(0x1e026dd8, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock.go:222 +0x9b
crypto/tls.(*listener).Accept(0x321188a0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/tls.go:52 +0x4f
net/http.(*Server).Serve(0x277614f0, 0x8b72780, 0x321188a0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2273 +0x216
github.com/FactomProject/btcd.(*rpcServer).Start.func3(0x277614f0, 0x26920380, 0x8b72780, 0x321188a0)
	/home/brian/go/src/github.com/FactomProject/btcd/rpcserver.go:349 +0x100
created by github.com/FactomProject/btcd.(*rpcServer).Start
	/home/brian/go/src/github.com/FactomProject/btcd/rpcserver.go:352 +0x2c7

goroutine 31 [select, 2 minutes]:
github.com/FactomProject/btcd.queueHandler(0x31817380, 0x318173c0, 0x31817440)
	/home/brian/go/src/github.com/FactomProject/btcd/rpcwebsocket.go:138 +0x31d
github.com/FactomProject/btcd.(*wsNotificationManager).queueHandler(0x316bf0e0)
	/home/brian/go/src/github.com/FactomProject/btcd/rpcwebsocket.go:179 +0x32
created by github.com/FactomProject/btcd.(*wsNotificationManager).Start
	/home/brian/go/src/github.com/FactomProject/btcd/rpcwebsocket.go:827 +0x50

goroutine 32 [select, 2 minutes]:
github.com/FactomProject/btcd.(*wsNotificationManager).notificationHandler(0x316bf0e0)
	/home/brian/go/src/github.com/FactomProject/btcd/rpcwebsocket.go:291 +0xd8e
created by github.com/FactomProject/btcd.(*wsNotificationManager).Start
	/home/brian/go/src/github.com/FactomProject/btcd/rpcwebsocket.go:828 +0x6d

goroutine 33 [chan receive, 2 minutes]:
github.com/FactomProject/btcd.factomForkInit.func1(0x3181e480)
	/home/brian/go/src/github.com/FactomProject/btcd/factom.go:45 +0x50
created by github.com/FactomProject/btcd.factomForkInit
	/home/brian/go/src/github.com/FactomProject/btcd/factom.go:80 +0x56

goroutine 34 [chan receive, 2 minutes]:
github.com/FactomProject/btcd.factomForkInit.func2()
	/home/brian/go/src/github.com/FactomProject/btcd/factom.go:83 +0x4b
created by github.com/FactomProject/btcd.factomForkInit
	/home/brian/go/src/github.com/FactomProject/btcd/factom.go:131 +0x6b

goroutine 35 [semacquire, 2 minutes]:
sync.runtime_Semacquire(0x3181e4f4)
	/usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*WaitGroup).Wait(0x3181e4e8)
	/usr/local/go/src/sync/waitgroup.go:131 +0xa4
github.com/FactomProject/btcd.(*server).WaitForShutdown(0x3181e480)
	/home/brian/go/src/github.com/FactomProject/btcd/server.go:995 +0x2b
github.com/FactomProject/btcd.btcdMain.func4(0x3181e480)
	/home/brian/go/src/github.com/FactomProject/btcd/btcd.go:151 +0x21
created by github.com/FactomProject/btcd.btcdMain
	/home/brian/go/src/github.com/FactomProject/btcd/btcd.go:154 +0x571

goroutine 36 [select, 2 minutes]:
github.com/FactomProject/btcd/addrmgr.(*AddrManager).addressHandler(0x18e13080)
	/home/brian/go/src/github.com/FactomProject/btcd/addrmgr/addrmanager.go:347 +0x21f
created by github.com/FactomProject/btcd/addrmgr.(*AddrManager).Start
	/home/brian/go/src/github.com/FactomProject/btcd/addrmgr/addrmanager.go:558 +0x1b2

goroutine 37 [select, 2 minutes]:
github.com/FactomProject/btcd.(*blockManager).blockHandler(0x3173ef00)
	/home/brian/go/src/github.com/FactomProject/btcd/blockmanager.go:1064 +0x8e3
created by github.com/FactomProject/btcd.(*blockManager).Start
	/home/brian/go/src/github.com/FactomProject/btcd/blockmanager.go:1347 +0x132
root@factom:/#

@carryforward
Copy link
Contributor

This covers M1 which has been deprecated for a few months now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants