quorum-examples/7nodes/start.sh may send transaction through node that has yet to come up fully

[conor10]

System information

Geth version: 1.5.0-unstable
OS & Version: Quickstart environment using Vagrant 1.7.4 on OS X Sierra
Commit hash : (if develop)

Expected behaviour

Node runs.

Actual behaviour

Node died.

Steps to reproduce the behaviour

I've not been able to reproduce the issue - the node subsequently started when I ran it manually. However, when I ran start.sh first time around while following the instructions on the Quorum quickstart, the first geth node fell over after starting. The log is below.

Backtrace

I1201 04:39:25.712308 ethdb/database.go:81] Allotted 128MB cache and 1024 file handles to /home/ubuntu/quorum-examples/7nodes/qdata/dd1/geth/chaindata
I1201 04:39:25.745917 ethdb/database.go:174] closed db:/home/ubuntu/quorum-examples/7nodes/qdata/dd1/geth/chaindata
I1201 04:39:25.746230 node/node.go:177] instance: Geth/v1.5.0-unstable-85327f74/linux/go1.7.3
I1201 04:39:25.746247 ethdb/database.go:81] Allotted 128MB cache and 1024 file handles to /home/ubuntu/quorum-examples/7nodes/qdata/dd1/geth/chaindata
I1201 04:39:26.615043 eth/db_upgrade.go:346] upgrading db log bloom bins
I1201 04:39:26.615161 eth/db_upgrade.go:354] upgrade completed in 121.438µs
I1201 04:39:26.615191 eth/backend.go:175] Protocol Versions: [63 62], Network Id: 87234
I1201 04:39:26.615627 core/blockchain.go:221] Last header: #0 [8bb91123…] TD=0
I1201 04:39:26.615644 core/blockchain.go:222] Last block: #0 [8bb91123…] TD=0
I1201 04:39:26.615653 core/blockchain.go:223] Fast block: #0 [8bb91123…] TD=0
I1201 04:39:26.616186 p2p/server.go:328] Starting Server
I1201 04:39:28.784700 p2p/discover/udp.go:217] Listening, enode://17cd61af0a5a9ae2b93fdf429502dde22476052c8ecd77de638ae33bc74c254a46b22a7247bb9d85f88381dd2f1dac684ee43b5e242c757f0deff2cc3fba55de@[::]:21000
I1201 04:39:28.788913 node/node.go:412] HTTP endpoint opened: http://localhost:22000
I1201 04:39:30.860670 p2p/server.go:582] Listening on [::]:21000
I1201 04:39:30.860941 node/node.go:342] IPC endpoint opened: /home/ubuntu/quorum-examples/7nodes/qdata/dd1/geth.ipc
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x534e12]

goroutine 84 [running]:
panic(0xc672a0, 0xc420012090)
        /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/ethereum/go-ethereum/core/state.(*ManagedState).GetNonce(0x0, 0x1848b382e3029ded, 0x71fec709a3888be8, 0x9d415fe6, 0x0)
        /home/ubuntu/quorum/build/_workspace/src/github.com/ethereum/go-ethereum/core/state/managed_state.go:87 +0x32
github.com/ethereum/go-ethereum/eth.(*EthApiBackend).GetPoolNonce(0xc4200de580, 0x7f8915388980, 0xc4201c7ec0, 0x1848b382e3029ded, 0x71fec709a3888be8, 0x9d415fe6, 0x0, 0x0, 0x0)
        /home/ubuntu/quorum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/api_backend.go:153 +0xb9
github.com/ethereum/go-ethereum/internal/ethapi.(*PublicTransactionPoolAPI).SendTransaction(0xc4201ce200, 0x7f8915388980, 0xc4201c7ec0, 0x1848b382e3029ded, 0x71fec709a3888be8, 0x9d415fe6, 0x0, 0xc4200db9c0, 0xc4200dbae0, 0xc4200dbdc0, ...)
        /home/ubuntu/quorum/build/_workspace/src/github.com/ethereum/go-ethereum/internal/ethapi/api.go:1076 +0xfd7
reflect.Value.call(0xc42014fc80, 0xc4200ded88, 0x13, 0xd4b09d, 0x4, 0xc4203e55c0, 0x3, 0x4, 0x0, 0xca5c40, ...)
        /usr/local/go/src/reflect/value.go:434 +0x5c8
reflect.Value.Call(0xc42014fc80, 0xc4200ded88, 0x13, 0xc4203e55c0, 0x3, 0x4, 0x1, 0x1, 0x0)
        /usr/local/go/src/reflect/value.go:302 +0xa4
github.com/ethereum/go-ethereum/rpc.(*Server).handle(0xc42028cf30, 0x7f8915388980, 0xc4201c7ec0, 0x154a720, 0xc42008f220, 0xc42026c700, 0x0, 0x0, 0x0)
        /home/ubuntu/quorum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/server.go:318 +0x8ed
github.com/ethereum/go-ethereum/rpc.(*Server).exec(0xc42028cf30, 0x7f8915388980, 0xc4201c7ec0, 0x154a720, 0xc42008f220, 0xc42026c700)
        /home/ubuntu/quorum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/server.go:340 +0x1e4
created by github.com/ethereum/go-ethereum/rpc.(*Server).serveRequest
        /home/ubuntu/quorum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/server.go:213 +0x371

[patrickmn]

This is probably because the start script tries to send a transaction before the node has come up fully (depending on disk I/O, CPU, etc.)

Right now we are relying on sleeps (and incrementing one between starting the nodes and sending the transaction should fix the problem,) but it could be more robust.