TestEngineBatch failed

[tbg]

The following test appears to have failed:

#10030:

=== RUN   TestEngineBatchCommit
I1210 13:27:00.051898 1044 storage/engine/rocksdb.go:137  closing in-memory rocksdb instance
--- PASS: TestEngineBatchCommit (0.06s)
=== RUN   TestEngineBatch
I1210 13:27:00.086015 1044 storage/engine/rocksdb.go:137  closing in-memory rocksdb instance
--- FAIL: TestEngineBatch (0.03s)
    engine_test.go:216: 0: expected , but got Coc~roachDBr  BC
    engine_test.go:237: 0: expected , but got Coc~roachDBr  BC
    engine_test.go:248: 0: expected , but got Coc~roachDBr  BC
=== RUN   TestEnginePutGetDelete
I1210 13:27:00.087016 1044 storage/engine/rocksdb.go:137  closing in-memory rocksdb instance
--- PASS: TestEnginePutGetDelete (0.00s)
=== RUN   TestEngineMerge
I1210 13:27:00.089425 1044 storage/engine/rocksdb.go:137  closing in-memory rocksdb instance
--- PASS: TestEngineMerge (0.00s)
=== RUN   TestEngineScan1
--
=== RUN   TestWillOverflow
--- PASS: TestWillOverflow (0.00s)
=== RUN   TestRocksDBCompaction
I1210 13:27:00.664346 1044 storage/engine/rocksdb.go:137  closing in-memory rocksdb instance
--- PASS: TestRocksDBCompaction (0.01s)
FAIL
FAIL    github.com/cockroachdb/cockroach/storage/engine 0.706s
?       github.com/cockroachdb/cockroach/storage/engine/rocksdb [no test files]
?       github.com/cockroachdb/cockroach/storage/simulation [no test files]
?       github.com/cockroachdb/cockroach/testutils  [no test files]
?       github.com/cockroachdb/cockroach/testutils/gossiputil   [no test files]
=== RUN   TestStoreTimeSeries
I1210 13:27:17.274467 1094 gossip/gossip.go:189  gossiping node descriptor node_id:1 address:<network_field:"" address_field:"" > attrs:<> 
I1210 13:27:17.275297 1094 multiraft/multiraft.go:579  node 1 starting
I1210 13:27:17.275561 1094 raft/raft.go:441  [group 1] 1 became follower at term 5
I1210 13:27:17.275642 1094 raft/raft.go:234  [group 1] newRaft 1 [peers: [1], term: 5, commit: 10, applied: 10, lastindex: 10, lastterm: 5]
I1210 13:27:17.275703 1094 multiraft/multiraft.go:999  node 1 campaigning because initial confstate is [1]

Please assign, take a look and update the issue accordingly.

[petermattis]

Looks like one of my MVCC changes caused instability in some of the storage/engine tests. I'm taking a look.

[petermattis]

I suspect #3337 as that is the first PR this test failed in. But I'm completely unable to reproduce a failure locally (10s of thousands of runs with and without -race).

[tbg]

are you running that test in isolation? Try throwing some others in the mix. But this looks ... weird.

[petermattis]

I've run it both in isolation and with all of the tests in the storage/engine directory.