multiple blocks per layer

[dshulyak]

{"L":"INFO","T":"2023-08-12T05:15:04.277Z","N":"9787b.trtl","M":"candidate layer is verified","node_id":"9787b82b2f7495406f8642446928feee3e3e52a351c5a6706ca6ce9bbc13e81e","module":"trtl","blocks":[{"decision":"support","id":"656ea3fcbf","weight":"+0'00000000000000/56","height":0,"data":true},{"decision":"against","id":"c15abd4ded","weight":"+0'00000000000000/56","height":0,"data":false},{"decision":"against","id":"feecfb6084","weight":"+0'00000000000000/56","height":0,"data":false}],"verifier":"verifying","candidate layer":8312,"margin":346615.39070514124,"global threshold":172808.66550925924,"uncounted":48874.87458790554,"total good weight":13138406.820582757,"good uncounted":12742916.55528971}

they got invalidated by verifying tortoise in this case, but this clearly not expected behavior

[dshulyak]

it seems like a bug around mempool. i checked blocks in layer 36277. they have same rewards, but one block includes more transactions

  &types.Block(
- 	{
- 		InnerBlock: types.InnerBlock{
- 			LayerIndex: s"36277",
- 			TickHeight: 56316,
- 			Rewards:    []types.AnyReward{{...}, {...}, {...}, {...}, ...},
- 			TxIDs:      []types.TransactionID{s"23f8965c26", s"4e2362007d", s"bccd2fba65", s"d279757a5d", ...},
- 		},
- 		blockID: s"71aaa11209",
- 	},
+ 	{
+ 		InnerBlock: types.InnerBlock{
+ 			LayerIndex: s"36277",
+ 			TickHeight: 56316,
+ 			Rewards:    []types.AnyReward{{...}, {...}, {...}, {...}, ...},
+ 			TxIDs:      []types.TransactionID{s"4e2362007d", s"299a90ba07", s"bccd2fba65", s"23f8965c26", ...},
+ 		},
+ 		blockID: s"8356f390c7",
+ 	},
  )


  []types.TransactionID{
- 	s"23f8965c26",
  	{0x4e, 0x23, 0x62, 0x00, ...},
+ 	s"299a90ba07",
  	{0xbc, 0xcd, 0x2f, 0xba, ...},
+ 	s"23f8965c26",
+ 	s"add50facaf",
  	{0xd2, 0x79, 0x75, 0x7a, ...},
  	{0x77, 0x2a, 0x95, 0x92, ...},
  	{0x1d, 0xec, 0x1a, 0x08, ...},
  }

[lrettig]

In my database today I see three layers that still contain multiple blocks:

> SELECT layer, COUNT(id) FROM blocks GROUP BY layer HAVING COUNT(id) > 1 ORDER BY layer DESC;

40557 | 2
36292 | 2
36290 | 2

> SELECT SUBSTR(HEX(id), 0, 10) as block_id, validity, layer FROM blocks WHERE layer IN (36290, 36292, 40557) ORDER BY layer, validity;

EFEAE6A08 | -1 | 36290
4FAA4D0D7 |  1 | 36290
BA11FDE80 | -1 | 36292
43CEDD89B |  1 | 36292
6186038AE | -1 | 40557
67907E533 |  1 | 40557

[dshulyak]

blocks are still in the database because we don't prune them. what i meant in my comment above is that if node somehow generated invalid block it will revert to the correct one. obviously needs to be fixed.