hare certification: sync flow is not working yet tests passed

[countvonzero]

Description

two issues in the implementation:

• early gossip messages are not handled properly, causing the following two logs

{"L":"ERROR","T":"2022-08-22T10:51:10.345-0400","N":"3dd22.blockCert    ","M":"failed to process certify message gossip","node_id":"3dd225dd843c54f9e711d71db4d013f283aca653f55632a1293f393b7cd6d8dc","module":"blockCert","requestId":"d47b5112-7630-49c8-b772-776accb6d6e8","errmsg":"unexpected lid/bid","name":"blockCert"}

{"L":"WARN","T":"2022-08-22T10:51:10.274-0400","N":"ec05f.blockCert    ","M":"layer not registered for cert","node_id":"ec05f85c6f081b7dcf0842a855aa37add882bfcaf8b94ee7463d2f17c2f6f6e1","module":"blockCert","requestId":"2ee0f936-bd18-4751-923f-1f148b52c2fc","layer_id":11,"block_id":"65a664148b"}

• sync'ed certificate is not actually verified, due to the layer/block check

{"L":"WARN","T":"2022-08-22T09:30:01.659-0400","N":"3d88b.layerFetcher ","M":"failed to handle certificate","node_id":"3d88bc875aabe5c1b59803b5d814d22886177bf5434693f99e670da07ca053e6","module":"layerFetcher","sessionId":"76ec9ad8-472e-4a7c-baa4-2c3ea7585b2a","layer_id":15,"peer":"12D3KooWSaj7fTVv33Be6BbXUdQqxxbuBXy9qA8VToQfq4EmE1qz","errmsg":"cert msg too old","name":"layerFetcher"}

another important question: why do all system tests passed?

[countvonzero]

another important question: why do all system tests passed?

mystery solved:

the expected behavior is that if a certificate is not synced from all peers, sync for that layer will fail. so it's expected that in TestAddNode, the last 2 nodes added will fail to catch up.

but here

go-spacemesh/systest/cluster/cluster.go

Lines 309 to 312 in 84b4163

	c.clients = nil
	c.clients = append(c.clients, bootnodes...)
	c.clients = append(c.clients, clients...)
	c.smeshers = len(clients)

the cluster smeshers have been reduced to only 2+bootnodes in TestAddNodes. the 2 nodes "added" also turned out to be existing nodes that are already synced. hence the passing tests.