fix: confirm block import notifier is closed properly #1736

edwardmack · 2021-08-18T19:47:24Z

Changes

Refactor Block Import notification so that channel is handled in block_notify logic.
Replace RegisterImportedBlockChannel with GetImportedBlockNotifierChannel which returns channel that will be signaled when a block is imported.
Replace UnregisterImportedBlockImported with FreeImportedBlockNotifierChannel which removes channel from block imported map, and closes channel.

Tests

make test

Issues

closes: panic: send on closed channel in block import notifier #1701

Primary Reviewer

@timwu20

codecov · 2021-08-18T20:18:32Z

Codecov Report

Merging #1736 (8b30110) into development (a96f06a) will decrease coverage by 0.08%.
The diff coverage is 93.33%.

@@               Coverage Diff               @@
##           development    #1736      +/-   ##
===============================================
- Coverage        59.60%   59.51%   -0.09%     
===============================================
  Files              186      186              
  Lines            19770    19738      -32     
===============================================
- Hits             11784    11748      -36     
- Misses            5968     5978      +10     
+ Partials          2018     2012       -6

Flag	Coverage Δ
unit-tests	`59.51% <93.33%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
lib/grandpa/grandpa.go	`60.41% <33.33%> (+0.47%)`	⬆️
dot/rpc/subscription/listeners.go	`73.57% <95.00%> (+1.90%)`	⬆️
dot/digest/digest.go	`67.16% <100.00%> (-0.80%)`	⬇️
dot/rpc/http.go	`80.95% <100.00%> (-0.18%)`	⬇️
dot/rpc/modules/api_mocks.go	`100.00% <100.00%> (ø)`
dot/rpc/subscription/websocket.go	`63.24% <100.00%> (-1.32%)`	⬇️
dot/state/block.go	`42.19% <100.00%> (-0.39%)`	⬇️
dot/state/block_notify.go	`85.18% <100.00%> (+11.32%)`	⬆️
lib/grandpa/message_tracker.go	`68.18% <100.00%> (-3.82%)`	⬇️
dot/network/sync_benchmark.go	`14.28% <0.00%> (-85.72%)`	⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a96f06a...8b30110. Read the comment docs.

timwu20 · 2021-08-18T20:26:03Z

dot/rpc/subscription/websocket.go

@@ -213,6 +213,8 @@ func (c *WSConn) initStorageChangeListener(reqID float64, params interface{}) (L
 }

 func (c *WSConn) initBlockListener(reqID float64, _ interface{}) (Listener, error) {
+	// TODO (ed) here the imported Channel get unregistered and closed in http.Stop
+	//  there doesn't seem to be an un-subscribe for this


where does BlockListener.Channel get closed? i don't see it in the BlockListener.Stop

Can we move this code to a constructor for BlockListener.

It seems there is logic to close imported channel in dot/http.go line 194-5, which in inside HTTPServer.Stop. There is also within Listener.Stop a call to cancelWithTimeout that gets called when unsubscribe is called. Should we move the imported channel close into Listener.Stop, then call listener.Stop from within HTTPServer.Stop?

Yea we should localise closing channels (managing internal state) to receiver functions of the struct.

@timwu20 the BlockListener.Channel channel is closed when l.wsconn.BlockAPI.UnregisterImportedChannel(l.ChanID) is called, we should not close the BlockListener.Channel directly because the BlockAPI is the owner of the channel. Every listener has a Stop method that calls the function cancelWithTimeout which will stop any running listener's goroutines, which in turn will call the defer function to unregister any active channel

But, the implementation of Unregister... function is to just remove the channel from the map without closing it, and that is the problem. @edwardmack I think we could close the imported channel inside the defer function of each Listen method to keep things closer.

I added imported channel close to defer in Listen methods.

timwu20 · 2021-08-18T20:26:59Z

dot/rpc/subscription/websocket.go

@@ -285,6 +287,7 @@ func (c *WSConn) initExtrinsicWatch(reqID float64, params interface{}) (Listener
 		return nil, err
 	}

+	// TODO (ed) the importedChan get deleted by unregister imported channel in defer in Listen (listeners.go line 236)
 	// listen for built blocks
 	esl := &ExtrinsicSubmitListener{


can we move this code to listeners.go to a NewExtrinsicSubmitListener constructor. That way we can see the creation and closing of channels all in the same file.

Moved code to new NewExtrinsicSubmitListener in listeners.go.

EclesioMeloJunior · 2021-08-20T18:11:53Z

dot/rpc/subscription/websocket.go

@@ -213,6 +213,8 @@ func (c *WSConn) initStorageChangeListener(reqID float64, params interface{}) (L
 }

 func (c *WSConn) initBlockListener(reqID float64, _ interface{}) (Listener, error) {
+	// TODO (ed) here the imported Channel get unregistered and closed in http.Stop
+	//  there doesn't seem to be an un-subscribe for this


@timwu20 the BlockListener.Channel channel is closed when l.wsconn.BlockAPI.UnregisterImportedChannel(l.ChanID) is called, we should not close the BlockListener.Channel directly because the BlockAPI is the owner of the channel. Every listener has a Stop method that calls the function cancelWithTimeout which will stop any running listener's goroutines, which in turn will call the defer function to unregister any active channel

EclesioMeloJunior · 2021-08-20T18:18:44Z

dot/rpc/subscription/websocket.go

@@ -213,6 +213,8 @@ func (c *WSConn) initStorageChangeListener(reqID float64, params interface{}) (L
 }

 func (c *WSConn) initBlockListener(reqID float64, _ interface{}) (Listener, error) {
+	// TODO (ed) here the imported Channel get unregistered and closed in http.Stop
+	//  there doesn't seem to be an un-subscribe for this


But, the implementation of Unregister... function is to just remove the channel from the map without closing it, and that is the problem. @edwardmack I think we could close the imported channel inside the defer function of each Listen method to keep things closer.

edwardmack · 2021-08-23T19:40:43Z

I believe I've updated this for the comments, please re-review and let me know.

noot · 2021-08-24T16:42:14Z

dot/digest/digest.go

@@ -74,6 +74,8 @@ type resume struct {

 // NewHandler returns a new Handler
 func NewHandler(blockState BlockState, epochState EpochState, grandpaState GrandpaState) (*Handler, error) {
+	// TODO (ed) imported channel is unregistered and closed in handler.Stop


is this still a TODO?

noot · 2021-08-24T17:25:55Z

dot/rpc/subscription/listeners.go

+// NewBlockListener constructor for creating BlockListener
+func NewBlockListener(conn *WSConn) *BlockListener {
+	bl := &BlockListener{
+		Channel:       make(chan *types.Block, DEFAULT_BUFFER_SIZE),


can this be renamed blockCh or something like that?

noot · 2021-08-24T17:51:19Z

lib/grandpa/grandpa.go

@@ -377,6 +377,7 @@ func (s *Service) initiate() error {

 func (s *Service) waitForFirstBlock() error {
 	ch := make(chan *types.Block)
+	// TODO (ed) import channel is unregistered by defer, doesn't seem to be closed


I think this function has a problem. you can update the defer func to first close(ch) then call Unregister(), I think that should fix the issue at least partially, maybe there is a case where the GC collects ch and closes it before the Unregister is finished

noot

I wonder if we should refactor the block_notify architecture to prevent this - just having some API like GetNotifierChannel and FreeNotifierChannel which returns a new channel that's managed by the BlockState and then deals with closing it as well. I think that might make more sense than what we currently have where the reader of the channel is closing it, since in general the writer should close the channel

EclesioMeloJunior · 2021-08-26T13:37:24Z

dot/rpc/subscription/listeners.go

@@ -181,6 +194,7 @@ func (l *BlockFinalizedListener) Listen() {
 		defer func() {
 			l.wsconn.BlockAPI.UnregisterFinalisedChannel(l.chanID)
 			close(l.done)
+			close(l.channel)


Following the @arijitAD comment on PR #1769 (https://github.com/ChainSafe/gossamer/pull/1769/files#r696558481), I think that would be nice if the Unregister... method closes the channel and remove it from the map.

this probably shouldn't be closed here? it's a reader of the channel, not the writer, so should remove this

or this should be calling FreeImportedBlockNotifierChannel instead?

@edwardmack re-checking on this

@noot yes, I removed close(l.channel) from here, I'm going to implement FreeFinalisedChannel in another PR, similar to what we did here with FreeImportedBlockNotifierChannel

edwardmack · 2021-09-10T20:49:58Z

@timwu20 This has been updated to reflect our conversation, the interfaces are now GetImportedBlockNotifierChannel() chan *types.Block and FreeImportedBlockNotifierChannel(ch chan *types.Block)

Ready for re-review, if this looks good I'll refactor finalised in another PR.

dot/digest/digest.go

dot/state/block_notify.go

noot

nice work, just a few comments!

timwu20 · 2021-09-21T15:07:14Z

dot/state/block_notify.go

-// UnregisterImportedChannel removes the block import notification channel with the given ID.
-// A channel must be unregistered before closing it.
-func (bs *BlockState) UnregisterImportedChannel(id byte) {
+// FreeImportedBlockNotifierChannel to free and close imported block notifier channel


where is the channel closed?

It's not being closed, I've updated comment.

timwu20 · 2021-09-22T15:08:08Z

dot/digest/digest.go

@@ -401,7 +401,7 @@ func newGrandpaChange(raw []types.GrandpaAuthoritiesRaw, delay uint32, currBlock
 	}, nil
 }

-func (*Handler) handleBABEOnDisabled(_ *types.ConsensusDigest, _ *types.Header) error {
+func (h *Handler) handleBABEOnDisabled(d types.BABEOnDisabled, _ *types.Header) error {


what does this function do? looks like it just logs some struct and returns nil.

I'm not sure why this is here...

noot

looks good - just one comment on the channel close in listeners.go

edwardmack · 2021-09-24T14:30:23Z

Can someone override the DeepSource requirement for this PR, it's complaining about code generated by mocks.
@timwu20 Please re-review and let me know if there are remaining issues.

EclesioMeloJunior · 2021-09-24T14:51:52Z

@edwardmack add the below items to the end of exclude_patterns at .deepsource.toml to DeepSource ignore the generated mock files

 "**/*_test.go",
 "**/mocks/*",
 "**/mock_*"

# [0.6.0](v0.5.0...v0.6.0) (2021-12-03) ### Bug Fixes * **babe:** Fix extrinsic format in block. ([#1530](#1530)) ([1a03b2a](1a03b2a)) * **ci:** add missing go-version matrix to fix development branch CI ([#2037](#2037)) ([6babe76](6babe76)) * **cmd/cfg:** Use Babe Lead value from toml config ([#2032](#2032)) ([06aa3e3](06aa3e3)) * cmd/gossamer: Generate random name if --name flag not set ([#1506](#1506)) ([3c05a88](3c05a88)) * confirm block import notifier is closed properly ([#1736](#1736)) ([ad2d85e](ad2d85e)) * **docs:** improve build-spec usage docs ([#1706](#1706)) ([2e164b4](2e164b4)) * **dot/core:** Add only extrinsic during chain reorg. ([#1609](#1609)) ([29413d4](29413d4)) * **dot/core:** Batch process transaction message. ([#1780](#1780)) ([0064836](0064836)) * **dot/core:** check transaction Validity.Propagate field to determine whether to propagate tx ([#1643](#1643)) ([81f23cc](81f23cc)) * **dot/core:** Fix handle transaction message test. ([#1607](#1607)) ([58b8725](58b8725)) * **dot/network, lib/grandpa:** fix handshake decoding and grandpa message handler sigabort ([#1631](#1631)) ([887f72c](887f72c)) * **dot/network, lib/grandpa:** fix node sync, improve devnet finality ([bcc7935](bcc7935)) * **dot/network:** add nil checks in connManager ([#2069](#2069)) ([7f9c042](7f9c042)) * **dot/network:** Check for size when decoding leb128. ([#1634](#1634)) ([d082b9e](d082b9e)) * **dot/network:** check if peer supports protocol ([#1617](#1617)) ([6bf66a4](6bf66a4)) * **dot/network:** decrease DHT find peers interval for gssmr nodes ([#1703](#1703)) ([08516a0](08516a0)) * **dot/network:** fix bugs in notifications protocol handlers; add metrics for inbound/outbound streams ([#2010](#2010)) ([8c2993d](8c2993d)) * **dot/network:** fix dht connection on discovery on devnet ([#2059](#2059)) ([da065b8](da065b8)) * **dot/network:** fix discovery between gossamer nodes ([#1594](#1594)) ([f4c79d3](f4c79d3)) * **dot/network:** fix justification request at head logic ([#1510](#1510)) ([98d1413](98d1413)) * **dot/network:** fix memory allocations with `sizedBufferPool` ([#1963](#1963)) ([e0b126b](e0b126b)) * **dot/network:** Fix missing digest in header ([#2092](#2092)) ([21ea85e](21ea85e)) * **dot/network:** Fix notification handshake and reuse stream. ([#1545](#1545)) ([a632dc4](a632dc4)) * **dot/network:** fix receiving notifications messages from substrate peers ([#1517](#1517)) ([fdf3c53](fdf3c53)) * **dot/network:** fix stream manager tests ([#1683](#1683)) ([e02eca4](e02eca4)) * **dot/network:** implement a handshake timeout ([#1615](#1615)) ([87c2f63](87c2f63)) * **dot/network:** Implement time based handle transaction ([#1942](#1942)) ([dd08424](dd08424)) * **dot/network:** move low reputation peer removal from network ConnManager to peer scoring logic (dot/peerstate) ([#2068](#2068)) ([ac16285](ac16285)), closes [#2039](#2039) * **dot/network:** Return on EOF error while reading stream. ([#1733](#1733)) ([f447eac](f447eac)) * **dot/network:** split stored streams and handshakeData into inbound and outbound ([#1553](#1553)) ([637050b](637050b)) * **dot/network:** update notificationsProtocol handshakeData to sync.Map ([#1492](#1492)) ([22f7269](22f7269)) * **dot/node:** Start websocket server only with `--ws` flag ([#1671](#1671)) ([6ecef3b](6ecef3b)) * **dot/state, lib/babe, lib/trie:** improve syncing between gossamer authority nodes ([#1613](#1613)) ([ca99fbf](ca99fbf)) * **dot/state, lib/grandpa:** update justification and SignedVote handling in database ([#1682](#1682)) ([bbdcd6f](bbdcd6f)) * **dot/state,dot/network:** improve memory usage when syncing ([#1491](#1491)) ([3b2ad8d](3b2ad8d)) * **dot/state:** add StorageState Lock/Unlock API for usage by babe and sync ([#1700](#1700)) ([3c22ace](3c22ace)) * **dot/state:** fix deadlock, fixes bootstrap syncing ([#1959](#1959)) ([dd80c09](dd80c09)) * **dot/state:** fix usage of trie.Snapshot ([#1489](#1489)) ([3880a40](3880a40)) * **dot/state:** track runtime per-block, fix runtime upgrades differing between forks ([#1638](#1638)) ([e133884](e133884)) * **dot/state:** update `*state.BlockState.AddBlockToBlockTree` to store block in `unfinalisedBlocksMap` ([#2006](#2006)) ([55d997f](55d997f)) * **dot/sync:** add nil header checks ([#2099](#2099)) ([a7d4be0](a7d4be0)) * **dot/sync:** fix block request and response logic ([#1907](#1907)) ([9c6283e](9c6283e)) * **dot/sync:** fix creating block response, fixes node sync between gossamer nodes ([#1572](#1572)) ([1328c80](1328c80)) * **dot/telemetry:** refactor telemetry to reduce CPU usage ([#1597](#1597)) ([bc31ac7](bc31ac7)) * **dot/types:** *types.Body to be of type []types.Extrinsic ([#1807](#1807)) ([4c09715](4c09715)) * **dot/types:** fix max value for digest ([#1687](#1687)) ([48405e7](48405e7)) * **dot:** fix `TestNewNode` ([#2070](#2070)) ([42908d0](42908d0)) * fix edit link ([#1507](#1507)) ([5089327](5089327)) * fix Kusama sync; add storageState lock in core.HandleTransactionMessage ([#1783](#1783)) ([1d688e4](1d688e4)) * **lib/babe, lib/runtime/wasmer:** fixes for v0.9.8+ runtime ([#2075](#2075)) ([2f9f80c](2f9f80c)) * **lib/babe:** add `--babe-lead` flag, update epoch handling logic ([#1895](#1895)) ([7abcce6](7abcce6)) * **lib/babe:** add pre-runtime digest before calling initialize_block ([#1581](#1581)) ([c1b26d3](c1b26d3)) * **lib/babe:** always use 2/3 of slot to produce block, re-add potentially valid txs to queue ([#1679](#1679)) ([cf93ad3](cf93ad3)) * **lib/babe:** call AddBlock in BABE synchronously ([#1585](#1585)) ([86acc43](86acc43)) * **lib/babe:** fix BABE state storing after building block ([#1536](#1536)) ([1a3dea2](1a3dea2)) * **lib/babe:** fix err log ([#1801](#1801)) ([a96f06a](a96f06a)) * **lib/babe:** fix setting first slot of network, fix loading BABE epoch params ([#1640](#1640)) ([5c3dbfe](5c3dbfe)) * **lib/babe:** fix timing for transition between epochs ([#1636](#1636)) ([57027db](57027db)) * **lib/blocktree:** fix blocktree bug ([#2060](#2060)) ([c17b53a](c17b53a)) * **lib/blocktree:** fix potential nil pointer dereference in `HighestCommonAncestor`, core `handleBlocksAsync` ([#1993](#1993)) ([f7f4463](f7f4463)) * **lib/blocktree:** fix setting leaves after blocktree pruning ([#1605](#1605)) ([58c0854](58c0854)) * **lib/blocktree:** removes the inconsistency to choose a deepest leaf ([#2094](#2094)) ([43d68e3](43d68e3)) * **lib/crypto/ed25519:** update ed25519 to use go-schnorrkel bip39 derivation ([#1488](#1488)) ([dfb95d2](dfb95d2)) * **lib/genesis:** Update missing and incorrect fields in genesis file. ([#1681](#1681)) ([8207704](8207704)) * **lib/grandpa:** fix grandpa stall and various bugs ([#1708](#1708)) ([67c93f4](67c93f4)) * **lib/grandpa:** fix grandpa vote message switch ([#2095](#2095)) ([461890c](461890c)) * **lib/grandpa:** fix threshold checking to be strictly greater than 2/3 ([#1891](#1891)) ([66ffe51](66ffe51)) * **lib/grandpa:** use `defaultGrandpaInterval` if not set, fixes error on startup ([#1982](#1982)) ([75627b5](75627b5)) * **lib/runtime/life:** remove import C from life ([#1923](#1923)) ([ed507d2](ed507d2)) * **lib/runtime:** update HOST_API_TEST_RUNTIME_URL to reference specific commit ([#1885](#1885)) ([666ed06](666ed06)) * **log-levels:** do not ignore configuration file log levels ([#2016](#2016)) ([80879b2](80879b2)) * pending bubble hidden after block included ([#1592](#1592)) ([5826322](5826322)) * persist node name ([#1543](#1543)) ([88b88f2](88b88f2)) * **pprof:** only run pprof service if enabled ([#2073](#2073)) ([55669c5](55669c5)) * **release:** Trigger release when pushed to main branch. ([#1566](#1566)) ([d445c97](d445c97)) * **rpc/subscription:** subscribe runtime version notify when version changes ([#1686](#1686)) ([9a76d39](9a76d39)) * Staging CI workflow ([#2034](#2034)) ([84ec792](84ec792)) * **trie:** memory leak fix in `lib/trie` ([#2009](#2009)) ([0ad5eb7](0ad5eb7)) * update deprecated package ([#1603](#1603)) ([f195204](f195204)) * update go-schnorrkel version ([#1557](#1557)) ([b86c7ff](b86c7ff)) * update gssmr genesis to use pallet_babe::SameAuthoritiesForever ([#1696](#1696)) ([fb0a751](fb0a751)) * update HOST_API_TEST_RUNTIME_URL ([#1898](#1898)) ([2ef59a8](2ef59a8)) * **utils:** create a specific folder for database ([#1598](#1598)) ([8c67795](8c67795)) ### Features * add --chain dev option ([#1561](#1561)) ([04a2969](04a2969)) * Add properties and chainId on build-spec command ([#1520](#1520)) ([b18290c](b18290c)) * cmd: implement import-runtime subcommand ([#1483](#1483)) ([d82b2da](d82b2da)) * **cmd/gossamer:** implement --telemetry-url parameter ([#1890](#1890)) ([b202e89](b202e89)), closes [#1502](#1502) * **cmd:** implement offline pruning of state trie ([#1564](#1564)) ([af9c925](af9c925)) * **devnet:** Local Gossamer Devnet ([#2008](#2008)) ([a520001](a520001)) * **dot/network, lib/grandpa:** request justification on receiving NeighbourMessage, verify justification on receipt ([#1529](#1529)) ([e1f9f42](e1f9f42)) * **dot/network:** Add cache for network message. ([#1511](#1511)) ([accaf69](accaf69)) * **dot/network:** add propagate return bool to messageHandler func type to determine whether to propagate message or not ([#1555](#1555)) ([0d6f488](0d6f488)) * **dot/network:** implement persistent peers functionality ([#1512](#1512)) ([7850532](7850532)) * **dot/network:** implement streamManager to cleanup not recently used streams ([#1611](#1611)) ([ba861bf](ba861bf)) * **dot/network:** request block justifications when near head ([#1499](#1499)) ([ae7012b](ae7012b)) * **dot/peerset:** Implement peer scoring ([#1791](#1791)) ([1c989ad](1c989ad)) * **dot/rpc/modules:** add `system_addReservedPeer` and `system_removeReservedPeer` RPC call ([#1712](#1712)) ([dba5922](dba5922)) * **dot/rpc:** Add `system_localListenAddresses` RPC call ([#1689](#1689)) ([c981d2e](c981d2e)) * **dot/rpc:** Implement `childstate_getKeys` rpc call ([#1800](#1800)) ([9b2f41e](9b2f41e)) * **dot/rpc:** implement sync_state_genSyncSpec RPC call ([#1827](#1827)) ([2186caf](2186caf)) * **dot/state:** implement online pruning of historical state tries ([#1596](#1596)) ([3eb9399](3eb9399)) * **dot/sync:** implement codeSubstitutes ([#1635](#1635)) ([d87aaeb](d87aaeb)) * dot/telemetry: Implement basic telemetry connection ([#1497](#1497)) ([fcb4159](fcb4159)) * **dot/telemetry:** Added connection retry ([#1904](#1904)) ([579a791](579a791)) * **dot/telemetry:** Added more telemetry messages in grandpa client ([#2043](#2043)) ([2e57d15](2e57d15)), closes [#1841](#1841) [#1842](#1842) * **dot/telemetry:** implement notify.finalized telemetry interface ([#1877](#1877)) ([de1a60d](de1a60d)) * **dot/telemetry:** implement substrate_number_leaves metrics ([#1926](#1926)) ([69823c0](69823c0)) * **dot/telemetry:** implement telemetry message network_state ([#1618](#1618)) ([a81844e](a81844e)) * **flags:** read log levels from flags ([#1953](#1953)) ([9694e46](9694e46)) * implement ext_default_child_storage_storage_kill_version_2 ([#1799](#1799)) ([c2908ae](c2908ae)) * implement ext_offchain_index_set_version_1 for wasmer runtime ([#1739](#1739)) ([96c30a6](96c30a6)) * **lib/babe:** add check of types.ConfigData.SecondarySlots for disabling secondary verification ([#1910](#1910)) ([cd27ae4](cd27ae4)) * **lib/grandpa:** fully verify justifications using GrandpaState ([#1544](#1544)) ([028d25e](028d25e)) * **lib/grandpa:** Include equivocatory nodes while creating justification ([#1911](#1911)) ([aca86b6](aca86b6)) * **lib/grandpa:** send NeighbourMessage to peers ([#1558](#1558)) ([322ccf9](322ccf9)) * **lib/runtime/wasmer:** implement ext_default_child_storage_storage_kill_version_3 ([#1878](#1878)) ([a719a60](a719a60)) * **lib/runtime/wasmer:** implement ext_offchain_local_storage_version_1 ([#1821](#1821)) ([0f63b17](0f63b17)) * **lib/runtime:** Implement `ext_offchain_http_request_add_header_version_1` host function ([#1994](#1994)) ([0a30b3d](0a30b3d)) * **lib/runtime:** Implement `ext_offchain_http_request_start_version_1` host function ([#1947](#1947)) ([974b1fc](974b1fc)) * **lib/runtime:** Implement `trie_blake2_256_verify_proof` host function ([#1920](#1920)) ([506565d](506565d)) * **lib/trie:** Implement `verify_proof` function ([#1883](#1883)) ([67bb5ef](67bb5ef)) * **lib/trie:** Implement limit for trie.ClearPrefix ([#1905](#1905)) ([becec9e](becec9e)) * **lib/trie:** Parallel hash trie. ([#1657](#1657)) ([22827e7](22827e7)) * **pprof:** Pprof HTTP server service ([#1991](#1991)) ([ce24ea9](ce24ea9)) * **rpc/subscription:** implement state_unsubscribeStorage ([#1574](#1574)) ([7574f10](7574f10)) * **rpc:** Implement `childstate_getChildStorage` RPC call ([#1832](#1832)) ([3d949f2](3d949f2)) * **rpc:** Implement `childstate_getStorageHash` RPC call ([#1805](#1805)) ([e539bd3](e539bd3)) * **rpc:** Implement `childstate_getStorageSize` RPC call ([#1810](#1810)) ([a04deb6](a04deb6)) * **rpc:** Implement `payment_queryInfo` RPC call ([#1826](#1826)) ([7a5deec](7a5deec)) * **rpc:** Implement `state_getReadProof` rpc call ([#1768](#1768)) ([865f80f](865f80f)) * **runtime:** implement custom logging handler that print function name ([#1825](#1825)) ([2b1276d](2b1276d)) * **telemetry:** send telemetry messages when GRANDPA receieves commit or vote messages ([#2015](#2015)) ([7bf40e1](7bf40e1)), closes [#1840](#1840) [#1839](#1839) [#1838](#1838) * **telemetry:** send txpool.import telemetry msg ([#1966](#1966)) ([ffc81bf](ffc81bf)) ### Reverts * Revert "feat(dot/rpc) implement `author_hasSessionKeys` RPC call (#1704)" (#1714) ([65380fd](65380fd)), closes [#1704](#1704) [#1714](#1714)

github-actions · 2021-12-03T21:09:47Z

🎉 This PR is included in version 0.6.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

# [0.5.0](v0.4.1...v0.5.0) (2021-12-06) ### Bug Fixes * **babe:** Fix extrinsic format in block. ([ChainSafe#1530](https://github.com/timwu20/gossamer/issues/1530)) ([1a03b2a](1a03b2a)) * **ci:** add missing go-version matrix to fix development branch CI ([ChainSafe#2037](https://github.com/timwu20/gossamer/issues/2037)) ([6babe76](6babe76)) * **cmd/cfg:** Use Babe Lead value from toml config ([ChainSafe#2032](https://github.com/timwu20/gossamer/issues/2032)) ([06aa3e3](06aa3e3)) * confirm block import notifier is closed properly ([ChainSafe#1736](https://github.com/timwu20/gossamer/issues/1736)) ([ad2d85e](ad2d85e)) * **docs:** improve build-spec usage docs ([ChainSafe#1706](https://github.com/timwu20/gossamer/issues/1706)) ([2e164b4](2e164b4)) * **dot/core:** Add only extrinsic during chain reorg. ([ChainSafe#1609](https://github.com/timwu20/gossamer/issues/1609)) ([29413d4](29413d4)) * **dot/core:** Batch process transaction message. ([ChainSafe#1780](https://github.com/timwu20/gossamer/issues/1780)) ([0064836](0064836)) * **dot/core:** check transaction Validity.Propagate field to determine whether to propagate tx ([ChainSafe#1643](https://github.com/timwu20/gossamer/issues/1643)) ([81f23cc](81f23cc)) * **dot/core:** Fix handle transaction message test. ([ChainSafe#1607](https://github.com/timwu20/gossamer/issues/1607)) ([58b8725](58b8725)) * **dot/network, lib/grandpa:** fix handshake decoding and grandpa message handler sigabort ([ChainSafe#1631](https://github.com/timwu20/gossamer/issues/1631)) ([887f72c](887f72c)) * **dot/network, lib/grandpa:** fix node sync, improve devnet finality ([bcc7935](bcc7935)) * **dot/network:** add nil checks in connManager ([ChainSafe#2069](https://github.com/timwu20/gossamer/issues/2069)) ([7f9c042](7f9c042)) * **dot/network:** Check for size when decoding leb128. ([ChainSafe#1634](https://github.com/timwu20/gossamer/issues/1634)) ([d082b9e](d082b9e)) * **dot/network:** check if peer supports protocol ([ChainSafe#1617](https://github.com/timwu20/gossamer/issues/1617)) ([6bf66a4](6bf66a4)) * **dot/network:** decrease DHT find peers interval for gssmr nodes ([ChainSafe#1703](https://github.com/timwu20/gossamer/issues/1703)) ([08516a0](08516a0)) * **dot/network:** fix bugs in notifications protocol handlers; add metrics for inbound/outbound streams ([ChainSafe#2010](https://github.com/timwu20/gossamer/issues/2010)) ([8c2993d](8c2993d)) * **dot/network:** fix dht connection on discovery on devnet ([ChainSafe#2059](https://github.com/timwu20/gossamer/issues/2059)) ([da065b8](da065b8)) * **dot/network:** fix discovery between gossamer nodes ([ChainSafe#1594](https://github.com/timwu20/gossamer/issues/1594)) ([f4c79d3](f4c79d3)) * **dot/network:** fix memory allocations with `sizedBufferPool` ([ChainSafe#1963](https://github.com/timwu20/gossamer/issues/1963)) ([e0b126b](e0b126b)) * **dot/network:** Fix missing digest in header ([ChainSafe#2092](https://github.com/timwu20/gossamer/issues/2092)) ([21ea85e](21ea85e)) * **dot/network:** Fix notification handshake and reuse stream. ([ChainSafe#1545](https://github.com/timwu20/gossamer/issues/1545)) ([a632dc4](a632dc4)) * **dot/network:** fix stream manager tests ([ChainSafe#1683](https://github.com/timwu20/gossamer/issues/1683)) ([e02eca4](e02eca4)) * **dot/network:** implement a handshake timeout ([ChainSafe#1615](https://github.com/timwu20/gossamer/issues/1615)) ([87c2f63](87c2f63)) * **dot/network:** Implement time based handle transaction ([ChainSafe#1942](https://github.com/timwu20/gossamer/issues/1942)) ([dd08424](dd08424)) * **dot/network:** move low reputation peer removal from network ConnManager to peer scoring logic (dot/peerstate) ([ChainSafe#2068](https://github.com/timwu20/gossamer/issues/2068)) ([ac16285](ac16285)), closes [ChainSafe#2039](https://github.com/timwu20/gossamer/issues/2039) * **dot/network:** Return on EOF error while reading stream. ([ChainSafe#1733](https://github.com/timwu20/gossamer/issues/1733)) ([f447eac](f447eac)) * **dot/network:** split stored streams and handshakeData into inbound and outbound ([ChainSafe#1553](https://github.com/timwu20/gossamer/issues/1553)) ([637050b](637050b)) * **dot/node:** Start websocket server only with `--ws` flag ([ChainSafe#1671](https://github.com/timwu20/gossamer/issues/1671)) ([6ecef3b](6ecef3b)) * **dot/state, lib/babe, lib/trie:** improve syncing between gossamer authority nodes ([ChainSafe#1613](https://github.com/timwu20/gossamer/issues/1613)) ([ca99fbf](ca99fbf)) * **dot/state, lib/grandpa:** update justification and SignedVote handling in database ([ChainSafe#1682](https://github.com/timwu20/gossamer/issues/1682)) ([bbdcd6f](bbdcd6f)) * **dot/state:** add StorageState Lock/Unlock API for usage by babe and sync ([ChainSafe#1700](https://github.com/timwu20/gossamer/issues/1700)) ([3c22ace](3c22ace)) * **dot/state:** fix deadlock, fixes bootstrap syncing ([ChainSafe#1959](https://github.com/timwu20/gossamer/issues/1959)) ([dd80c09](dd80c09)) * **dot/state:** track runtime per-block, fix runtime upgrades differing between forks ([ChainSafe#1638](https://github.com/timwu20/gossamer/issues/1638)) ([e133884](e133884)) * **dot/state:** update `*state.BlockState.AddBlockToBlockTree` to store block in `unfinalisedBlocksMap` ([ChainSafe#2006](https://github.com/timwu20/gossamer/issues/2006)) ([55d997f](55d997f)) * **dot/sync:** add nil header checks ([ChainSafe#2099](https://github.com/timwu20/gossamer/issues/2099)) ([a7d4be0](a7d4be0)) * **dot/sync:** fix block request and response logic ([ChainSafe#1907](https://github.com/timwu20/gossamer/issues/1907)) ([9c6283e](9c6283e)) * **dot/sync:** fix creating block response, fixes node sync between gossamer nodes ([ChainSafe#1572](https://github.com/timwu20/gossamer/issues/1572)) ([1328c80](1328c80)) * **dot/telemetry:** refactor telemetry to reduce CPU usage ([ChainSafe#1597](https://github.com/timwu20/gossamer/issues/1597)) ([bc31ac7](bc31ac7)) * **dot/types:** *types.Body to be of type []types.Extrinsic ([ChainSafe#1807](https://github.com/timwu20/gossamer/issues/1807)) ([4c09715](4c09715)) * **dot/types:** fix max value for digest ([ChainSafe#1687](https://github.com/timwu20/gossamer/issues/1687)) ([48405e7](48405e7)) * **dot:** fix `TestNewNode` ([ChainSafe#2070](https://github.com/timwu20/gossamer/issues/2070)) ([42908d0](42908d0)) * fix Kusama sync; add storageState lock in core.HandleTransactionMessage ([ChainSafe#1783](https://github.com/timwu20/gossamer/issues/1783)) ([1d688e4](1d688e4)) * **lib/babe, lib/runtime/wasmer:** fixes for v0.9.8+ runtime ([ChainSafe#2075](https://github.com/timwu20/gossamer/issues/2075)) ([2f9f80c](2f9f80c)) * **lib/babe:** add `--babe-lead` flag, update epoch handling logic ([ChainSafe#1895](https://github.com/timwu20/gossamer/issues/1895)) ([7abcce6](7abcce6)) * **lib/babe:** add pre-runtime digest before calling initialize_block ([ChainSafe#1581](https://github.com/timwu20/gossamer/issues/1581)) ([c1b26d3](c1b26d3)) * **lib/babe:** always use 2/3 of slot to produce block, re-add potentially valid txs to queue ([ChainSafe#1679](https://github.com/timwu20/gossamer/issues/1679)) ([cf93ad3](cf93ad3)) * **lib/babe:** call AddBlock in BABE synchronously ([ChainSafe#1585](https://github.com/timwu20/gossamer/issues/1585)) ([86acc43](86acc43)) * **lib/babe:** fix BABE state storing after building block ([ChainSafe#1536](https://github.com/timwu20/gossamer/issues/1536)) ([1a3dea2](1a3dea2)) * **lib/babe:** fix err log ([ChainSafe#1801](https://github.com/timwu20/gossamer/issues/1801)) ([a96f06a](a96f06a)) * **lib/babe:** fix setting first slot of network, fix loading BABE epoch params ([ChainSafe#1640](https://github.com/timwu20/gossamer/issues/1640)) ([5c3dbfe](5c3dbfe)) * **lib/babe:** fix timing for transition between epochs ([ChainSafe#1636](https://github.com/timwu20/gossamer/issues/1636)) ([57027db](57027db)) * **lib/blocktree:** fix blocktree bug ([ChainSafe#2060](https://github.com/timwu20/gossamer/issues/2060)) ([c17b53a](c17b53a)) * **lib/blocktree:** fix potential nil pointer dereference in `HighestCommonAncestor`, core `handleBlocksAsync` ([ChainSafe#1993](https://github.com/timwu20/gossamer/issues/1993)) ([f7f4463](f7f4463)) * **lib/blocktree:** fix setting leaves after blocktree pruning ([ChainSafe#1605](https://github.com/timwu20/gossamer/issues/1605)) ([58c0854](58c0854)) * **lib/blocktree:** removes the inconsistency to choose a deepest leaf ([ChainSafe#2094](https://github.com/timwu20/gossamer/issues/2094)) ([43d68e3](43d68e3)) * **lib/genesis:** Update missing and incorrect fields in genesis file. ([ChainSafe#1681](https://github.com/timwu20/gossamer/issues/1681)) ([8207704](8207704)) * **lib/grandpa:** fix grandpa stall and various bugs ([ChainSafe#1708](https://github.com/timwu20/gossamer/issues/1708)) ([67c93f4](67c93f4)) * **lib/grandpa:** fix grandpa vote message switch ([ChainSafe#2095](https://github.com/timwu20/gossamer/issues/2095)) ([461890c](461890c)) * **lib/grandpa:** fix threshold checking to be strictly greater than 2/3 ([ChainSafe#1891](https://github.com/timwu20/gossamer/issues/1891)) ([66ffe51](66ffe51)) * **lib/grandpa:** use `defaultGrandpaInterval` if not set, fixes error on startup ([ChainSafe#1982](https://github.com/timwu20/gossamer/issues/1982)) ([75627b5](75627b5)) * **lib/runtime/life:** remove import C from life ([ChainSafe#1923](https://github.com/timwu20/gossamer/issues/1923)) ([ed507d2](ed507d2)) * **lib/runtime:** update HOST_API_TEST_RUNTIME_URL to reference specific commit ([ChainSafe#1885](https://github.com/timwu20/gossamer/issues/1885)) ([666ed06](666ed06)) * **log-levels:** do not ignore configuration file log levels ([ChainSafe#2016](https://github.com/timwu20/gossamer/issues/2016)) ([80879b2](80879b2)) * pending bubble hidden after block included ([ChainSafe#1592](https://github.com/timwu20/gossamer/issues/1592)) ([5826322](5826322)) * persist node name ([ChainSafe#1543](https://github.com/timwu20/gossamer/issues/1543)) ([88b88f2](88b88f2)) * **pprof:** only run pprof service if enabled ([ChainSafe#2073](https://github.com/timwu20/gossamer/issues/2073)) ([55669c5](55669c5)) * **release:** Trigger release when pushed to main branch. ([ChainSafe#1566](https://github.com/timwu20/gossamer/issues/1566)) ([d445c97](d445c97)) * **rpc/subscription:** subscribe runtime version notify when version changes ([ChainSafe#1686](https://github.com/timwu20/gossamer/issues/1686)) ([9a76d39](9a76d39)) * Staging CI workflow ([ChainSafe#2034](https://github.com/timwu20/gossamer/issues/2034)) ([84ec792](84ec792)) * **trie:** memory leak fix in `lib/trie` ([ChainSafe#2009](https://github.com/timwu20/gossamer/issues/2009)) ([0ad5eb7](0ad5eb7)) * update deprecated package ([ChainSafe#1603](https://github.com/timwu20/gossamer/issues/1603)) ([f195204](f195204)) * update go-schnorrkel version ([ChainSafe#1557](https://github.com/timwu20/gossamer/issues/1557)) ([b86c7ff](b86c7ff)) * update gssmr genesis to use pallet_babe::SameAuthoritiesForever ([ChainSafe#1696](https://github.com/timwu20/gossamer/issues/1696)) ([fb0a751](fb0a751)) * update HOST_API_TEST_RUNTIME_URL ([ChainSafe#1898](https://github.com/timwu20/gossamer/issues/1898)) ([2ef59a8](2ef59a8)) * **utils:** create a specific folder for database ([ChainSafe#1598](https://github.com/timwu20/gossamer/issues/1598)) ([8c67795](8c67795)) ### Features * add --chain dev option ([ChainSafe#1561](https://github.com/timwu20/gossamer/issues/1561)) ([04a2969](04a2969)) * Add properties and chainId on build-spec command ([ChainSafe#1520](https://github.com/timwu20/gossamer/issues/1520)) ([b18290c](b18290c)) * **cmd/gossamer:** implement --telemetry-url parameter ([ChainSafe#1890](https://github.com/timwu20/gossamer/issues/1890)) ([b202e89](b202e89)), closes [ChainSafe#1502](https://github.com/timwu20/gossamer/issues/1502) * **cmd:** implement offline pruning of state trie ([ChainSafe#1564](https://github.com/timwu20/gossamer/issues/1564)) ([af9c925](af9c925)) * **devnet:** Local Gossamer Devnet ([ChainSafe#2008](https://github.com/timwu20/gossamer/issues/2008)) ([a520001](a520001)) * **dot/network, lib/grandpa:** request justification on receiving NeighbourMessage, verify justification on receipt ([ChainSafe#1529](https://github.com/timwu20/gossamer/issues/1529)) ([e1f9f42](e1f9f42)) * **dot/network:** add propagate return bool to messageHandler func type to determine whether to propagate message or not ([ChainSafe#1555](https://github.com/timwu20/gossamer/issues/1555)) ([0d6f488](0d6f488)) * **dot/network:** implement streamManager to cleanup not recently used streams ([ChainSafe#1611](https://github.com/timwu20/gossamer/issues/1611)) ([ba861bf](ba861bf)) * **dot/peerset:** Implement peer scoring ([ChainSafe#1791](https://github.com/timwu20/gossamer/issues/1791)) ([1c989ad](1c989ad)) * **dot/rpc/modules:** add `system_addReservedPeer` and `system_removeReservedPeer` RPC call ([ChainSafe#1712](https://github.com/timwu20/gossamer/issues/1712)) ([dba5922](dba5922)) * **dot/rpc:** Add `system_localListenAddresses` RPC call ([ChainSafe#1689](https://github.com/timwu20/gossamer/issues/1689)) ([c981d2e](c981d2e)) * **dot/rpc:** Implement `childstate_getKeys` rpc call ([ChainSafe#1800](https://github.com/timwu20/gossamer/issues/1800)) ([9b2f41e](9b2f41e)) * **dot/rpc:** implement sync_state_genSyncSpec RPC call ([ChainSafe#1827](https://github.com/timwu20/gossamer/issues/1827)) ([2186caf](2186caf)) * **dot/state:** implement online pruning of historical state tries ([ChainSafe#1596](https://github.com/timwu20/gossamer/issues/1596)) ([3eb9399](3eb9399)) * **dot/sync:** implement codeSubstitutes ([ChainSafe#1635](https://github.com/timwu20/gossamer/issues/1635)) ([d87aaeb](d87aaeb)) * **dot/telemetry:** Added connection retry ([ChainSafe#1904](https://github.com/timwu20/gossamer/issues/1904)) ([579a791](579a791)) * **dot/telemetry:** Added more telemetry messages in grandpa client ([ChainSafe#2043](https://github.com/timwu20/gossamer/issues/2043)) ([2e57d15](2e57d15)), closes [ChainSafe#1841](https://github.com/timwu20/gossamer/issues/1841) [ChainSafe#1842](https://github.com/timwu20/gossamer/issues/1842) * **dot/telemetry:** implement notify.finalized telemetry interface ([ChainSafe#1877](https://github.com/timwu20/gossamer/issues/1877)) ([de1a60d](de1a60d)) * **dot/telemetry:** implement substrate_number_leaves metrics ([ChainSafe#1926](https://github.com/timwu20/gossamer/issues/1926)) ([69823c0](69823c0)) * **dot/telemetry:** implement telemetry message network_state ([ChainSafe#1618](https://github.com/timwu20/gossamer/issues/1618)) ([a81844e](a81844e)) * **flags:** read log levels from flags ([ChainSafe#1953](https://github.com/timwu20/gossamer/issues/1953)) ([9694e46](9694e46)) * implement ext_default_child_storage_storage_kill_version_2 ([ChainSafe#1799](https://github.com/timwu20/gossamer/issues/1799)) ([c2908ae](c2908ae)) * implement ext_offchain_index_set_version_1 for wasmer runtime ([ChainSafe#1739](https://github.com/timwu20/gossamer/issues/1739)) ([96c30a6](96c30a6)) * **lib/babe:** add check of types.ConfigData.SecondarySlots for disabling secondary verification ([ChainSafe#1910](https://github.com/timwu20/gossamer/issues/1910)) ([cd27ae4](cd27ae4)) * **lib/grandpa:** fully verify justifications using GrandpaState ([ChainSafe#1544](https://github.com/timwu20/gossamer/issues/1544)) ([028d25e](028d25e)) * **lib/grandpa:** Include equivocatory nodes while creating justification ([ChainSafe#1911](https://github.com/timwu20/gossamer/issues/1911)) ([aca86b6](aca86b6)) * **lib/grandpa:** send NeighbourMessage to peers ([ChainSafe#1558](https://github.com/timwu20/gossamer/issues/1558)) ([322ccf9](322ccf9)) * **lib/runtime/wasmer:** implement ext_default_child_storage_storage_kill_version_3 ([ChainSafe#1878](https://github.com/timwu20/gossamer/issues/1878)) ([a719a60](a719a60)) * **lib/runtime/wasmer:** implement ext_offchain_local_storage_version_1 ([ChainSafe#1821](https://github.com/timwu20/gossamer/issues/1821)) ([0f63b17](0f63b17)) * **lib/runtime:** Implement `ext_offchain_http_request_add_header_version_1` host function ([ChainSafe#1994](https://github.com/timwu20/gossamer/issues/1994)) ([0a30b3d](0a30b3d)) * **lib/runtime:** Implement `ext_offchain_http_request_start_version_1` host function ([ChainSafe#1947](https://github.com/timwu20/gossamer/issues/1947)) ([974b1fc](974b1fc)) * **lib/runtime:** Implement `trie_blake2_256_verify_proof` host function ([ChainSafe#1920](https://github.com/timwu20/gossamer/issues/1920)) ([506565d](506565d)) * **lib/trie:** Implement `verify_proof` function ([ChainSafe#1883](https://github.com/timwu20/gossamer/issues/1883)) ([67bb5ef](67bb5ef)) * **lib/trie:** Implement limit for trie.ClearPrefix ([ChainSafe#1905](https://github.com/timwu20/gossamer/issues/1905)) ([becec9e](becec9e)) * **lib/trie:** Parallel hash trie. ([ChainSafe#1657](https://github.com/timwu20/gossamer/issues/1657)) ([22827e7](22827e7)) * **pprof:** Pprof HTTP server service ([ChainSafe#1991](https://github.com/timwu20/gossamer/issues/1991)) ([ce24ea9](ce24ea9)) * **rpc/subscription:** implement state_unsubscribeStorage ([ChainSafe#1574](https://github.com/timwu20/gossamer/issues/1574)) ([7574f10](7574f10)) * **rpc:** Implement `childstate_getChildStorage` RPC call ([ChainSafe#1832](https://github.com/timwu20/gossamer/issues/1832)) ([3d949f2](3d949f2)) * **rpc:** Implement `childstate_getStorageHash` RPC call ([ChainSafe#1805](https://github.com/timwu20/gossamer/issues/1805)) ([e539bd3](e539bd3)) * **rpc:** Implement `childstate_getStorageSize` RPC call ([ChainSafe#1810](https://github.com/timwu20/gossamer/issues/1810)) ([a04deb6](a04deb6)) * **rpc:** Implement `payment_queryInfo` RPC call ([ChainSafe#1826](https://github.com/timwu20/gossamer/issues/1826)) ([7a5deec](7a5deec)) * **rpc:** Implement `state_getReadProof` rpc call ([ChainSafe#1768](https://github.com/timwu20/gossamer/issues/1768)) ([865f80f](865f80f)) * **runtime:** implement custom logging handler that print function name ([ChainSafe#1825](https://github.com/timwu20/gossamer/issues/1825)) ([2b1276d](2b1276d)) * **telemetry:** send telemetry messages when GRANDPA receieves commit or vote messages ([ChainSafe#2015](https://github.com/timwu20/gossamer/issues/2015)) ([7bf40e1](7bf40e1)), closes [ChainSafe#1840](https://github.com/timwu20/gossamer/issues/1840) [ChainSafe#1839](https://github.com/timwu20/gossamer/issues/1839) [ChainSafe#1838](https://github.com/timwu20/gossamer/issues/1838) * **telemetry:** send txpool.import telemetry msg ([ChainSafe#1966](https://github.com/timwu20/gossamer/issues/1966)) ([ffc81bf](ffc81bf)) ### Reverts * Revert "feat(dot/rpc) implement `author_hasSessionKeys` RPC call (ChainSafe#1704)" (ChainSafe#1714) ([65380fd](65380fd)), closes [ChainSafe#1704](https://github.com/timwu20/gossamer/issues/1704) [ChainSafe#1714](https://github.com/timwu20/gossamer/issues/1714)

* add TODOs to identify where block imported channel is handled * added comments for imported channels * create constructor for listeners * added close channel to defer in listen * move imported chan to block_notify * remove comments, lint * handle lint issues * replace imported channel map with sync.Map * fix mocks in listeners test * fix mock functions for new imported notification channel * fix deep source issues * add debugging printf * remove sync.Pool, and sync.Map * handle channel closing * add sleep before close * remove channel close * run go imported * defer importedLock unlock * wrap notifier channel in struct * store channel by interface{} key * update storage key for imported block listeners * refacter GetImportedBlockNotifierChannel arugments * GetImportedBlockNotifierChannel doesn't return error, fixed related test * remove un-needed comments * remove close for FinalisedChannel listener * added test for free imported channel * add mocks paths to .deepsource exclude_patterns

# [0.6.0](v0.5.0...v0.6.0) (2021-12-06) ### Bug Fixes * **ci:** add missing go-version matrix to fix development branch CI ([ChainSafe#2037](https://github.com/timwu20/gossamer/issues/2037)) ([dfc6ce6](dfc6ce6)) * **cmd/cfg:** Use Babe Lead value from toml config ([ChainSafe#2032](https://github.com/timwu20/gossamer/issues/2032)) ([406d083](406d083)) * confirm block import notifier is closed properly ([ChainSafe#1736](https://github.com/timwu20/gossamer/issues/1736)) ([ce4b9d7](ce4b9d7)) * **docs:** improve build-spec usage docs ([ChainSafe#1706](https://github.com/timwu20/gossamer/issues/1706)) ([65b4753](65b4753)) * **dot/core:** Add only extrinsic during chain reorg. ([ChainSafe#1609](https://github.com/timwu20/gossamer/issues/1609)) ([16340ef](16340ef)) * **dot/core:** Batch process transaction message. ([ChainSafe#1780](https://github.com/timwu20/gossamer/issues/1780)) ([3d209a3](3d209a3)) * **dot/core:** check transaction Validity.Propagate field to determine whether to propagate tx ([ChainSafe#1643](https://github.com/timwu20/gossamer/issues/1643)) ([81c8f0a](81c8f0a)) * **dot/core:** Fix handle transaction message test. ([ChainSafe#1607](https://github.com/timwu20/gossamer/issues/1607)) ([dca9ad7](dca9ad7)) * **dot/network, lib/grandpa:** fix handshake decoding and grandpa message handler sigabort ([ChainSafe#1631](https://github.com/timwu20/gossamer/issues/1631)) ([b02525b](b02525b)) * **dot/network, lib/grandpa:** fix node sync, improve devnet finality ([de91262](de91262)) * **dot/network:** add nil checks in connManager ([ChainSafe#2069](https://github.com/timwu20/gossamer/issues/2069)) ([09e07cf](09e07cf)) * **dot/network:** Check for size when decoding leb128. ([ChainSafe#1634](https://github.com/timwu20/gossamer/issues/1634)) ([c20a83c](c20a83c)) * **dot/network:** check if peer supports protocol ([ChainSafe#1617](https://github.com/timwu20/gossamer/issues/1617)) ([fb26dd2](fb26dd2)) * **dot/network:** decrease DHT find peers interval for gssmr nodes ([ChainSafe#1703](https://github.com/timwu20/gossamer/issues/1703)) ([7823c57](7823c57)) * **dot/network:** fix bugs in notifications protocol handlers; add metrics for inbound/outbound streams ([ChainSafe#2010](https://github.com/timwu20/gossamer/issues/2010)) ([88174ca](88174ca)) * **dot/network:** fix dht connection on discovery on devnet ([ChainSafe#2059](https://github.com/timwu20/gossamer/issues/2059)) ([d9008aa](d9008aa)) * **dot/network:** fix discovery between gossamer nodes ([ChainSafe#1594](https://github.com/timwu20/gossamer/issues/1594)) ([ec59e3b](ec59e3b)) * **dot/network:** fix memory allocations with `sizedBufferPool` ([ChainSafe#1963](https://github.com/timwu20/gossamer/issues/1963)) ([ddd96f7](ddd96f7)) * **dot/network:** Fix missing digest in header ([ChainSafe#2092](https://github.com/timwu20/gossamer/issues/2092)) ([6f34b1b](6f34b1b)) * **dot/network:** fix stream manager tests ([ChainSafe#1683](https://github.com/timwu20/gossamer/issues/1683)) ([2b4353c](2b4353c)) * **dot/network:** implement a handshake timeout ([ChainSafe#1615](https://github.com/timwu20/gossamer/issues/1615)) ([4ab856e](4ab856e)) * **dot/network:** Implement time based handle transaction ([ChainSafe#1942](https://github.com/timwu20/gossamer/issues/1942)) ([72455d1](72455d1)) * **dot/network:** move low reputation peer removal from network ConnManager to peer scoring logic (dot/peerstate) ([ChainSafe#2068](https://github.com/timwu20/gossamer/issues/2068)) ([8ea3d9f](8ea3d9f)), closes [ChainSafe#2039](https://github.com/timwu20/gossamer/issues/2039) * **dot/network:** Return on EOF error while reading stream. ([ChainSafe#1733](https://github.com/timwu20/gossamer/issues/1733)) ([a9d32fb](a9d32fb)) * **dot/node:** Start websocket server only with `--ws` flag ([ChainSafe#1671](https://github.com/timwu20/gossamer/issues/1671)) ([fd43774](fd43774)) * **dot/state, lib/babe, lib/trie:** improve syncing between gossamer authority nodes ([ChainSafe#1613](https://github.com/timwu20/gossamer/issues/1613)) ([2e62a3d](2e62a3d)) * **dot/state, lib/grandpa:** update justification and SignedVote handling in database ([ChainSafe#1682](https://github.com/timwu20/gossamer/issues/1682)) ([9dda6b5](9dda6b5)) * **dot/state:** add StorageState Lock/Unlock API for usage by babe and sync ([ChainSafe#1700](https://github.com/timwu20/gossamer/issues/1700)) ([eaf8ba9](eaf8ba9)) * **dot/state:** fix deadlock, fixes bootstrap syncing ([ChainSafe#1959](https://github.com/timwu20/gossamer/issues/1959)) ([13c92ea](13c92ea)) * **dot/state:** track runtime per-block, fix runtime upgrades differing between forks ([ChainSafe#1638](https://github.com/timwu20/gossamer/issues/1638)) ([b31bba5](b31bba5)) * **dot/state:** update `*state.BlockState.AddBlockToBlockTree` to store block in `unfinalisedBlocksMap` ([ChainSafe#2006](https://github.com/timwu20/gossamer/issues/2006)) ([8fac690](8fac690)) * **dot/sync:** add nil header checks ([ChainSafe#2099](https://github.com/timwu20/gossamer/issues/2099)) ([abbefce](abbefce)) * **dot/sync:** fix block request and response logic ([ChainSafe#1907](https://github.com/timwu20/gossamer/issues/1907)) ([fcfbabe](fcfbabe)) * **dot/sync:** fix creating block response, fixes node sync between gossamer nodes ([ChainSafe#1572](https://github.com/timwu20/gossamer/issues/1572)) ([d2197b0](d2197b0)) * **dot/telemetry:** refactor telemetry to reduce CPU usage ([ChainSafe#1597](https://github.com/timwu20/gossamer/issues/1597)) ([3445d84](3445d84)) * **dot/types:** *types.Body to be of type []types.Extrinsic ([ChainSafe#1807](https://github.com/timwu20/gossamer/issues/1807)) ([953d6f4](953d6f4)) * **dot/types:** fix max value for digest ([ChainSafe#1687](https://github.com/timwu20/gossamer/issues/1687)) ([13380af](13380af)) * **dot:** fix `TestNewNode` ([ChainSafe#2070](https://github.com/timwu20/gossamer/issues/2070)) ([87f41ba](87f41ba)) * fix Kusama sync; add storageState lock in core.HandleTransactionMessage ([ChainSafe#1783](https://github.com/timwu20/gossamer/issues/1783)) ([d03f6d1](d03f6d1)) * **lib/babe, lib/runtime/wasmer:** fixes for v0.9.8+ runtime ([ChainSafe#2075](https://github.com/timwu20/gossamer/issues/2075)) ([eb963d3](eb963d3)) * **lib/babe:** add `--babe-lead` flag, update epoch handling logic ([ChainSafe#1895](https://github.com/timwu20/gossamer/issues/1895)) ([8a65c7f](8a65c7f)) * **lib/babe:** add pre-runtime digest before calling initialize_block ([ChainSafe#1581](https://github.com/timwu20/gossamer/issues/1581)) ([f0060ae](f0060ae)) * **lib/babe:** always use 2/3 of slot to produce block, re-add potentially valid txs to queue ([ChainSafe#1679](https://github.com/timwu20/gossamer/issues/1679)) ([b224f10](b224f10)) * **lib/babe:** call AddBlock in BABE synchronously ([ChainSafe#1585](https://github.com/timwu20/gossamer/issues/1585)) ([549dc25](549dc25)) * **lib/babe:** fix err log ([ChainSafe#1801](https://github.com/timwu20/gossamer/issues/1801)) ([0ab395b](0ab395b)) * **lib/babe:** fix setting first slot of network, fix loading BABE epoch params ([ChainSafe#1640](https://github.com/timwu20/gossamer/issues/1640)) ([63f73ca](63f73ca)) * **lib/babe:** fix timing for transition between epochs ([ChainSafe#1636](https://github.com/timwu20/gossamer/issues/1636)) ([e67b1e2](e67b1e2)) * **lib/blocktree:** fix blocktree bug ([ChainSafe#2060](https://github.com/timwu20/gossamer/issues/2060)) ([de8f573](de8f573)) * **lib/blocktree:** fix potential nil pointer dereference in `HighestCommonAncestor`, core `handleBlocksAsync` ([ChainSafe#1993](https://github.com/timwu20/gossamer/issues/1993)) ([7006320](7006320)) * **lib/blocktree:** fix setting leaves after blocktree pruning ([ChainSafe#1605](https://github.com/timwu20/gossamer/issues/1605)) ([a00f881](a00f881)) * **lib/blocktree:** removes the inconsistency to choose a deepest leaf ([ChainSafe#2094](https://github.com/timwu20/gossamer/issues/2094)) ([5210dc0](5210dc0)) * **lib/genesis:** Update missing and incorrect fields in genesis file. ([ChainSafe#1681](https://github.com/timwu20/gossamer/issues/1681)) ([f00a5a2](f00a5a2)) * **lib/grandpa:** fix grandpa stall and various bugs ([ChainSafe#1708](https://github.com/timwu20/gossamer/issues/1708)) ([fe67586](fe67586)) * **lib/grandpa:** fix grandpa vote message switch ([ChainSafe#2095](https://github.com/timwu20/gossamer/issues/2095)) ([8312490](8312490)) * **lib/grandpa:** fix threshold checking to be strictly greater than 2/3 ([ChainSafe#1891](https://github.com/timwu20/gossamer/issues/1891)) ([ed51f97](ed51f97)) * **lib/grandpa:** use `defaultGrandpaInterval` if not set, fixes error on startup ([ChainSafe#1982](https://github.com/timwu20/gossamer/issues/1982)) ([84e2005](84e2005)) * **lib/runtime/life:** remove import C from life ([ChainSafe#1923](https://github.com/timwu20/gossamer/issues/1923)) ([b93152a](b93152a)) * **lib/runtime:** update HOST_API_TEST_RUNTIME_URL to reference specific commit ([ChainSafe#1885](https://github.com/timwu20/gossamer/issues/1885)) ([e6689e8](e6689e8)) * **log-levels:** do not ignore configuration file log levels ([ChainSafe#2016](https://github.com/timwu20/gossamer/issues/2016)) ([44c9b7a](44c9b7a)) * pending bubble hidden after block included ([ChainSafe#1592](https://github.com/timwu20/gossamer/issues/1592)) ([16e7eef](16e7eef)) * **pprof:** only run pprof service if enabled ([ChainSafe#2073](https://github.com/timwu20/gossamer/issues/2073)) ([f45eff4](f45eff4)) * **rpc/subscription:** subscribe runtime version notify when version changes ([ChainSafe#1686](https://github.com/timwu20/gossamer/issues/1686)) ([d6e9ab0](d6e9ab0)) * Staging CI workflow ([ChainSafe#2034](https://github.com/timwu20/gossamer/issues/2034)) ([aaa633c](aaa633c)) * **trie:** memory leak fix in `lib/trie` ([ChainSafe#2009](https://github.com/timwu20/gossamer/issues/2009)) ([71a8a4f](71a8a4f)) * update deprecated package ([ChainSafe#1603](https://github.com/timwu20/gossamer/issues/1603)) ([ee5a5b6](ee5a5b6)) * update gssmr genesis to use pallet_babe::SameAuthoritiesForever ([ChainSafe#1696](https://github.com/timwu20/gossamer/issues/1696)) ([281eef2](281eef2)) * update HOST_API_TEST_RUNTIME_URL ([ChainSafe#1898](https://github.com/timwu20/gossamer/issues/1898)) ([afe350c](afe350c)) * **utils:** create a specific folder for database ([ChainSafe#1598](https://github.com/timwu20/gossamer/issues/1598)) ([4720d27](4720d27)) ### Features * add --chain dev option ([ChainSafe#1561](https://github.com/timwu20/gossamer/issues/1561)) ([dc8ee9d](dc8ee9d)) * **cmd/gossamer:** implement --telemetry-url parameter ([ChainSafe#1890](https://github.com/timwu20/gossamer/issues/1890)) ([154d6e2](154d6e2)), closes [ChainSafe#1502](https://github.com/timwu20/gossamer/issues/1502) * **cmd:** implement offline pruning of state trie ([ChainSafe#1564](https://github.com/timwu20/gossamer/issues/1564)) ([d4d61c7](d4d61c7)) * **devnet:** Local Gossamer Devnet ([ChainSafe#2008](https://github.com/timwu20/gossamer/issues/2008)) ([23ce39f](23ce39f)) * **dot/network:** implement streamManager to cleanup not recently used streams ([ChainSafe#1611](https://github.com/timwu20/gossamer/issues/1611)) ([005932b](005932b)) * **dot/peerset:** Implement peer scoring ([ChainSafe#1791](https://github.com/timwu20/gossamer/issues/1791)) ([05f7e32](05f7e32)) * **dot/rpc/modules:** add `system_addReservedPeer` and `system_removeReservedPeer` RPC call ([ChainSafe#1712](https://github.com/timwu20/gossamer/issues/1712)) ([79c250e](79c250e)) * **dot/rpc:** Add `system_localListenAddresses` RPC call ([ChainSafe#1689](https://github.com/timwu20/gossamer/issues/1689)) ([0d199b5](0d199b5)) * **dot/rpc:** Implement `childstate_getKeys` rpc call ([ChainSafe#1800](https://github.com/timwu20/gossamer/issues/1800)) ([c5181ae](c5181ae)) * **dot/rpc:** implement sync_state_genSyncSpec RPC call ([ChainSafe#1827](https://github.com/timwu20/gossamer/issues/1827)) ([9991a9c](9991a9c)) * **dot/state:** implement online pruning of historical state tries ([ChainSafe#1596](https://github.com/timwu20/gossamer/issues/1596)) ([31700c6](31700c6)) * **dot/sync:** implement codeSubstitutes ([ChainSafe#1635](https://github.com/timwu20/gossamer/issues/1635)) ([1a238b1](1a238b1)) * **dot/telemetry:** Added connection retry ([ChainSafe#1904](https://github.com/timwu20/gossamer/issues/1904)) ([1c0c5c7](1c0c5c7)) * **dot/telemetry:** Added more telemetry messages in grandpa client ([ChainSafe#2043](https://github.com/timwu20/gossamer/issues/2043)) ([7c870a4](7c870a4)), closes [ChainSafe#1841](https://github.com/timwu20/gossamer/issues/1841) [ChainSafe#1842](https://github.com/timwu20/gossamer/issues/1842) * **dot/telemetry:** implement notify.finalized telemetry interface ([ChainSafe#1877](https://github.com/timwu20/gossamer/issues/1877)) ([b4e5ded](b4e5ded)) * **dot/telemetry:** implement substrate_number_leaves metrics ([ChainSafe#1926](https://github.com/timwu20/gossamer/issues/1926)) ([847de6a](847de6a)) * **dot/telemetry:** implement telemetry message network_state ([ChainSafe#1618](https://github.com/timwu20/gossamer/issues/1618)) ([73e99c9](73e99c9)) * **flags:** read log levels from flags ([ChainSafe#1953](https://github.com/timwu20/gossamer/issues/1953)) ([abf9f3a](abf9f3a)) * implement ext_default_child_storage_storage_kill_version_2 ([ChainSafe#1799](https://github.com/timwu20/gossamer/issues/1799)) ([541f580](541f580)) * implement ext_offchain_index_set_version_1 for wasmer runtime ([ChainSafe#1739](https://github.com/timwu20/gossamer/issues/1739)) ([619306d](619306d)) * **lib/babe:** add check of types.ConfigData.SecondarySlots for disabling secondary verification ([ChainSafe#1910](https://github.com/timwu20/gossamer/issues/1910)) ([7bc2e32](7bc2e32)) * **lib/grandpa:** Include equivocatory nodes while creating justification ([ChainSafe#1911](https://github.com/timwu20/gossamer/issues/1911)) ([af7725f](af7725f)) * **lib/runtime/wasmer:** implement ext_default_child_storage_storage_kill_version_3 ([ChainSafe#1878](https://github.com/timwu20/gossamer/issues/1878)) ([454689b](454689b)) * **lib/runtime/wasmer:** implement ext_offchain_local_storage_version_1 ([ChainSafe#1821](https://github.com/timwu20/gossamer/issues/1821)) ([01dbffc](01dbffc)) * **lib/runtime:** Implement `ext_offchain_http_request_add_header_version_1` host function ([ChainSafe#1994](https://github.com/timwu20/gossamer/issues/1994)) ([2e722c7](2e722c7)) * **lib/runtime:** Implement `ext_offchain_http_request_start_version_1` host function ([ChainSafe#1947](https://github.com/timwu20/gossamer/issues/1947)) ([4a8598b](4a8598b)) * **lib/runtime:** Implement `trie_blake2_256_verify_proof` host function ([ChainSafe#1920](https://github.com/timwu20/gossamer/issues/1920)) ([1bd33a6](1bd33a6)) * **lib/trie:** Implement `verify_proof` function ([ChainSafe#1883](https://github.com/timwu20/gossamer/issues/1883)) ([68650b7](68650b7)) * **lib/trie:** Implement limit for trie.ClearPrefix ([ChainSafe#1905](https://github.com/timwu20/gossamer/issues/1905)) ([9756b2c](9756b2c)) * **lib/trie:** Parallel hash trie. ([ChainSafe#1657](https://github.com/timwu20/gossamer/issues/1657)) ([22f0b2b](22f0b2b)) * **pprof:** Pprof HTTP server service ([ChainSafe#1991](https://github.com/timwu20/gossamer/issues/1991)) ([5996229](5996229)) * **rpc/subscription:** implement state_unsubscribeStorage ([ChainSafe#1574](https://github.com/timwu20/gossamer/issues/1574)) ([56f594a](56f594a)) * **rpc:** Implement `childstate_getChildStorage` RPC call ([ChainSafe#1832](https://github.com/timwu20/gossamer/issues/1832)) ([4977d06](4977d06)) * **rpc:** Implement `childstate_getStorageHash` RPC call ([ChainSafe#1805](https://github.com/timwu20/gossamer/issues/1805)) ([0e31754](0e31754)) * **rpc:** Implement `childstate_getStorageSize` RPC call ([ChainSafe#1810](https://github.com/timwu20/gossamer/issues/1810)) ([77f3458](77f3458)) * **rpc:** Implement `payment_queryInfo` RPC call ([ChainSafe#1826](https://github.com/timwu20/gossamer/issues/1826)) ([d6e3583](d6e3583)) * **rpc:** Implement `state_getReadProof` rpc call ([ChainSafe#1768](https://github.com/timwu20/gossamer/issues/1768)) ([00c4951](00c4951)) * **runtime:** implement custom logging handler that print function name ([ChainSafe#1825](https://github.com/timwu20/gossamer/issues/1825)) ([5301ce4](5301ce4)) * **telemetry:** send telemetry messages when GRANDPA receieves commit or vote messages ([ChainSafe#2015](https://github.com/timwu20/gossamer/issues/2015)) ([90d5577](90d5577)), closes [ChainSafe#1840](https://github.com/timwu20/gossamer/issues/1840) [ChainSafe#1839](https://github.com/timwu20/gossamer/issues/1839) [ChainSafe#1838](https://github.com/timwu20/gossamer/issues/1838) * **telemetry:** send txpool.import telemetry msg ([ChainSafe#1966](https://github.com/timwu20/gossamer/issues/1966)) ([9f9c213](9f9c213)) ### Reverts * Revert "feat(dot/rpc) implement `author_hasSessionKeys` RPC call (ChainSafe#1704)" (ChainSafe#1714) ([ed56d29](ed56d29)), closes [ChainSafe#1704](https://github.com/timwu20/gossamer/issues/1704) [ChainSafe#1714](https://github.com/timwu20/gossamer/issues/1714)

edwardmack added 3 commits August 17, 2021 17:33

add TODOs to identify where block imported channel is handled

0a59cb3

Merge branch 'development' into ed/fix_send_on_closed_channel

a8cffd8

added comments for imported channels

30338f7

edwardmack requested review from arijitAD, EclesioMeloJunior, jimjbrettj, noot and timwu20 as code owners August 18, 2021 19:47

timwu20 reviewed Aug 18, 2021

View reviewed changes

dutterbutter added the Status: Review Needed label Aug 19, 2021

create constructor for listeners

6ed9d63

edwardmack requested a review from timwu20 August 19, 2021 20:40

dutterbutter removed request for EclesioMeloJunior, noot and jimjbrettj August 20, 2021 15:36

EclesioMeloJunior requested changes Aug 20, 2021

View reviewed changes

dutterbutter added the Status: Changes Requested label Aug 20, 2021

edwardmack added 2 commits August 23, 2021 15:02

Merge branch 'development' into ed/fix_send_on_closed_channel

0bc1f97

added close channel to defer in listen

2ae9572

edwardmack requested a review from EclesioMeloJunior August 23, 2021 19:40

EclesioMeloJunior approved these changes Aug 23, 2021

View reviewed changes

noot reviewed Aug 24, 2021

View reviewed changes

Merge branch 'development' into ed/fix_send_on_closed_channel

0f95d68

EclesioMeloJunior requested changes Aug 26, 2021

View reviewed changes

noot mentioned this pull request Aug 26, 2021

chore(lib/grandpa): update grandpa message tracker to store messages with map of maps, also store commit messages #1769

Merged

GetImportedBlockNotifierChannel doesn't return error, fixed related test

1acfba7

Merge branch 'development' into ed/fix_send_on_closed_channel

ce706cc