Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: docs: Docs Overhaul #1438

Merged
merged 31 commits into from
Apr 6, 2021
Merged

refactor: docs: Docs Overhaul #1438

merged 31 commits into from
Apr 6, 2021

Conversation

RyRy79261
Copy link
Contributor

@RyRy79261 RyRy79261 commented Mar 5, 2021

Review instructions

The contents of each page can be edited at the HackMD link attached to each, for where said files reside in this branch, the path is provided

Changes

Getting Started

Resources

Usage

Integrate

Testing & Debugging @dutterbutter

Deployment

Advanced

  • SCALE - docs/docs/advanced/scale-examples.md

Contributing

Tests


Checklist

  • I have read CODE_OF_CONDUCT and CONTRIBUTING
  • I have provided as much information as possible and necessary
  • I have reviewed my own pull request before requesting a review
  • All integration tests and required coverage checks are passing

Issues

@RyRy79261 RyRy79261 self-assigned this Mar 5, 2021
@RyRy79261 RyRy79261 changed the title Docs Overhaul refactor: docs: Docs Overhaul Mar 9, 2021
@noot
Copy link
Contributor

noot commented Mar 18, 2021

@RyRy79261 can you remove Run official nodes? I don't think it's needed

@dutterbutter
Copy link
Contributor

dutterbutter commented Mar 19, 2021

@RyRy79261 Can you update the Overview section, it shouldn't contain our readme badges, and there is no need to have the outdated table of contents there. You can also remove the Contribute part in there as that is already its own section.

@noot
Copy link
Contributor

noot commented Mar 19, 2021

@RyRy79261 could you add this to the docs as well? https://hackmd.io/@nZ-twauPRISEa6G9zg3XRw/SJ8BgXOXO

@dutterbutter
Copy link
Contributor

dutterbutter commented Mar 24, 2021

@RyRy79261 List of items to update / change. Please also make sure each section is up to date with the linked above HackMD pages.

Getting Started Section

  • Remove table of contents in overview section
  • Update initial overview section, for example this section

For more information about Gossamer, the Polkadot ecosystem, and how to use Gossamer to build and run nodes for various blockchain protocols within the Polkadot ecosystem, check out the Gossamer Docs.

  • General resources link is broken
  • Get Started link is broken
  • Configuration link is broken
  • import runtime link is broken
  • custom services link is broken

Host architecture

  • Remove TODO: update "Host Architecture" document update "Host Architecture" document #918
  • Custom services link is broken
  • Add embedded link when referencing BABE and GRANDPA
  • Add link to JSON-RPC PSP under RPC service
  • Include section for block production and add diagram and dialogue

Package libs

  • Update all referenced links as they seem to be broken
  • Fill out Test packages section

Installation

  • Update to include docker installation in hackmd links

General Resources

  • terms link is broken
  • Please add latest Gossamer blog links

Developer Resources

  • Fix parachain list format
  • Fix block production & consensus list format

Usage Section

Configuration

  • Remove or add TODO: How to target & use custom config

Import Runtime

  • The word containing has a typo under 1. Create genesis spec file with custom runtime section
  • Can you add a screenshot of a running node to illustrate at the end

Integrate

  • Add in the polkadot.js material
  • Once Telemetry is added please include here as well

Testing

  • Please include the testing material created

Deployment

  • You may remove at this time

@codecov
Copy link

codecov bot commented Mar 30, 2021

Codecov Report

Merging #1438 (9b27244) into development (f5de11e) will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #1438      +/-   ##
===============================================
+ Coverage        58.65%   58.67%   +0.02%     
===============================================
  Files              153      153              
  Lines            15196    15196              
===============================================
+ Hits              8913     8917       +4     
+ Misses            4727     4725       -2     
+ Partials          1556     1554       -2     
Flag Coverage Δ
unit-tests 58.67% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cmd/gossamer/flags.go 57.14% <ø> (ø)
lib/blocktree/blocktree.go 56.12% <0.00%> (+1.29%) ⬆️
dot/network/host.go 72.97% <0.00%> (+1.35%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f5de11e...9b27244. Read the comment docs.

@dutterbutter dutterbutter self-requested a review March 30, 2021 15:21
@RyRy79261 RyRy79261 marked this pull request as ready for review March 30, 2021 15:32
Copy link
Contributor

@noot noot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getting an error with make build-mkdocs:

$ make build-mkdocs
docker run --rm -it -v /home/elizabeth/go/src/github.com/ChainSafe/gossamer:/docs/docs squidfunk/mkdocs-material build
Unable to find image 'squidfunk/mkdocs-material:latest' locally
latest: Pulling from squidfunk/mkdocs-material
9aae54b2144e: Pull complete 
70269619ad46: Pull complete 
8f02791f0836: Pull complete 
a70ad90f2b5e: Pull complete 
b0c8198636f4: Pull complete 
4f4fb700ef54: Pull complete 
233bd90a8b53: Pull complete 
ae622ab2271e: Pull complete 
f920f6049f8e: Pull complete 
33d90df435f7: Pull complete 
d3daafa05d8b: Pull complete 
063762c42527: Pull complete 
65898433bd03: Pull complete 
6d0321a18334: Pull complete 
Digest: sha256:be1d882064d0a6589b32f37af8e051efe27aabb867f36dd507f56a49a94890d6
Status: Downloaded newer image for squidfunk/mkdocs-material:latest

Config file '/docs/mkdocs.yml' does not exist.
Makefile:132: recipe for target 'build-mkdocs' failed
make: *** [build-mkdocs] Error 1

@RyRy79261
Copy link
Contributor Author

@noot Makefile commands dont run on my machine properly so just guessed how it might work there.

Would maybe need to add PWD to that command to target the yaml file correctly

@arijitAD
Copy link
Contributor

@noot Makefile commands dont run on my machine properly so just guessed how it might work there.

Would maybe need to add PWD to that command to target the yaml file correctly

The command uses a docker image. It should works unless there is some issue with docker.

@noot
Copy link
Contributor

noot commented Mar 31, 2021

tried editing the makefile command, but seems like it doesn't use the correct path still:

docker run --rm -it -v /home/elizabeth/go/src/github.com/ChainSafe/gossamer/docs/docs squidfunk/mkdocs-material build

Config file '/docs/mkdocs.yml' does not exist.

same with using ${PWD}/docs

is there some other option to specifiy config path?

docs/docs/contributing.md Show resolved Hide resolved
docs/docs/contributing.md Outdated Show resolved Hide resolved
docs/docs/getting-started/overview/host-architecture.md Outdated Show resolved Hide resolved
docs/docs/getting-started/resources/developer-resources.md Outdated Show resolved Hide resolved
@RyRy79261 RyRy79261 merged commit 976f53a into development Apr 6, 2021
@RyRy79261 RyRy79261 deleted the feat/docs-overhaul branch April 6, 2021 08:13
@github-actions
Copy link

github-actions bot commented Apr 9, 2021

🎉 This PR is included in version 0.4.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@github-actions
Copy link

🎉 This PR is included in version 0.4.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

1 similar comment
@github-actions
Copy link

🎉 This PR is included in version 0.4.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

arijitAD added a commit that referenced this pull request Apr 13, 2021
* feat: cmd: implement import-runtime subcommand (#1483)

* fix(lib/crypto/ed25519): update ed25519 to use go-schnorrkel bip39 derivation (#1488)

* fix(state) : Update StorageState to load storage from database. (#1486)

* fix(dot/state): fix usage of trie.Snapshot (#1489)

* fix(dot/state,dot/network): improve memory usage when syncing (#1491)

* fix(dot/network): update notificationsProtocol handshakeData to sync.Map (#1492)

* remove log (#1493)

* feat(dot/network): request block justifications when near head (#1499)

* chore(cmd): rename genesis-raw flag and files to genesis, allow raw and human-readable genesis to be passed to it (#1500)

* refactor: docs: Docs Overhaul (#1438)

* Beginning new docs

* Theming set up

* General layouts n links sorted

* Cleaned up

* Updated Commandline

* Cleaned a bit

* Minor cleaning

* Removed todo

* Config page

* Connect to Polkadotjs

* Fixed dev resources

* New categories sorted

* Updated permalink

* Corrected contributor doc

* Removed unused partials

* Tidying

* Removed offical nodes page

* Amended with feed back

* Resolved a majority of feedback

* Ammeded workflows

* Feedback

* Added diagram

* Recent feedback

* Test package content

* Removed comment

* Debugging page

* Block production expansion

* Typos

* Host spec link

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore(codeowners): update codeowners file (#1505)

* fix: cmd/gossamer: Generate random name if --name flag not set (#1506)

* generate random name if --name flag not set

* update tests with name flag to handle random names

* update tests that reference name cfg

* chore(release): Automated release using semantic-release. (#1503)

* feat: dot/telemetry: Implement basic telemetry connection (#1497)

* implement telemetry connection server

* add telemetry for import block

* setup telemetry to use node name from Global config

* init telemetry connections based on genesis.json values

* fix error message formatting

* add tests for telemetry

* implement no-telemetry cli flag

* fix typos

* cleanup error check

* create TelemetryEndpoint struct

* created connection data struct to hold connection data

* make TelemetryEndpoints a pointer reference

* add tests for interfaceToTelemetryEndpoint

* update test to fix random name

* fix: fix edit link (#1507)

* chore(readme): Fix broken image link (#1509)

Co-authored-by: Arijit Das <arijit@chainsafe.io>

* fix(dot/network): fix justification request at head logic (#1510)

* feat(dot/network): implement persistent peers functionality (#1512)

* chore(release): Update name in package.json. (#1514)

* Update package.json.

* Disable package publish to npm.

* Add support to skip a release.

* feat(dot/network): Add cache for network message. (#1511)

* Add cache for network message.

* chore(release): Fix Deepsource error on development branch. (#1516)

* Fix Deepsource error.

* Fix lint issues.

Co-authored-by: noot <36753753+noot@users.noreply.github.com>
Co-authored-by: Ryan Noble <ryanjnoble@gmail.com>
Co-authored-by: Dustin Brickwood <dustinbrickwood204@gmail.com>
Co-authored-by: Edward Mack <ed@edwardmack.com>
edwardmack added a commit that referenced this pull request Aug 13, 2021
* feat: cmd: implement import-runtime subcommand (#1483)

* fix(lib/crypto/ed25519): update ed25519 to use go-schnorrkel bip39 derivation (#1488)

* fix(state) : Update StorageState to load storage from database. (#1486)

* fix(dot/state): fix usage of trie.Snapshot (#1489)

* fix(dot/state,dot/network): improve memory usage when syncing (#1491)

* fix(dot/network): update notificationsProtocol handshakeData to sync.Map (#1492)

* remove log (#1493)

* feat(dot/network): request block justifications when near head (#1499)

* chore(cmd): rename genesis-raw flag and files to genesis, allow raw and human-readable genesis to be passed to it (#1500)

* refactor: docs: Docs Overhaul (#1438)

* Beginning new docs

* Theming set up

* General layouts n links sorted

* Cleaned up

* Updated Commandline

* Cleaned a bit

* Minor cleaning

* Removed todo

* Config page

* Connect to Polkadotjs

* Fixed dev resources

* New categories sorted

* Updated permalink

* Corrected contributor doc

* Removed unused partials

* Tidying

* Removed offical nodes page

* Amended with feed back

* Resolved a majority of feedback

* Ammeded workflows

* Feedback

* Added diagram

* Recent feedback

* Test package content

* Removed comment

* Debugging page

* Block production expansion

* Typos

* Host spec link

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore(codeowners): update codeowners file (#1505)

* fix: cmd/gossamer: Generate random name if --name flag not set (#1506)

* generate random name if --name flag not set

* update tests with name flag to handle random names

* update tests that reference name cfg

* chore(release): Automated release using semantic-release. (#1503)

* feat: dot/telemetry: Implement basic telemetry connection (#1497)

* implement telemetry connection server

* add telemetry for import block

* setup telemetry to use node name from Global config

* init telemetry connections based on genesis.json values

* fix error message formatting

* add tests for telemetry

* implement no-telemetry cli flag

* fix typos

* cleanup error check

* create TelemetryEndpoint struct

* created connection data struct to hold connection data

* make TelemetryEndpoints a pointer reference

* add tests for interfaceToTelemetryEndpoint

* update test to fix random name

* fix: fix edit link (#1507)

* chore(readme): Fix broken image link (#1509)

Co-authored-by: Arijit Das <arijit@chainsafe.io>

* fix(dot/network): fix justification request at head logic (#1510)

* feat(dot/network): implement persistent peers functionality (#1512)

* chore(release): Update name in package.json. (#1514)

* Update package.json.

* Disable package publish to npm.

* Add support to skip a release.

* feat(dot/network): Add cache for network message. (#1511)

* Add cache for network message.

* chore(release): Fix Deepsource error on development branch. (#1516)

* Fix Deepsource error.

* Fix lint issues.

* chore(codeowners): update codeowners file (#1523)

* fix(dot/network): fix receiving notifications messages from substrate peers (#1517)

* chore(.github): remove development from release flow (#1526)

* chore(dot/network, lib/grandpa): update network.ConsensusMessage, add grandpa.NeighbourMessage and handle accordingly (#1519)

* feat: Add properties and chainId on build-spec command (#1520)

* feat(dot/network, lib/grandpa): request justification on receiving NeighbourMessage, verify justification on receipt (#1529)

* fix (dot/rpc, dot/state): state_subscribeStorage to only notify for value changes (#1460)

* move websocket messages and listeners into own files

* fix notifyStorageSubscriptions to only notify for changes

* address PR comments

* add to websocket tests

* repair append, cleanup filter declareation

* fix anti-pattern in log message

* create notifyStorageSubscription for individual sub notify

* add websocket listeners unit tests

* cleanup merge conflicts

* lint

* add sleep timer

* refactor websocket files

* lint

* a locks to fix data race

* implement observer design pattern

* fix race conditions

* add tests

* add tests

* add tests

* add tests

* add tests

* add tests

* add troubleshooting stuff for testing transactions

* save commit

* address PR comments

* lint

* remove unused printf and comments

* fix test

* update tests

* add return from error

* fix(lib/babe): fix BABE state storing after building block (#1536)

* chore(lib/grandpa): update grandpa message types to match substrate (#1534)

* chore(linter): change locale to UK from US (#1533)

* chore(linter): change locale to UK from US

* fix(babe): Fix extrinsic format in block. (#1530)

* chore:remove codecov token (#1542)

* chore(dot/core, dot/state, lib/grandpa): implement GrandpaState, use in DigestHandler and grandpa.Service (#1540)

* chore(dot/state): create BaseState for accessing non-prefixed db keys (#1546)

* feat(lib/grandpa): fully verify justifications using GrandpaState (#1544)

* chore(lib/common) implement MustHexToBigInt (#1547)

* implement MustHexToBigInt

* add tests for panics

* clean-up string checking

* fix(dot/network): Fix notification handshake and reuse stream. (#1545)

* fix: persist node name (#1543)

* fix(dot/network): split stored streams and handshakeData into inbound and outbound (#1553)

* chore(docs): fix image paths for integrate docs page (#1552)

* fix: update go-schnorrkel version (#1557)

* chore: close db if node initialized check fails (#1551)

* feat(dot/network): add propagate return bool to messageHandler func type to determine whether to propagate message or not (#1555)

* feat (dot/telemetry): implement telemetry system.interval message (#1528)

* implement network data sytem interval telemetry message

* add lock to telemetry struct to fix concurrent websocket writes

* implement block data system.interval telemetry message

* address comments

* fix race condition

* fix lint

* update tests

* refactor tests

* use interface{} for channel, add recover

* rename channel doneNetworkTelemetry to closeCh

* fix check for closed channel

* fix error checking

* feat(lib/grandpa): send NeighbourMessage to peers (#1558)

* chore: merge main into development before release (#1559)

* chore: fix merge issues before release

* fix(release): Trigger release when pushed to main branch. (#1566)

* feat: add --chain dev option (#1561)

* fix(dot/sync): fix creating block response, fixes node sync between gossamer nodes (#1572)

* fix(lib/babe): add pre-runtime digest before calling initialize_block (#1581)

* fix(polkadot.js) fix -infinity in keys error by adding genesis field (#1573)

* chore: add polkadotjs tests to ci (#1567)

* chore: add polkadotjs tests to ci

* Update .github/workflows/tests.yml

Co-authored-by: Edward Mack <ed@edwardmack.com>

* chore: update dockerfile to install yarn and cd into js test dir and install dependencies

* chore:updated dockerfile to run polkadotjs tests

* chore: updated deps to include js tests

* chore: removed unneeded command in test wrkflow

* add rpc module to gossamer start-up

* fix linter rants

Co-authored-by: Edward Mack <ed@edwardmack.com>
Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* fix(lib/babe): call AddBlock in BABE synchronously (#1585)

* chore(docs): update README and installation doc to include dev node (#1583)

* fix(docs) rename hosts package to dot packages (#1582)

* chore: improved ci caching for mod and build. Also add codecov id-ci-fail success attribute (#1590)

Co-authored-by: Arijit Das <arijit@chainsafe.io>

* chore: fix block production path and add article links (#1593)

* feat(rpc/subscription): implement state_unsubscribeStorage (#1574)

* implement state_unsubscribeStorage

* add value checks, add tests

* handle string parameter, add tests, use const for error messages

* parse to uint

* update type

* update variable names (based on comments)

Co-authored-by: Arijit Das <arijit@chainsafe.io>

* fix: pending bubble hidden after block included (#1592)

* fix: pending bubble hidden after block included

* chore: fix typo

* chore: change FindExtrinsic to HasExtrinsic

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* fix(dot/network) track sent/received requests by hash (#1575)

* fix(dot/network): add map to track requests by hash

* chore: remove store request by hash when sync fails

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* feat(cmd): implement offline pruning of state trie (#1564)

* fix(dot/network): fix discovery between gossamer nodes (#1594)

* fix(utils): create a specific folder for database (#1598)

* feat: create a specific folder for database

* chore: remove unused const

* fix: add default database dir on NodeInitialized function

* fix: change default database dir to db and fix typo

* chore(review): Add Eclesio to code owners (#1601)

* chore(dot/network): use sync.Pool for network message buffers (#1600)

* chore(lib/babe): create BlockBuilder type (#1602)

* fix: update deprecated package (#1603)

* fix(lib/blocktree): fix setting leaves after blocktree pruning (#1605)

* fix(dot/telemetry): refactor telemetry to reduce CPU usage (#1597)

* chore(dot/network): improve CPU usage again; no more mallocs (#1608)

* fix(dot/core): Fix handle transaction message test. (#1607)

* fix(dot/core): Fix handle transaction message test.

* Remove hardcoded extrinsic value.

* Add subkey for Macos.

* Add subkey in the coverage report.

* chore(dot/network): Add test for leb128 to uint64 conversion. (#1614)

* Add test for leb128 to uint86 conversion.

* feat(dot/network): implement streamManager to cleanup not recently used streams (#1611)

* fix(dot/network): implement a handshake timeout (#1615)

* chore: return the bytes read by leb128

* feat: add handshake timeout

* chore: remove debug network loglevel

* chore: get back to trace

* chore: stop ticker no matter the case

* chore: implement timer and use unbuffered

* chore: using defer

* chore: remove the timer.Stop() when the <-timer.C was called

* fix(dot/network): check if peer supports protocol (#1617)

* fix(dot/core): Add only extrinsic during chain reorg. (#1609)

* maintainence: fix network tests on CI  (#1627)

* fix(dot/state, lib/babe, lib/trie): improve syncing between gossamer authority nodes (#1613)

* fix(dot/network, lib/grandpa): fix handshake decoding and grandpa message handler sigabort (#1631)

* chore: remove finalnum inherent, clear babe `slotToProof` map (#1632)

* fix(dot/network): Check for size when decoding leb128. (#1634)

* chore: replace handmade test mocks with auto-generated mocks (#1626)

* chore(lib/blocktree): cache nodes in map (#1633)

* fix(lib/babe): fix timing for transition between epochs (#1636)

* feat(dot/sync): implement codeSubstitutes (#1635)

* chore(deps): bump glob-parent in /tests/polkadotjs_test (#1637)

* feat(dot/state): implement online pruning of historical state tries (#1596)

* chore: updated doc links (#1647)

* chore: updated doc links

* chore: increase timeout

* chore: update linter config

* chore(dot/core, dot/sync, lib/babe): refactor handling on block import to reduce duplicate code (#1645)

* chore: change tests log level to info (#1644)

* chore: change tests log level to info

* chore: revert previus log levels

* chore: fix the corrects levels

* ajust make mock command

* exec go mod tidy

* fix lint

* chore: update docs

* fix(dot/core): check transaction Validity.Propagate field to determine whether to propagate tx (#1643)

* check validate.Propagate field

* add functionality to remove non-propagate txs

* fix mocks

* chore: adding verbosity to golangci

* chore: change the out-format argument golangci

* using print-issued-lines arg

* ínstalling golangci instead use github actions

* lint

* clean-up logic checks

* run make mock

* update comments, fix mock output

* add test for bogus extrinsic

* fix spelling

* change logging level

Co-authored-by: Eclésio Júnior <eclesiomelo.1@gmail.com>

* refactor(lib/scale): refactor lib/scale into own package with idiomatic types, marshalling, and optionality (#1548)

* scale encoding with optionality, struct tag field ordering, uint128

* add indices caching

* add mtx

* add variable data type interface, test to compare old vs new encoding

* fix width int decode and tests

* wip

* []byte, string decoding

* encodeBool and tests

* refactor tests, include optionality tests

* use shared tests in decode

* struct decode tests, and unmarshal refactor

* wip

* decode of VariantDataType, wip tests

* add optionality testing

* fix struct tests and optionality tests

* test VaryingDataType

* wip decode refactor, use reflect.Value as passed param

* repurpose int and uint as compact length encoded integers, remove unnecessary handling of []int

* cleanup, and all tests benchmark

* add README

* update README

* update readme

* update readme

* update README

* update README

* rResult encode/decode and RegisterResult

* wip cr feedback

* add licenses

* add custom primitive encode/decode

* more cr feedback

* cr feedback

* wip

* revise Result

* add readme

* add usage example for Result

* refactor(lib/scale): Revise VaryingDataType interfaces, and introduce VaryingDataTypeSlice (#1651)

* wip VaryingDataType and VaryingDataTypeSlice

* refactor VaryingDataType and add VaryingDataTypeSlice

* update README

* fix lint

* update examples

* fix deepsource

* check decoded in comparison_test

* cr feedback

* fix result.set with nil value

* feat(dot/babe) implement block production time metric (#1648)

* metric: gossamer_proposer_block_constructed

* chore: improve metrics instance and add tests

* chore: add peer check and peerstore metrics

* chore: fix pacakge name and interface to gauge struct

* chore: change to const

* chore: adjust babe metrics

* chore: removing duplicated net metrics

* chore: improve babe time to build block test

* chore: fix CI tests failures

* exec go mod tidy

* chore: go mod

* change docker-compose instructions to docs/ folder

* fix(lib/babe): fix setting first slot of network, fix loading BABE epoch params (#1640)

* chore: replace time.After with time.NewTicker (#1650)

* replace time.After with time.NewTimer

* replace time.Affer with time.NewTicker

* lint

* replace time.After is discovery so ttl var is used

* replace time.After in if statement

* add configuration variables for time duration functions

* feat(dot/telemetry): implement telemetry message network_state (#1618)

* refactor telemetry messages to map format

* add basic network state telemetry message

* refactor message sender to handle interface{} types

* refactor telemetry messages to be structs

* lint

* go fmt

* lint

* move msg building logic outside msg sending loop

* make telemetry messages an interface

* Lookup transactions count from TransactionsState

* address comments

* fix mocks for tests

* lint

* refactor TelemetryMessage to Message

* update mock handler to return result

* add TransactionsCount to mockhandler

* move logic to build new network state message

* lint

* fix interface

* update mockhandler

* lint

* chore: update dev, gssmr to use genesis-spec by default (#1665)

* feat(lib/transactions) ready transactions metrics (#1656)

* pool ready transaction metrics

* chore: add priority queue metrics

* chore: fix lint

* chore: remove println

* chore: add copyright comments

* chore: resolve comments

* chore: remove stop function and add comments to exported funcs

* chore: adjust tests

* chore: make metrics.Start blocking and using wg to inner gorout

* chore: fix typo

* chore: fix metrics test

* fix: solving data race condition on metrics tests

* feat(lib/trie): Parallel hash trie. (#1657)

* feat(lib/trie): Parallel hash trie.

* Fix race.

* Use bytes.Buffer in pool.

* chore: update gssmr genesis to v0.9; stub missing ext_ funcs (#1625)

* refactor(dot/rpc/modules) Add author unit tests, migrate old tests to integration tests (#1666)

* chore (pkg/scale) scale babe integration (#1670)

* integrate scale into babe library

* use results for unmarshalling

* clean up code

* fix switch to pass checks

* integrate scale into dot/sync

* cr feedback

* merge

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* feat(lib/grandpa) implement grandpa finality round metrics (#1655)

* chore: implement grandpa finality round

* pool ready transaction metrics

* chore: add priority queue metrics

* chore: fix lint

* chore: add gauge collector interface

* chore: fix lint

* remove unused metrics timeout

* chore: remove unused test

* remove unused consts

* chore: adding tests

* fix(dot/network, lib/grandpa): fix node sync, improve devnet finality

* fix(dot/node): Start websocket server only with `--ws` flag (#1671)

* fix (dot/telemetry): NoTelemetry flag stops telemetry (#1660)

* refactor(dot/rpc/subscription): refactor websocket `HandleComm` (#1673)

* chore(pkg/scale): scale integration into genesis (#1669)

* fix(dot/network): fix stream manager tests (#1683)

* chore (pkg/scale) integrate scale into dot/core and dot/sync (#1676)

* scale into messages_test

* integrate scale into dot core

* integrate scale into dot/sync

* integrate scale into sync and core

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore (pkg/scale) integrate scale pkg into dot/rpc (#1678)

* integrate scale into dot/sync

* integrate scale into dot/rpc

* scale into epoch

* integrate scale into pruner

* integrate scale pkg into dot/state

* WIP/Fix digest encoding errors

* remove changes in dot/state to include in separate PR

* feat(lib/grandpa) add round state rpc call (#1664)

* chore: add interface for grandpa in rpc pkg

* chore: create roundState rpc call

* chore: coment unused branch

* chore: fix lint

* chore: add test case

* chore: fix lint

* chore: address comments

* chore: fix the diff implementation

* chore: address comment

* chore: change PreVotes and PreCommits interface signature

* chore: remove check

* chore: improve code defs

* chore: replace unbuffered channels with buffered channels (#1668)

* add buffer to channels

* add buffers to channels in tests

* remove buffers from channels that shouldn't be buffered

* added DEFAULT_BUFFERS_SIZE const

* lint

* addres comments

* fix(lib/babe): always use 2/3 of slot to produce block, re-add potentially valid txs to queue (#1679)

* fix(dot/types): fix max value for digest (#1687)

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* fix(dot/state, lib/grandpa): update justification and SignedVote handling in database (#1682)

* feat(dot/rpc) Add `system_localPeerId` rpc call (#1690)

* chore: add local peer id rpc call

* chore: add func desc

* cover empty test case

* chore: ignore Id on func name

* chore: adjusts tests

* chore: added cname for docs (#1692)

* feat(dot/rpc): Add `system_localListenAddresses` RPC call (#1689)

* chore(lib/grandpa, dot/network): send `CommitMessage` directly to peer on round mismatch; cleanup grandpa `receiveMessages`  (#1684)

* feat(dot/rpc) Add `system_syncState` rpc call (#1691)

* chore: adjust tests

* chore: exposing network methods

* chore: add test

* chore: resolve lint and improve func description

* chore: resolve lint

* fix: update gssmr genesis to use pallet_babe::SameAuthoritiesForever (#1696)

* chore(lib/common) implement byte pool to improve websocket subscription efficiency (#1693)

* implement bufferpool

* implement byte pool for subscription setup

* lint

* fix lint

* address comments

* refactor UnregisterFinalizedChannel to UnregisterFinalisedChannel

* lint

* change error handling

* refactor NumPooled to Len

* fix(dot/state): track runtime per-block, fix runtime upgrades differing between forks (#1638)

* fix runtime upgrade

* Add runtime in blocktree.

* Remove runtime instance from babe service.

* Remove runtime instance from sync service.

* Self review.

* Fix chain reorg test.

* Fix failing test.

* Self review.

* Address comments.

* Remove unused functions from interface.

* Fix failing test.

* fix TestService_HandleSubmittedExtrinsic

* update HandleTransactionMessage to set runtime storage before validating

* Fix failing test.

* address comment

* lint

* cleanup

Co-authored-by: noot <elizabethjbinks@gmail.com>
Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* feat(dot/network) add `network_is_major_syncing` metric (#1697)

* chore: transform goal into atomic

* add tests

* remove debug from polkadot config

* chore: add comment

* chore: remove comments

* change var name

Co-authored-by: Timothy Wu <timwu20@gmail.com>

* update var name

* chore: add metrics port

* chore: add clean up to close dbs while testing

Co-authored-by: Timothy Wu <timwu20@gmail.com>

* fix(dot/network): decrease DHT find peers interval for gssmr nodes (#1703)

* fix(docs): improve build-spec usage docs (#1706)

* feat: improve build-spec usage docs

* add doc to use  build-spec command using --output flag

* fix(dot/state): add StorageState Lock/Unlock API for usage by babe and sync  (#1700)

* add Jimmy to codeowners (#1711)

* feat(dot/rpc) implement `author_hasSessionKeys` RPC call (#1704)

* chore: add interface for grandpa in rpc pkg

* chore: create roundState rpc call

* chore: coment unused branch

* chore: fix lint

* chore: add test case

* chore: fix lint

* chore: add grandpa subscribe justification rpc call

* chore: add cancel function to stop goroutine when unsuubscribe

* chore: resolve lint

* chore: add tests to subscribe justification call

* remove deps from round state rpc call pr

* chore: remove unecessary changes

* remove inpackage well mock is in mocks folder

* chore: fix lint

* wip: fixing tests

* chore: use channels to control goroutines

* chore: resolve lint

* add time.Duration on structs

* chore: add runtime method and rpc method

* wip: fix runtime response scale decoding

* chore: adding data to tests

* wip: improve test coverage and assertions

* chore: hasSessionKey rpc call done

* chore: change config.toml back

* chore: remove log, add string.EqualFold

* chore: remove unused logs, get back new private key

* update hasSessionKey method to use coreapi insted of runtime api

* chore: resolve lint issues

* chore: change from []byte to []uint8

* chore: fix tests failures

* chore: fix HasSessionKeyResponse comments

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore: fix KeyTypeID comments

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore: update comments, unexport struct and remove qtyCheck

* chore: improve hasSessionKey key check

* chore: update params to represent better data

* chore: improve func name to DecodeKeyPairFromHex

* chore: get back func names to avoid naming conflicts

* chore: update go.sum

* chore: update grandpa_subscribeJustification return response

* chore: fix lint issues

* chore: check the len of the slice of decoded keys

* chore: update exports comments

* chore: update the import style

* chore: improve DecodeKeyPairFromHex export comment

* chore: hasSessionKeys test fixed

* chore: improve export function comment

* chore: group rpc methods string in a unique place

* chore: use chan struct{}

* chore: fix tests that uses wsconn

* chore: fix subscription test

* chore: improve log message

* chore: fix lint issues

* chore: increase the author RPC method qty

* chore: fix deepsource style error

Co-authored-by: Arijit Das <arijitad.in@gmail.com>
Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* Revert "feat(dot/rpc) implement `author_hasSessionKeys` RPC call (#1704)" (#1714)

This reverts commit 86df957.

* feat(dot/rpc) add `grandpa_subscribeJustifications` rpc call (#1672)

* chore: add interface for grandpa in rpc pkg

* chore: create roundState rpc call

* chore: coment unused branch

* chore: fix lint

* chore: add test case

* chore: fix lint

* chore: add grandpa subscribe justification rpc call

* chore: add cancel function to stop goroutine when unsuubscribe

* chore: resolve lint

* chore: add tests to subscribe justification call

* remove deps from round state rpc call pr

* chore: remove unecessary changes

* remove inpackage well mock is in mocks folder

* chore: fix lint

* wip: fixing tests

* chore: use channels to control goroutines

* chore: resolve lint

* add time.Duration on structs

* chore: update grandpa_subscribeJustification return response

* chore: fix lint issues

* chore: resolve tests issues

* chore: fix channels and methods const

* chore: improve error messages

Co-authored-by: Arijit Das <arijitad.in@gmail.com>

* chore(lib/runtime): stub v0.9.8 runtime funcs, partially implement storage ext_ funcs (#1710)

* fix(lib/grandpa): fix grandpa stall and various bugs (#1708)

* feat(dot/rpc) implement `author_hasSessionKeys` RPC call (#1715)

* chore: add interface for grandpa in rpc pkg

* chore: create roundState rpc call

* chore: coment unused branch

* chore: fix lint

* chore: add test case

* chore: fix lint

* chore: add grandpa subscribe justification rpc call

* chore: add cancel function to stop goroutine when unsuubscribe

* chore: resolve lint

* chore: add tests to subscribe justification call

* remove deps from round state rpc call pr

* chore: remove unecessary changes

* remove inpackage well mock is in mocks folder

* chore: fix lint

* wip: fixing tests

* chore: use channels to control goroutines

* chore: resolve lint

* add time.Duration on structs

* chore: add runtime method and rpc method

* wip: fix runtime response scale decoding

* chore: adding data to tests

* wip: improve test coverage and assertions

* chore: hasSessionKey rpc call done

* chore: change config.toml back

* chore: remove log, add string.EqualFold

* chore: remove unused logs, get back new private key

* update hasSessionKey method to use coreapi insted of runtime api

* chore: resolve lint issues

* chore: change from []byte to []uint8

* chore: fix tests failures

* chore: fix HasSessionKeyResponse comments

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore: fix KeyTypeID comments

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore: update comments, unexport struct and remove qtyCheck

* chore: improve hasSessionKey key check

* chore: update params to represent better data

* chore: improve func name to DecodeKeyPairFromHex

* chore: get back func names to avoid naming conflicts

* chore: update go.sum

* chore: update grandpa_subscribeJustification return response

* chore: fix lint issues

* chore: check the len of the slice of decoded keys

* chore: update exports comments

* chore: update the import style

* chore: improve DecodeKeyPairFromHex export comment

* chore: hasSessionKeys test fixed

* chore: improve export function comment

* chore: group rpc methods string in a unique place

* chore: use chan struct{}

* chore: fix tests that uses wsconn

* chore: fix subscription test

* chore: improve log message

* chore: fix lint issues

* chore: increase the author RPC method qty

* chore: fix deepsource style error

* chore: keep just related changes

* chore: improve test coverage

* chore: improve test coverage on lib/keystore/helpers.go

* chore: improve testing and remove unused function

* chore: create more test cases to hasSessionKey RPC method

Co-authored-by: Arijit Das <arijitad.in@gmail.com>
Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* feat(dot/rpc/modules) implement `state_queryStorage` rpc method (#1707)

* wip: state query storage rpc

* chore: query keys from block stateRoot

* chore: implemented rpc method and format response correctly

* chore: check starting block hash is nil

* chore: use GetStorage instead of TrieState

* chore: remove logs

* chore: resolve lint issues

* chore: preallocate slice, use varidic param

* chore: update variable name

* chore: made from not nulable

* chore: get best block hash if to is nil

* chore: remove pointer and compare with EmptyHash

* chore: fix tests, remove write to nil reference

* chore: fix deepsource rule

* chore: fix rpc test

* chore: uncomment conditional test

* chore: improve testing

* chore: change from null to a empty array

* chore: skip state_queryStorage RPC method test

* chore: remove unconsitent test check

* feat(dot/rpc/modules): add `system_addReservedPeer` and `system_removeReservedPeer` RPC call (#1712)

* chore: include rpc methods and adding peers on store

* chore: add unit tests for reserved peers on net layer

* chore: add unit tests for reserved peers on net layer

* chore: remove redundancy conditionals

* chore: add and remove peers from protected map

* increase the system RPC methods qtt

* chore: test add and remove protected peers

* chore: improve comment at removeReservedPeers

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore: improve comment at addReservedPeers

* chore: jump when there is no peer to disconect

* chore: fix lint issues

* chore: remove unused checks and use trim space

* chore: improve test coverage

* chore: remove global websocket test variable

* chore: improve tests on modules/system

Co-authored-by: noot <36753753+noot@users.noreply.github.com>

* chore: updated pr template to include primary reviewer (#1724)

Co-authored-by: noot <36753753+noot@users.noreply.github.com>
Co-authored-by: Arijit Das <arijit@chainsafe.io>
Co-authored-by: Ryan Noble <ryanjnoble@gmail.com>
Co-authored-by: Dustin Brickwood <dustinbrickwood204@gmail.com>
Co-authored-by: Eclésio Junior <eclesiomelo.1@gmail.com>
Co-authored-by: Kanishka <kanishkatn@gmail.com>
Co-authored-by: noot <elizabethjbinks@gmail.com>
Co-authored-by: Daniel Gómez <dani91.gomez@gmail.com>
Co-authored-by: Arijit Das <arijitad.in@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Timothy Wu <timwu20@gmail.com>
Co-authored-by: JimboJ <40345116+jimjbrettj@users.noreply.github.com>
@github-actions
Copy link

github-actions bot commented Dec 3, 2021

🎉 This PR is included in version 0.6.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants