Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

algodump is a tcpdump-like tool for algod's network protocol #3166

Merged
merged 5 commits into from
Jan 13, 2022

Conversation

zeldovich
Copy link
Contributor

This PR introduces algodump, a tcpdump-like tool for monitoring algod network messages.

@tsachiherman
Copy link
Contributor

looks good to me - please fix the reviewdog errors before merging this code.

@codecov-commenter
Copy link

codecov-commenter commented Oct 29, 2021

Codecov Report

Merging #3166 (57a996b) into master (b8ac4ba) will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3166      +/-   ##
==========================================
- Coverage   43.80%   43.78%   -0.02%     
==========================================
  Files         392      392              
  Lines       86881    86881              
==========================================
- Hits        38055    38043      -12     
- Misses      42796    42803       +7     
- Partials     6030     6035       +5     
Impacted Files Coverage Δ
agreement/proposal.go 71.96% <ø> (ø)
agreement/vote.go 77.10% <ø> (ø)
util/metrics/gauge.go 68.00% <0.00%> (-2.67%) ⬇️
util/metrics/counter.go 78.57% <0.00%> (-2.39%) ⬇️
txnsync/mainloop.go 85.91% <0.00%> (-1.41%) ⬇️
catchup/peerSelector.go 98.95% <0.00%> (-1.05%) ⬇️
catchup/service.go 69.07% <0.00%> (-0.75%) ⬇️
network/wsPeer.go 72.00% <0.00%> (-0.54%) ⬇️
ledger/acctupdates.go 64.25% <0.00%> (-0.40%) ⬇️
data/transactions/verify/txn.go 44.29% <0.00%> (ø)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8ac4ba...57a996b. Read the comment docs.

@algorandskiy
Copy link
Contributor

LGTM, maybe add a short readme on what is it and how to use

@algorandskiy
Copy link
Contributor

@tsachiherman fixed reviewdogs, let's merge!

@zeldovich
Copy link
Contributor Author

Thanks for fixing up this PR! Sorry I had lost track of it. I've also added a README describing the tool.

@tsachiherman tsachiherman merged commit a1edadc into algorand:master Jan 13, 2022
jannotti pushed a commit that referenced this pull request Jan 13, 2022
* testing: Fix unit test TestAsyncTelemetryHook_QueueDepth (#2685)

Fix the unit test TestAsyncTelemetryHook_QueueDepth

* Deprecate `FastPartitionRecovery` from `ConsensusParams` (#3386)

## Summary

This PR removes `FastPartitionRecovery` option from consensus parameters. The code now acts as if this value is set to true.

Closes https://github.com/algorand/go-algorand-internal/issues/1830.

## Test Plan

None.

* Remaking a PR for CI (#3398)

* Allow setting manager, reserve, freeze, and clawback at goal asset create

* Add e2e tests

* Add more tests for goal asset create flags

Co-authored-by: Fionna <fionnacst@gmail.com>

* [Other] CircleCI pipeline change for binary uploads (#3381)

For nightly builds ("rel/nightly"), we want to have deadlock enabled.

For rel/beta and rel/stable, we want to make sure we can build and upload a binary with deadlock disabled so that it can be used for release testing and validation purposes.

* signer.KeyDilution need not depend on config package (#3265)

crypto package need not depend on config.
There is an unnecessary dependency on config.

signer.KeyDilution takes the `config.ConsensusParams` as argument to pick the DefaultKeyDilution from it. 
This introduces dependency from the crypto package to config package. 
Instead, only the DefaultKeyDilution value can be passed to signer.KeyDilution.

* algodump is a tcpdump-like tool for algod's network protocol (#3166)

This PR introduces algodump, a tcpdump-like tool for monitoring algod network messages.

* Removing C/crypto dependencies from `data/abi` package (#3375)

* Feature Networks pipeline related changes (#3393)

Added support for not having certain files in signing script

Co-authored-by: Tsachi Herman <tsachi.herman@algorand.com>
Co-authored-by: Tolik Zinovyev <tolik@algorand.com>
Co-authored-by: Jack <87339414+algojack@users.noreply.github.com>
Co-authored-by: Fionna <fionnacst@gmail.com>
Co-authored-by: algobarb <78746954+algobarb@users.noreply.github.com>
Co-authored-by: Shant Karakashian <55754073+algonautshant@users.noreply.github.com>
Co-authored-by: Nickolai Zeldovich <nickolai@csail.mit.edu>
Co-authored-by: Hang Su <87964331+ahangsu@users.noreply.github.com>
jannotti added a commit that referenced this pull request Jan 18, 2022
* Three new globals for to help contract-to-contract usability

* detritis

* Check error

* doc comments

* Impose limits on the entire "tree" of inner calls.

This also increases the realism of testing of multiple app calls in a
group by creating the EvalParams with the real constructor, thus
getting the pooling stuff tested here without playing games
manipulating the ep after construction.

* Move appID tracking into EvalContext, out of LedgerForLogic

This change increases the seperation between AVM execution and the
ledger being used to lookup resources.  Previously, the ledger kept
track of the appID being executed, to offer a narrower interface to
those resources. But now, with app-to-app calls, the appID being
executed must change, and the AVM needs to maintain the current appID.

* Stupid linter

* Fix unit tests error messages

* Allow access to resources created in the same transaction group

The method will be reworked, but the tests are correct and want to get
them visible to team.

* Access to apps created in group

Also adds some tests that are currently skipped for testing
- access to addresses of newly created apps
- use of gaid in inner transactions

Both require some work to implement the thing being tested.

* Remove tracked created mechanism in favor of examining applydata.

* Allow v6 AVM code to use in-group created asas, apps (& their accts)

One exception - apps can not mutate (put or del) keys from the app
accounts, because EvalDelta cannot encode such changes.

* lint docs

* typo

* The review dog needs obedience training.

* Use one EvalParams for logic evals, another for apps in dry run

We used to use one ep per transaction, shared between sig and and
app. But the new model of ep usage is to keep using one while
evaluating an entire group.

The app ep is now built logic.NewAppEvalParams which, hopefully, will
prevent some bugs when we change something in the EvalParams and don't
reflect it in what was a "raw" EvalParams construction in debugger and
dry run.

* Use logic.NewAppEvalParams to decrease copying and bugs in debugger

* Simplify use of NewEvalParams. No more nil return when no apps.

This way, NewEvalParams can be used for all creations of EvalParams,
whether they are intended for logicsig or app use, greatly simplifying
the way we make them for use by dry run or debugger (where they serve
double duty).

* Remove explicit PastSideEffects handling in tealdbg

* Always create EvalParams to evaluate a transaction group.

We used to have an optimization to avoid creating EvalParams unless
there was an app call in the transaction group.  But the interface to
allow transaction processing to communicate changes into the
EvalParams is complicated by that (we must only do it if there is
one!)

This also allows us to use the same construction function for eps
created for app and logic evaluation, simplifying dry-run and
debugger.

The optimization is less needed now anyway:
1) The ep is now shared for the whole group, so it's only one.
2) The ep is smaller now, as we only store nil pointers instead of
larger scratch space objects for non-app calls.

* Correct mistaken commit

* Spec improvments

* More spec improvments, including resource "availability"

* Recursively return inner transaction tree

* Lint

* No need for ConfirmedRound, so don't deref a nil pointer!

* license check

* Shut up, dawg.

* testing: Fix unit test TestAsyncTelemetryHook_QueueDepth (#2685)

Fix the unit test TestAsyncTelemetryHook_QueueDepth

* Deprecate `FastPartitionRecovery` from `ConsensusParams` (#3386)

## Summary

This PR removes `FastPartitionRecovery` option from consensus parameters. The code now acts as if this value is set to true.

Closes algorand/go-algorand-internal#1830.

## Test Plan

None.

* base64 merge cleanup

* Remaking a PR for CI (#3398)

* Allow setting manager, reserve, freeze, and clawback at goal asset create

* Add e2e tests

* Add more tests for goal asset create flags

Co-authored-by: Fionna <fionnacst@gmail.com>

* Remove the extraneous field type arrays.

* bsqrt

* acct_holding_get, a unified opcode for account field access

* Thanks, dawg

* [Other] CircleCI pipeline change for binary uploads (#3381)

For nightly builds ("rel/nightly"), we want to have deadlock enabled.

For rel/beta and rel/stable, we want to make sure we can build and upload a binary with deadlock disabled so that it can be used for release testing and validation purposes.

* signer.KeyDilution need not depend on config package (#3265)

crypto package need not depend on config.
There is an unnecessary dependency on config.

signer.KeyDilution takes the `config.ConsensusParams` as argument to pick the DefaultKeyDilution from it. 
This introduces dependency from the crypto package to config package. 
Instead, only the DefaultKeyDilution value can be passed to signer.KeyDilution.

* CR and more spec simplification

* algodump is a tcpdump-like tool for algod's network protocol (#3166)

This PR introduces algodump, a tcpdump-like tool for monitoring algod network messages.

* Removing C/crypto dependencies from `data/abi` package (#3375)

* Feature Networks pipeline related changes (#3393)

Added support for not having certain files in signing script

* e2e test for inner transaction appls

* testing: Add slightly more coverage to TestAcctUpdatesLookupRetry (#3384)

Add slightly more coverage to TestAcctUpdatesLookupRetry

* add context to (most) agreement logged writes (#3411)

Current agreement code only writes a `context : agreement` to a subset of the logged messages.
This change extends the said entry, which would make it easier to pre-process logs entries by their corresponding component. The change in this PR is focused on:
1. make sure that the "root" agreement logger always injects the `context : agreement` argument.
2. change the various locations in the agreement code to use the root agreement logger instead of referring to the application-global instance (`logging.Base()`).

* network: faster node shutdown (#3416)

During the node shutdown, all the current outgoing connections are being disconnected.
Since these connections are web sockets, they require a close connection message to be sent.
However, sending this message can take a while, and in situations where the other party has already shut down, we might never get a response. That, in turn, would lead the node waiting until the deadline is reached.

The current deadline was 5 seconds. This PR changes the deadline during shutdown to be 50ms.

* Give max group size * 16 inner txns, regardless of apps present

* Adjust test for allowing 256 inners

Co-authored-by: Tsachi Herman <tsachi.herman@algorand.com>
Co-authored-by: Tolik Zinovyev <tolik@algorand.com>
Co-authored-by: Jack <87339414+algojack@users.noreply.github.com>
Co-authored-by: Fionna <fionnacst@gmail.com>
Co-authored-by: algobarb <78746954+algobarb@users.noreply.github.com>
Co-authored-by: Shant Karakashian <55754073+algonautshant@users.noreply.github.com>
Co-authored-by: Nickolai Zeldovich <nickolai@csail.mit.edu>
Co-authored-by: Hang Su <87964331+ahangsu@users.noreply.github.com>
Co-authored-by: chris erway <51567+cce@users.noreply.github.com>
@algobarb algobarb mentioned this pull request Jan 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants