Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go-algorand 3.6.2-stable Release PR #3960

Merged
merged 72 commits into from May 9, 2022

Conversation

Algo-devops-service
Copy link
Contributor

@Algo-devops-service Algo-devops-service commented May 7, 2022

GitHub Logo

Overview

Maintenance release of incremental enhancements and fixes.

What's New

  • Minor enhancements to logging
  • Improved debugging support in tealdbg
  • Golang 1.16 upgrade
  • Upgrade arm32 builder to use arm32v7 (deprecates raspberry pi 1 support)

Changelog

New Features

  • Goal: Source Mapping PC to TEAL (#3726)

Enhancements

  • AVM: Deadcode and basic block analysis (#3870)
  • AVM: Perform json_ref expected type validation before expensive JSON processing (#3866)
  • AVM: Extract divideCeilUnsafely to help document opcode costing rationale (#3867)
  • AVM: Assembly simplification & flexible costs (#3857)
  • AVM: Rework around opcode fields for more flexible costs (#3832)
  • AVM: Improve error handling and execution time (#3612)
  • Agreement: add counters for dropped tasks and slow responses (#3861)
  • Agreement: report proposal and vote message buffer sizes at end of round (#3932)
  • Algod: Write to stdout when config.LogSizeLimit is 0 or -o is passed to algod. (#3903)
  • Algod: Add disassemble endpoint (#3908)
  • Algonet: use lower value of LastPartKeyRound for bootstrappedScenario (#3865)
  • Build: use 1.16.15 instead of 1.16.11 (#3833)
  • Build: add build essentials to dependency list (#3782)
  • Build: update build dependencies to use go 1.16 (#3816)
  • Build: compile with golang 1.16 (#3803)
  • Build: Update build image from arm32v6 to arm32v7 (#3933)
  • Ci: remove buildpulse upload step (#3883)
  • Ci: add test partitioning based on partition test verifier output (#3859)
  • Config: add some warning comments about changing RewardUnit value (#3846)
  • Devmode: create a block for external txns only (#3784)
  • Devops: Txnsync recipe updates (#3906)
  • Devops: Added recipe for transaction sync testing. (#3884)
  • Enhancement: Nightly e2e test upload (#3849)
  • Enhancement: Re-enable fillBytes method in ABI and eval.go implementation (#3856)
  • Goal: Add asset optin command (#3881)
  • Goal: Set default key dilution to be the same as algokey (#3801)
  • Goal: Warn on invalid method signature assembly (#3614)
  • Kmd: use updated karalabe lib (#3819)
  • Ledger: test OnlineAccountData (#3831)
  • Ledger: Refactor online totals (#3770)
  • Netdeploy: Add override to enable devmode for a network template. (#3904)
  • Netdeploy: Allow empty private network directory. (#3911)
  • Prefetcher: improve error codes (#3815)
  • REST API: Check for pending transactions in most recent rounds first. (#3836)
  • REST Client: always set Exclude value in accountInformationParams (#3728)
  • Tealdbg: Support for StepOver and refactoring object IDs (#3653)
  • Tealdbg: Replace LocalRunner.Run with LocalRunner.RunAll (#3805)
  • Telemetry: report heartbeat metrics as JSON numbers (#3802)
  • Tests: Disable bandwidth pprof svg generation by default (#3779)
  • Abi: Fix typo in abi_type.go (#3940)

Bugfixes

  • AVM: Fix langspec (and some disassembly errors) for itxn_field (#3869)
  • AVM: Add acct_params_get to langspec.json (#3862)
  • AVM: getbyte and setbyte description to say index must be within length of array (#3772)
  • Algocfg: Fix algocfg get for non-string parameters. (#3907)
  • Algod: Fix issue with nil accounts in local-deltas array (#3790)
  • Algokey: clarify error messages. (#3727)
  • Bug-fix: ABI inferToSlice nil testcases (#3826)
  • Bug-fix: ABI encode infer to slice IsNil handler removal (#3823)
  • Diagcfg: Allow diagcfg to create logging.config in data directory. (#3912)
  • Goal: Rename source map output file (#3905)
  • Goal: Add check for signer passed in the case of logic sig rekeyed account (#3773)
  • Ledger: fix possible dbRound unsynchronization for trackers and registry (#3910)
  • Tests: fix TestAcctUpdateslookupLatestCacheRetry (#3804)
  • Tests: avoid generating large output on TestDeadlockLogging (#3771)
  • Utils: add arm32-specific NanoSleep implementation (#3930)
  • Ledger: fix lookupLatest usage of the resources cache (#3939)

Other

Protocol Upgrade

This release does not contain a protocol upgrade.


NOTE
vFuture changes are not yet available in MainNet or TestNet but can be used in private networks.

  • AVM: vFuture: inner appls can call v4 (#3896)

Additional Resources

tsachiherman and others added 30 commits March 15, 2022 08:47
## Summary

The TestDeadlockLogging was generating large amount of "junk" on top of the expected stack trace.
This PR addresses that by dropping the redundant data.

Example: https://circleci.com/api/v1.1/project/github/algorand/go-algorand/92568/output/106/0?file=true&allocation-id=622fb105959ff4390b6b098f-0-build%2F5VSI2HBA
## Test Plan

Unit test exists.
* Some simple optimizations for app execution

Also some benchmarks that show relative costs for the CPU part of
processing transactions

* Convert opcodes to return error, rather than set cx.err


* Pavel recomendation. Considering bigger change to refreshDebugState


Tests more opcodes, including branches and others that have not return
values.  Automates handling of more opcodes by understanding immediate
arguments.  Fixes the ordering of return value testing.

* Unify naming


Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com>

* make sure return types tested even when simple errors occur

Co-authored-by: michaeldiamant <michaeldiamant@users.noreply.github.com>
* Warn on invalid method signature assembly

* Print warnings to stderr
…count (algorand#3773)

## Summary
When trying to sign a transaction file with goal, if the sender is a normal address but the signer is a logic program, the call fails with an error message like:

`send.txn: txn[0] error LogicNot signed and not a Logic-only account`


The error message comes from https://github.com/algorand/go-algorand/blob/33b87c432065d3be0ce1fdad682604f416676133/data/transactions/verify/txn.go#L314


It notes that a valid case is when Auth is set to the hash of the program but we aren't setting the auth on the transaction even if the -S flag is passed

## Test Plan
Made a txn where sender is a regular account and signed it with

```sh
goal clerk sign -i tmp.txn -o tmp.txn -S BJATCHES5YJZJ7JITYMVLSSIQAVAWBQRVGPQUDT5AZ2QSLDSXWWM46THOY -p tmp.teal
```

I borked the git history on the last one algorand#3459 so just nuked it and starting over here.

Left it at @tsachiherman wanting better tests for this algorand#3459 (comment)
In DevMode, do not create a block for internal events (i.e. compact-cert creation).
This is to assure reproducibility and eliminate the random shifts in round numbers.
## Summary

The existing Ledger.OnlineTotals is going ( at the end of the 320 project ), be using the onlineAccountsTracker to get its information from.
But unlike the existing l.accts.Totals(rnd) call, it won't get the totals, but only the amount of online totals ( i.e. basics.MicroAlgos ).

Prepare the Ledger implementation for that change by deprecating the accountUpdates.Totals and naming it OnlineTotals. As such, it would return just the online portion of that structure ( i.e. totals.Online.Money ).

## Test Plan

Ran existing tests on Ledger.OnlineTotals
## Summary

The changes in algorand#3770 didn't include the changes in algorand#3769 and so tests don't compile on master currently — this updates another test that uses `Totals` to use `LatestTotals` instead.

## Test Plan

Fixes tests.
## Summary

This PR upgrade the go-algorand repository to use go 1.16 while adding workarounds for golang/go#44343.

## Test Plan

Use performance testing to verify no regressions.
## Summary

update go(mod/sum) to use golang 1.16 on the following dependencies:

1. go-codec
2. go-deadlock
3. msgp
4. websocket
5. graphtrace
6. oapi-codegen

## Test Plan

Use existing tests.
* remove slice nil handling, add comments to explain

* update testcases

* partition, dog
`goal` supports signing transactions on a Ledger device.
This is done by using github.com/karalabe/usb which is deprecated.
Since we go-algorand is being updated to 1.16 we update the library
and manually test this version with Ledger.

Tested `goal` with Ledger Nano X and S.
…gorand#3728)

* nit: always set Exclude value in accountInformationParams

* fix comments from CR
* This is a prerequisite test before retiring AccountData.OnlineAccountData()
that is used only in tests agreement and committee tests.
* Historically all these tests depends on genesis data that is basic.AccountData.
* Our GA genesis files have stake, status and voting data and never had resources
basic.AccountData can be reduced to the original stake + voting data state and
a new basic.AccountDataEx can be introduced for using in REST API
* Ledger tests and test network generation tools need to be updated to
accept a new reduced (original) basic.AccountData as genesis and resource records
for assets and apps needed for tests after the ledger initialization.
…tation (algorand#3856)

* Revert "Revert `fillBytes` method to `bigIntToBytes` for lower golang version (algorand#3498)"


* leadingZeros update

* minor updates with elliptics

* byteLength
* dont make array with length if we're just appending to it

* make it ahead of time like other slice init in the same file

* tweaking unit tests to be less forgiving to local deltas slice
Unify most assembly routines to minimize difference and ease doc gen
Flexible opcode costs and fewer assembly routines

Co-authored-by: michaeldiamant <michaeldiamant@users.noreply.github.com>
Also pretty print langsoec so that chnages are easier to see.
jannotti and others added 21 commits April 27, 2022 10:10
AllowV4InnerAppls also means that apps can not perform an inner opt-in
to apps with <v4 CSP (since they might then be locked in). And no app
can downgrade, as that is a straightforward rule to prevent an app
from lowering its CSP and locking other apps into it.

The DoubleLedger does a lot of double checking on ledgers produced by
rerunning transactions after creating blocks, deserializing the txns
from the block and validating the results.

testConsensusRange makes it easy for ledger tests to test behavior in
many protocol levels. A particular problem it handles is that test can
be written to run from some consensus version and in every subsequent
version. So if we change behavior in a new version, we'll find out.


Co-authored-by: algochoi <86622919+algochoi@users.noreply.github.com>
In order to run private networks in docker, it is convenient to mount
the data directory. By disallowing the network directory to exist it is
not possible to mount the node directory.
arm32 has int32 fields in Timespec struct, but Nanoseconds() returns int64.
The new arm32-specific implementation of NanoSleep casts it to int32 as Timespec expects.
This is a quick fix. A better fix would replace this container with
something more lightweight, as it's doing too much for this step.
In some situations the method might incorrectly count
deleted items towards total items. This might result
not returning all the resources to REST API clients.
@codecov-commenter
Copy link

codecov-commenter commented May 7, 2022

Codecov Report

Merging #3960 (0d1e448) into rel/stable (aa2fb0e) will increase coverage by 0.01%.
The diff coverage is 66.73%.

@@              Coverage Diff               @@
##           rel/stable    #3960      +/-   ##
==============================================
+ Coverage       49.78%   49.79%   +0.01%     
==============================================
  Files             392      409      +17     
  Lines           68780    68897     +117     
==============================================
+ Hits            34241    34309      +68     
- Misses          30777    30887     +110     
+ Partials         3762     3701      -61     
Impacted Files Coverage Δ
agreement/cryptoVerifier.go 69.71% <0.00%> (-5.46%) ⬇️
cmd/algod/main.go 0.00% <0.00%> (ø)
cmd/goal/application.go 11.16% <0.00%> (ø)
cmd/goal/bundledGenesisInject.go 100.00% <ø> (ø)
cmd/goal/commands.go 10.40% <0.00%> (-0.04%) ⬇️
cmd/goal/logging.go 8.92% <0.00%> (ø)
cmd/tealdbg/cdtState.go 84.47% <0.00%> (-0.15%) ⬇️
cmd/tealdbg/util.go 84.61% <ø> (-5.13%) ⬇️
cmd/tealdbg/webdbg.go 57.79% <0.00%> (-4.71%) ⬇️
config/localTemplate.go 42.85% <ø> (ø)
... and 104 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa2fb0e...0d1e448. Read the comment docs.

Copy link
Contributor

@michaeldiamant michaeldiamant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable though I assumed the PR's What's New should match #3947.

@algojohnlee algojohnlee merged commit 5e6bc6f into algorand:rel/stable May 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet