block height gate balance snapshot comparison #516
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mattfoley8
added a commit
that referenced
this pull request
May 4, 2023
* make sure donePeerChan is not nil (#512) Co-authored-by: Lazy Nina <> * Add epoch data type. * Update func naming. * Fix nil error. * Add prefix encoder db util. * Rename field to FinalBlockHeight. * Avoid snapshotting issue for now. * Update ancestral record before setting in db. * Incorporate epoch number into PoS txns. * block height gate balance snapshot comparison (#516) * [stable] Release 3.4.1 * Re-enable force-checksum (#519) --------- Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com>
mattfoley8
added a commit
that referenced
this pull request
May 12, 2023
* bump go version to 1.20 (#490) * bump go version to 1.20 * run gofmt * Balance Model (#494) * [stable] Release 3.3.0 * Fix access group spending limit encoding (#493) * save current progress * check in current progress * hacky fix to make block reward disconnects work - temporary fix * Check in progress * run gofmt * add back reorg test, fix derived key test, and add more comments where things aren't currently working * more fixes to tests * check in progress * gofmt * fix failing old tests? * skip old tests * fix GetVersionByte for UtxoOperation * turn off badger logging for now: * skip broken tests for serialization for now * remove skips in block view test * revert changes to global deso limit in hardcore derived key test * uncomment balance model height = 7 in balance model derived key test * fill in placeholders for balance model balances * fix swap identity and cc tests * check in progress on tests * break up cc tests * break 'em up more * individually run all balance model tests * Ln/cleanup in new low diff bc (#481) * add cleanup method * Fix bug * more test fixes * more test fixes * use mempool for view by default in _getBalance * refactor uses of _getBalance in help creator coin test func * fix formatting of int to string * check in progress on tests * don't loop in UpdateProfileChangeBack test * skip broken test for now * fix message tests and one cc test * fix panic in postgres * cleanup * more memory cleanup * fix buy now nft bid disconnect logic * remove skip for fixed test * remove skip on fixed test * remove conditions used for debugging * fix auth derived key w/ access groups test * fix network tests * fix associations test and fix bug in connect transaction related to setting nonce * remove todos * gofmt * clean up block view.go * clean up block view association test.go * clean up bv bitcoin and dao coin limit order * move buy now fee computation to create nft bid func instead of add inputs and change w/ subsidy * more clean up * more clean up * turn off badger logging again, fix like test * only turn off badger logging for tests * ensure migration heights are versions properly * fix create max send for balance model txns, set block height for balance model in regtest to block 1 for convenience * gofmt * Fix fee computation by setting TxnFeeNanos to max uint64 * fix fee calculation * add minFeeRate not equal to zero check * fix fee calc and cleanup * address MF feedback * gofmt * more small cleanup * put next nonce in view in GetNextNonceForPKID * address mf feedback * gofmt * Address SA feedback and add lib/mem.log to gitignore * remove explicit spend from txn fees (#485) * remove explicit spend from txn fees * remove errant character * Fix derived key accounting * only call unspend in disconnect nft bid if past block height, small cleanup * add more detailed comments * fix dao coin limit orders * revert change to fee check for dao coin limit order in connect transaction * simplify derived key accounting for dao coin limit order * address MF feedback, delete filled orders from view after connecting basic transfer * Fix derived key accounting again * address MF feedback * revert removal of -1 from block height in calls to spend balance * use new _connectBasicTransferWithExtraSpend * fix utxo op expectations in derived key tests * simplify disconnect basic transfer * burn create NFT and Create profile fees (#487) * fix fee compute in connect transaction * use fees to compare to dao coin limit order fee nanos * Fix nft tests * break up tests to lower memory usage * address SA feedback * cleanup comment * New Nonce Scheme (#489) * remove explicit spend from txn fees * remove errant character * Fix derived key accounting * only call unspend in disconnect nft bid if past block height, small cleanup * add more detailed comments * fix dao coin limit orders * revert change to fee check for dao coin limit order in connect transaction * simplify derived key accounting for dao coin limit order * address MF feedback, delete filled orders from view after connecting basic transfer * Fix derived key accounting again * address MF feedback * revert removal of -1 from block height in calls to spend balance * use new _connectBasicTransferWithExtraSpend * fix utxo op expectations in derived key tests * simplify disconnect basic transfer * burn create NFT and Create profile fees (#487) * fix fee compute in connect transaction * use fees to compare to dao coin limit order fee nanos * Fix nft tests * break up tests to lower memory usage * address SA feedback * Ln/new nonce scheme (#486) * Save broken progress * save current progress * save current progress * Initial check in for new nonce scheme * check in current progress * stop block producer on quit * turn off badger logging for snapshot in regtest mode * cleanup * add support for param updater setting the max nonce expiration block buffer global param, comment out one test that is failing rn * gofmt * bump cached recent blocks on test block producer, assorted cleanup * gofmt and fix nft test * fix basic transfer balance model test * skip network test temporarily, another fix for nft tests * cleanup and add comments * address SA feedback * fix refactor * move nonce expiration clearout logic * fix block version 0 test * address most of MF feedback * add test to check that mempool rejects txns with a bad expiration block height in the nonce * check in progress on adding utxo op to delete expired nonces * fix typo * fix disconnect block logic * make transactor nonce entry and deso nonce implement the encoder interface * clean up * address SA feedback * add len of utxo ops check in disconnect block * gofmt * address MF feedback * refactor _addDESO to take in a utxo entry instead of a func returning a utxo entry * cleanup * fix nft spending of bidder DESO * gofmt * wip * initial checkin for addressing DH feedback (#491) * initial checkin for addressing DH feedback * fix utxo op expectations in tests * add balance validations * copy coin entries, track NFT bid/accept bid txn changes to CC deso locked * gofmt * gofmt * cleanup * always validate that we aren't printing any DESO * use db adapter to get balance from badger or postgres * fix minor nits * nit formatting and comments * Diamondhands/balance model review (#492) * wip * initial checkin for addressing DH feedback (#491) * initial checkin for addressing DH feedback * fix utxo op expectations in tests * add balance validations * copy coin entries, track NFT bid/accept bid txn changes to CC deso locked * gofmt * gofmt * cleanup * always validate that we aren't printing any DESO * use db adapter to get balance from badger or postgres * fix minor nits * nit formatting and comments * Address DH Feedback * set default max expiration to 288 (1 days worth of blocks) and fix expiration block height computation to prevent underflow while still reducing the expiration buffer to prevent nodes from rejecting transactions with nonces that are too high --------- Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix access group spending limit encoding * Fix access group spending limit encoding * Remove safety bytes addition from AccessGroupSpendingLimit fix * Tests passing: Fix balance model bugs and tidy * PartialID was being generated using math/rand which is deterministic. Switched it to wire.RandomUint64(). Actually found this thanks to Rosetta-- it generates a lot of transactions at once and they all had the same nonces. * _computeMaxTxSize() could return a different value depending on the size of the txn nonce. This means you can get a different suggested fee depending on what you chose as your nonce, which made Rosetta fail in a non-deterministic way. Took a while to hunt this one down... * Defined Txn version as its own type so we can reference it better. * Made checksum false by default temporarily. Will revert it after the upgrade. * Fixed all the tests (took forever). * Test access group spending limit encoder migration * Fix merge artifacts * gofmt... * Fix params * Fix params v2 * Fix paramss v3 lol * Make encoder test global * clean up deso encoder testing before logging CLEANUP_END --------- Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * include extra data, txn version, txn nonce, and txn fee nanos when unmarshaling in go (#496) * make Read transaction functions public (#498) Co-authored-by: Lazy Nina <> * Ln/fix peer panic issue (#499) * only expect v2 bundle if peer has a srv and a blockchain * use .Params instead of .srv.blockchain.Params * add recover from panic in _handleOutExpectedResponse * add comment * convert _handleOutExpectedResponse to switch case * fix hypersync nonce issue (#500) * RegisterAsValidator + UnregisterAsValidator TxnTypes (#495) * Add block view validator. * Add unregister as validator types. * Add validator entry db utils. * Add blockchain utils. * Add utxo view getters. * Delete then set instead of updating validator entry. * Add connect register as validator logic. * Add disconnect register as validator. * Add connect and disconnect unregister as validator. * Add unregister as validator blockchain and mempool utils. * Only overwrite validatorID if new validator. * Use url parse request URI to validate domains. * Tie in register/unregister txn types. * Finish blockchain.go construct validator txns. * Add test file template. * Add tests for validations. * Add tests for registering, querying, unregistering w/o flushing. * Find top validators by stake across the db and UtxoView. * Fix tests when flushing to db. * Test updating a validator. * Store blockheights as uint64. * Fix disconnect logic for register as validator txn. * Fix disconnect tests for unregister txn. * Add stubbed voting public key fields. * Change weird UtxoOperation version to see if fixes CI. * Fix block heights in tests. * Add passing test for creating validator derived key. * Fix merge conflicts more. * Test registering + unregistering validators using derived keys. * Update TODO comments. * Clean up comments. * Tie in txindex metadata creation. * Split out setting/flushing global stake amount nanos. * include extra data, txn version, txn nonce, and txn fee nanos when unmarshaling in go (#496) * Address PR feedback. * make Read transaction functions public (#498) Co-authored-by: Lazy Nina <> * Ln/fix peer panic issue (#499) * only expect v2 bundle if peer has a srv and a blockchain * use .Params instead of .srv.blockchain.Params * add recover from panic in _handleOutExpectedResponse * add comment * convert _handleOutExpectedResponse to switch case * Address PR feedback. * Update the ValidatorMapKey fields. * Make copy of value when merging extradata. * Make a copy of the ValidatorEntries on a new view. * Add comment re: validators by stake index value. --------- Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * error on no nonce after block height (#508) * [stable] Release 3.4.0 * Add txn v1 fields to postgres (#510) * check in initial progress * gofmt * fix migration * add balance model associations test to postgres ci * add balance model associations test to postgres ci (#511) * add balance model associations test to postgres ci * don't flush utxos if len is 0 * Mf/add block view stake (#501) * Add block view validator. * Add unregister as validator types. * Add validator entry db utils. * Add blockchain utils. * Add utxo view getters. * Delete then set instead of updating validator entry. * Add connect register as validator logic. * Add disconnect register as validator. * Add connect and disconnect unregister as validator. * Add unregister as validator blockchain and mempool utils. * Only overwrite validatorID if new validator. * Use url parse request URI to validate domains. * Tie in register/unregister txn types. * Finish blockchain.go construct validator txns. * Add test file template. * Add tests for validations. * Add tests for registering, querying, unregistering w/o flushing. * Find top validators by stake across the db and UtxoView. * Fix tests when flushing to db. * Test updating a validator. * Store blockheights as uint64. * Fix disconnect logic for register as validator txn. * Fix disconnect tests for unregister txn. * Add stubbed voting public key fields. * Change weird UtxoOperation version to see if fixes CI. * Fix block heights in tests. * Add passing test for creating validator derived key. * Add block view stake. * Add types and byte encodings. * Fix merge conflicts more. * Add db utils. * Fix failing CI. * Add metadata validations. * Reorder keys in prefix. * Test registering + unregistering validators using derived keys. * Update TODO comments. * Clean up comments. * Add UtxoView utils. * Add mempool utils. * Tie in txindex metadata creation. * Tie in where txindex metadata gets set. * Start on connect stake txn. * Split out setting/flushing global stake amount nanos. * Start in on connect + disconnect stake operation. * Add first stake connect tests. * Add test for staking more. * Add connect unstake. * Test happy path unstaking connect logic. * Add connect unstake sad path tests. * Add unlock stake connect + disconnect logic. * Add happy path connect unlock stake test. * Address PR feedback. * Build out db loop for retrieving matching locked stake entries. * Add unlock stake sad path tests. * Add stake, unstake, unlock stake txn spending limits. * Fix failing CI pt 1. * Set test fork heights using helper. * Fix CI pt 2. * Add skeleton for testing staking w/ derived keys. * Start adding tests for staking with derived keys. * Add tests for unlock stake with derived key. * Fix failing stake with derived key tests. * Add tests for validating stake derived keys. * Add tests for derived key that can stake w/ any validator. * Add tests for staking with unlimited derived key. * Start writing to metamask string logic + tests. * Add unlock stake txn spending limits to metamask string logic. * Set min fee on register as validator test. * Fix derived key min fee tests. * Re-enable rollbacks in tests. * Disconnect basic transfer when disconnecting txns. * Use nondeterministic signing alg in tests. * Account for DESO when staking. * Account for DESO moving in unlock stake txn. * Move GetValidatorByPK to block_view_validator. * adding generic encode/decode deso encoder slices and single structs (#503) * check in progress on adding generic encode/decode deso encoder slices * add DecodeDeSoEncoder for decode single structs * trying stuff w/ generics * more messing w/ generics * fix validator entry decoding * revert to old decode deso encoder slice * remove comment * safe make * add test case * Address PR feedback pt1. * Address PR feedback pt1. * Encode validator total stake amount nanos in key. (#502) * Encode validator total stake amount nanos in key. * Add comment for EncodeUint256 vs EncodeOptionalUint256. * Fallback to global stake limit if scoped is exceeded. * Add moving DESO to totalInput, totalOutput. * Store stake/unstake spending limits as uint256.Ints. * use deso encoder decode (#505) * use deso encoder decode * don't use decodedesoencoder for public keys * Wrap error messages. * Address more PR feedback. * Add tests for top validators by stake. (#506) * Add tests for top validators by stake. * Split up tests. * Add TestTopValidatorsByStakeMergingDbAndUtxoView. * Fix GetLockedStakeEntriesInRange and add test. * Resolve PR comments. --------- Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * make sure donePeerChan is not nil (#512) Co-authored-by: Lazy Nina <> * Mf/unstake stakers for unregistering validator (#513) * Add block view validator. * Add unregister as validator types. * Add validator entry db utils. * Add blockchain utils. * Add utxo view getters. * Delete then set instead of updating validator entry. * Add connect register as validator logic. * Add disconnect register as validator. * Add connect and disconnect unregister as validator. * Add unregister as validator blockchain and mempool utils. * Only overwrite validatorID if new validator. * Use url parse request URI to validate domains. * Tie in register/unregister txn types. * Finish blockchain.go construct validator txns. * Add test file template. * Add tests for validations. * Add tests for registering, querying, unregistering w/o flushing. * Find top validators by stake across the db and UtxoView. * Fix tests when flushing to db. * Test updating a validator. * Store blockheights as uint64. * Fix disconnect logic for register as validator txn. * Fix disconnect tests for unregister txn. * Add stubbed voting public key fields. * Change weird UtxoOperation version to see if fixes CI. * Fix block heights in tests. * Add passing test for creating validator derived key. * Add block view stake. * Add types and byte encodings. * Fix merge conflicts more. * Add db utils. * Fix failing CI. * Add metadata validations. * Reorder keys in prefix. * Test registering + unregistering validators using derived keys. * Update TODO comments. * Clean up comments. * Add UtxoView utils. * Add mempool utils. * Tie in txindex metadata creation. * Tie in where txindex metadata gets set. * Start on connect stake txn. * Split out setting/flushing global stake amount nanos. * Start in on connect + disconnect stake operation. * Add first stake connect tests. * Add test for staking more. * Add connect unstake. * Test happy path unstaking connect logic. * Add connect unstake sad path tests. * Add unlock stake connect + disconnect logic. * Add happy path connect unlock stake test. * Address PR feedback. * Build out db loop for retrieving matching locked stake entries. * Add unlock stake sad path tests. * Add stake, unstake, unlock stake txn spending limits. * Fix failing CI pt 1. * Set test fork heights using helper. * Fix CI pt 2. * Add skeleton for testing staking w/ derived keys. * Start adding tests for staking with derived keys. * Add tests for unlock stake with derived key. * Fix failing stake with derived key tests. * Add tests for validating stake derived keys. * Add tests for derived key that can stake w/ any validator. * Add tests for staking with unlimited derived key. * Start writing to metamask string logic + tests. * Add unlock stake txn spending limits to metamask string logic. * Set min fee on register as validator test. * Fix derived key min fee tests. * Re-enable rollbacks in tests. * Disconnect basic transfer when disconnecting txns. * Use nondeterministic signing alg in tests. * Account for DESO when staking. * Account for DESO moving in unlock stake txn. * Move GetValidatorByPK to block_view_validator. * adding generic encode/decode deso encoder slices and single structs (#503) * check in progress on adding generic encode/decode deso encoder slices * add DecodeDeSoEncoder for decode single structs * trying stuff w/ generics * more messing w/ generics * fix validator entry decoding * revert to old decode deso encoder slice * remove comment * safe make * add test case * Address PR feedback pt1. * Address PR feedback pt1. * Encode validator total stake amount nanos in key. (#502) * Encode validator total stake amount nanos in key. * Add comment for EncodeUint256 vs EncodeOptionalUint256. * Fallback to global stake limit if scoped is exceeded. * Add moving DESO to totalInput, totalOutput. * Store stake/unstake spending limits as uint256.Ints. * use deso encoder decode (#505) * use deso encoder decode * don't use decodedesoencoder for public keys * Wrap error messages. * Address more PR feedback. * Add tests for top validators by stake. (#506) * Add tests for top validators by stake. * Split up tests. * Add TestTopValidatorsByStakeMergingDbAndUtxoView. * Fix GetLockedStakeEntriesInRange and add test. * Resolve PR comments. * Unstake stakers for unregistering validator. * Dont allow updating DisableDelegatedStakers with existing DelegatedStakers. * Move DisableDelegatedStake validation to metadata validation util. * Add test for unregistering validator unstaking stakers. * Resolve PR feedback. --------- Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * block height gate balance snapshot comparison (#516) * [stable] Release 3.4.1 * Re-enable force-checksum (#519) * Add epoch data type. (#514) * Add epoch data type. * Update func naming. * Fix nil error. * Add prefix encoder db util. * Rename field to FinalBlockHeight. * Avoid snapshotting issue for now. * Update ancestral record before setting in db. * Flush the CurrentEpochEntry to the db like we do for other entries. * Return error instead of logging. * Incorporate epoch number into PoS txns. (#515) * make sure donePeerChan is not nil (#512) Co-authored-by: Lazy Nina <> * Add epoch data type. * Update func naming. * Fix nil error. * Add prefix encoder db util. * Rename field to FinalBlockHeight. * Avoid snapshotting issue for now. * Update ancestral record before setting in db. * Incorporate epoch number into PoS txns. * block height gate balance snapshot comparison (#516) * [stable] Release 3.4.1 * Re-enable force-checksum (#519) --------- Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> * Mf/validate stake lockup period (#517) * Add epoch data type. * Update func naming. * Fix nil error. * Add prefix encoder db util. * Rename field to FinalBlockHeight. * Avoid snapshotting issue for now. * Update ancestral record before setting in db. * Incorporate epoch number into PoS txns. * Validate stake lockup period. * Validate stake lockup period. * Clean up test file changes. * Set stake lockup period to 3 epochs. * Resolve PR feedback. * Remove accidental newline. * Add jailed status to validator entries. (#518) * Add epoch data type. * Update func naming. * Fix nil error. * Add prefix encoder db util. * Rename field to FinalBlockHeight. * Avoid snapshotting issue for now. * Update ancestral record before setting in db. * Incorporate epoch number into PoS txns. * Validate stake lockup period. * Validate stake lockup period. * Clean up test file changes. * Add jailed status to validator entries. * Set stake lockup period to 3 epochs. * Set LastActiveEpochNumber to CurrentEpochNumber. * Add ValidatorEntry.JailedAtEpochNumber field. * Resolve PR feedback. * Mf/add unjail validator txn type (#520) * Add epoch data type. * Update func naming. * Fix nil error. * Add prefix encoder db util. * Rename field to FinalBlockHeight. * Avoid snapshotting issue for now. * Update ancestral record before setting in db. * Incorporate epoch number into PoS txns. * Validate stake lockup period. * Validate stake lockup period. * Clean up test file changes. * Add jailed status to validator entries. * Set stake lockup period to 3 epochs. * Set LastActiveEpochNumber to CurrentEpochNumber. * Add unjail validator txn type. * Add ValidatorEntry.JailedAtEpochNumber field. * Validate sufficient num epochs have passed to unjail. * Resolve merge conflicts more. * Start adding tests for unjail connect logic. * Add unjail validator tests. * Retrigger buildkite ci. * Address PR feedback. * Mf/convert validator voting keys to bls keys (#522) * Add BLS signature utils. * Get tests passing. * Update test dockerfile to install relic. * Add relic + no relic files. * Add more private key, public key, signature utils. * Remove types_ filename prefix. * Install go v1.20 in the test.Dockerfile. * Dont apk add install go. * Dont re-install flow-go. * Add utils to aggregate + verify aggregate signatures. * Add no relic version of aggregate BLS sig funcs. * Conver validator voting keys to BLS keys. * Add copy funcs. * Fix failing stake tests. * Validate validator voting signature block height. * Retrigger CI. * Update no relic func signatures. * Add better edge-case handling for BLS types. * Merge upstream branch. Resolve conflicts. * Move BLS utils to own package. * Clean up naming conventions. * Reference BLS utils from package. * Expose NewPrivateKey(). * Incorporate new BLS package in tests. * Dont error if parsing empty string or bytes. * Try non-recursive go test syntax. * Retrigger CI. * Add bls package to Dockerfile. * Test Copy() func. * Add encoder/decoder wrappers. * Increase voting signature window from 60 to 300. --------- Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.