feat(script): Add binary for finding references to closed issues #6347

arya2 · 2023-03-16T17:56:53Z

Motivation

We want a full list of comments in the repo referencing issues that are closed.

Closes #6278.

Solution

Add a search-issue-refs binary to zebra-utils
- Search all .rs, .yml, .yaml, and .toml files in the local directory for possible issue refs
- Filter out any possible issue refs that don't have closed issues in Zebra
- Print out a list of closed issues and their locations

Review

Anyone can review.

Reviewer Checklist

Will the PR name make sense to users?
- Does it need extra CHANGELOG info? (new features, breaking changes, large changes)
Are the PR labels correct?
Does the code do what the ticket and PR says?
- Does it change concurrent code, unsafe code, or consensus rules?
How do you know it works? Does it have tests?

codecov · 2023-03-16T21:43:18Z

Codecov Report

Merging #6347 (f2eee16) into main (a97a975) will decrease coverage by 0.04%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6347      +/-   ##
==========================================
- Coverage   77.78%   77.74%   -0.04%     
==========================================
  Files         304      304              
  Lines       39528    39585      +57     
==========================================
+ Hits        30745    30775      +30     
- Misses       8783     8810      +27

zebra-utils/src/bin/search-issue-refs/main.rs

upbqdn

This is awesome.

I noticed a potential bug in the regex and left some comments.

zebra-utils/src/bin/search-issue-refs/main.rs

mpguerra · 2023-03-23T10:26:03Z

@upbqdn are you able to review this one? Otherwise @oxarbitrage can you please take over this review?

oxarbitrage

This looks really good, i made a few optional comments.

There are a few things that i think we have to consider:

We now need to use it to fix some/all of the findings (there are currently 184). Should we open a ticket ?
I am a bit concerned that we do more rust utilities in general, which will mean more dependencies, more CI, etc. We hide them behind rust features but i think that is still attached to the main project. Should we consider making utilities outside of the zebra repo or encourage to do what we did here in the future? Any opinions?

zebra-utils/src/bin/search-issue-refs/main.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

zebra-utils/src/bin/search-issue-refs/main.rs

mpguerra · 2023-03-28T09:36:02Z

This looks really good, i made a few optional comments.

There are a few things that i think we have to consider:
* We now need to use it to fix some/all of the findings (there are currently 184). Should we open a ticket ?

There's a tracking issue #6281 to deal with/list these already

mpguerra · 2023-03-31T11:01:17Z

Edit: @arya2 can you please add the output of this script as new comment on #6281 so someone can go through them and create issues if necessary?

arya2 · 2023-03-31T19:09:12Z

Edit: @arya2 can you please add the output of this script as new comment on #6281 so someone can go through them and create issues if necessary?

Yes! (script lightly modified for permalinks and markdown output):

➜ GITHUB_TOKEN=XXX cargo run --package zebra-utils --bin search-issue-refs --all-features --features search-issue-refs

Searching files in this repo with a ".rs", ".yml", ".yaml", or ".toml" file extension for issue references..

Found 209 possible references to 104 issues, checking statuses on Github..

{results - see below}

Confirmed 179 references to 91 closed issues.

References to Closed Issues

Add a timeout to the PeerServer event loop. #65

zebra/zebra-network/src/protocol/internal/response_status.rs

Line 52 in 5db2243

    
           // Based on Result::map from https://doc.rust-lang.org/src/core/result.rs.html#765

Dedupe Verification error types (Block, Chain) #1186

zebra/zebra-consensus/src/error.rs

Line 156 in 5db2243

// temporary error type until #1186 is fixed

see more:

Design and implement graceful shutdown for Zebra #1678

zebra/zebra-network/src/peer_set/set.rs

Line 423 in 5db2243

// TODO: implement graceful shutdown for InventoryRegistry (#1678)

Improve fake_v5_transaction_with_orchard_actions_has_inputs_and_outputs test #2409

zebra/zebra-consensus/src/transaction/tests.rs

Line 93 in 5db2243

    
           // TODO: Avoid new calls to `insert_fake_orchard_shielded_data` for each check #2409.

Clearly identify 32-bit and 64-bit times in serialization and proptests #2171

zebra/zebra-network/src/protocol/external/message.rs

Line 325 in 5db2243

/// TODO: replace with a custom DateTime64 type (#2171)

Move debug stop at height to the zebrad start main loop #3442

zebra/zebra-state/src/service/finalized_state.rs

Line 117 in 5db2243

// TODO: move debug_stop_at_height into a task in the start command (#3442)

zebra/zebra-state/src/service/finalized_state.rs

Line 331 in 5db2243

// TODO: move the stop height check to the syncer (#3442)

zebra/zebra-state/src/service/finalized_state.rs

Line 441 in 5db2243

/// TODO: move the stop height check to the syncer (#3442)

zebra/zebra-state/src/service/finalized_state/disk_db.rs

Line 660 in 5db2243

    
           /// TODO: make private after the stop height check has moved to the syncer (#3442)

Perform listener DNS lookups asynchronously, with a timeout #1631

zebra/zebra-network/src/config.rs

Line 328 in 5db2243

// TODO: perform listener DNS lookups asynchronously with a timeout (#1631)

Move syncer block download retries inside the peer set #3235

zebra/zebra-network/src/policies.rs

Line 39 in 5db2243

// so we always choose different peers (#3235)

Don't consider the "grace period" when decrypting coinbase outputs #3027

zebra/zebra-consensus/src/transaction/check.rs

Line 319 in 5db2243

/// <https://github.com/ZcashFoundation/zebra/issues/3027>

Remove redundant watch values once our Tokio dependency is updated #2573

zebra/zebrad/src/components/sync/recent_sync_lengths.rs

Line 21 in 5db2243

    
           // TODO: Replace with calls to `watch::Sender::borrow` once Tokio is updated to 1.0.0 (#2573)

Test if the block downloader and verifier error checking is correct #2909

zebra/zebrad/src/components/sync.rs

Line 1156 in 5db2243

// https://github.com/ZcashFoundation/zebra/issues/2909

Hide debug-only copy-state command from users #3279

zebra/zebrad/src/commands.rs

Line 32 in 5db2243

// TODO: hide this command from users in release builds (#3279)

Choose between backpressure and load shedding for each service #1618

zebra/zebrad/src/components/inbound.rs

Line 301 in 5db2243

// "load shed directly" pattern from #1618.

zebra-chain: Port array-wrapper types to const generics #2042

zebra/zebra-chain/src/orchard/note/ciphertexts.rs

Line 14 in 5db2243

    
           // TODO: use const generics https://github.com/ZcashFoundation/zebra/issues/2042

Combine proptests and transcripts to perform top-level integration tests #1021

zebra/zebra-chain/src/block/header.rs

Line 89 in 5db2243

    
           /// See <https://github.com/ZcashFoundation/zebra/issues/1021> for more details.

zebra/zebra-chain/src/block/header.rs

Line 104 in 5db2243

    
           /// See <https://github.com/ZcashFoundation/zebra/issues/1021> for more details.

Zebra should support separate local bind and external advertise addresses #1890

zebra/zebra-network/src/config.rs

Line 46 in 5db2243

    
           /// - [Advertising a different external IP address #1890](https://github.com/ZcashFoundation/zebra/issues/1890), or

seed command #83

zebra/zebra-chain/src/block/serialize.rs

Line 134 in 5db2243

// we start generating our own blocks (see #483).

zebra/zebra-chain/src/transaction/serialize.rs

Line 472 in 5db2243

// transactions (see #483).

Speed up Zebra's CI #2995

zebra/zebrad/Cargo.toml

Line 91 in 5db2243

    
           # TODO: replace with environmental variables that skip the tests when not set (part of #2995)

Fix windows state path log message and acceptance test #1654

zebra/zebrad/tests/acceptance.rs

Line 1936 in 5db2243

// Windows problems with this match will be worked on at #1654

Revert note commitment and history trees when forking non-finalized chains #4794

zebra/zebra-rpc/src/methods/get_block_template_rpcs.rs

Line 114 in 5db2243

/// until bug #4794 is fixed.)

Limit fanouts to the number of active peers, Credit: Equilibrium #2214

zebra/zebrad/src/components/sync.rs

Line 644 in 5db2243

    
           // TODO: move fanouts into the PeerSet, so we always choose different peers (#2214)

zebra/zebrad/src/components/sync.rs

Line 777 in 5db2243

    
           // TODO: move fanouts into the PeerSet, so we always choose different peers (#2214)

zebra/zebrad/src/components/mempool/crawler.rs

Line 213 in 5db2243

    
           // TODO: move fanouts into the PeerSet, so we always choose different peers (#2214)

zebra/zebra-network/src/constants.rs

Line 189 in 5db2243

    
           /// TODO: Restore the fanout to 3, once fanouts are limited to the number of ready peers (#2214)

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 277 in 5db2243

    
           // TODO: move fanouts into the PeerSet, so we always choose different peers (#2214)

Validate Transaction::V5 Halo2 proofs #2105

zebra/zebra-consensus/src/transaction.rs

Line 716 in 5db2243

// - verify orchard shielded pool (ZIP-224) (#2105)

Test if checkpoint verifier and state service are correctly reset on block commit errors #2654

zebra/zebra-state/src/service.rs

Line 171 in 5db2243

// TODO: add tests for finalized and non-finalized resets (#2654)

Zebra should auto-discover its own external IP address #1893

zebra/zebra-network/src/config.rs

Line 47 in 5db2243

    
           /// - [Auto-discovering its own external IP address #1893](https://github.com/ZcashFoundation/zebra/issues/1893).

zebra/zebra-network/src/peer/handshake.rs

Line 636 in 5db2243

// TODO: detect external address (#1893)

Propagate errors from the node's internal Service to the PeerServer. #50

zebra/zebra-network/src/protocol/internal/response_status.rs

Line 62 in 5db2243

    
           // Based on Result::map_err from https://doc.rust-lang.org/src/core/result.rs.html#850

Security: Limit the number of inbound peer connections in the listener task #1851

zebra/zebra-network/src/peer_set/limit.rs

Line 124 in 5db2243

    
           // TODO: This channel will be bounded by the connection limit (#1850, #1851, #2902).

Turn the CandidateSet and AddressBook into tower Services #1976

zebra/zebra-network/src/address_book_updater.rs

Line 84 in 5db2243

// to avoid deadlocks (see #1976)

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 128 in 5db2243

    
           //              so all operations are performed on a blocking thread (see #1976).

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 333 in 5db2243

// to avoid deadlocks (see #1976).

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 381 in 5db2243

// TODO: only mark the peer as AttemptPending when it is actually used (#1976)

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 392 in 5db2243

    
           // Correctness: Spawn address book accesses on a blocking thread, to avoid deadlocks (see #1976).

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 412 in 5db2243

// to avoid deadlocks (see #1976).

Import and test against Sinsemilla test vectors from zcash-hackworks #2079

zebra/zebra-chain/src/orchard/sinsemilla.rs

Line 163 in 5db2243

// https://github.com/ZcashFoundation/zebra/issues/2079

getrawtransaction JSON-RPC method #3145

zebra/zebra-state/src/service/finalized_state/zebra_db/block/tests/snapshot.rs

Line 133 in 5db2243

// TODO: after #3145, replace with:

Test Zebra reject message handling #4633

zebra/zebra-network/src/protocol/external/message.rs

Line 400 in 5db2243

    
           // TODO: add tests for Error conversion and Reject message serialization (#4633)

Allow MockServices to return readiness errors #3204

zebra/zebra-network/src/peer_set/unready_service/tests/vectors.rs

Line 3 in 5db2243

//! TODO: test that inner service errors are handled correctly (#3204)

Support batch verification of JoinSplit Groth16 proofs #3127

zebra/zebra-consensus/src/primitives/groth16.rs

Line 144 in 5db2243

/// <https://github.com/ZcashFoundation/zebra/issues/3127>

Use valuable::Enumerable to iterate through network upgrades #1974

zebra/zebra-state/src/service/arbitrary.rs

Line 129 in 5db2243

// TODO: use the latest network upgrade (#1974)

zebra/zebra-chain/src/block/arbitrary.rs

Line 292 in 5db2243

// TODO: dynamically select any future network upgrade (#1974)

zebra/zebra-chain/src/parameters/arbitrary.rs

Line 17 in 5db2243

// TODO: add future network upgrades (#1974)

Upgrade Zebra to Rust 2021 edition #2709

zebra/zebra-network/src/peer_set/set.rs

Line 508 in 5db2243

// TODO: Remove when the code base migrates to Rust 2021 edition (#2709).

Zebra should limit the number of addresses it uses from a single Addrs response, to avoid address book takeover #1869

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 452 in 5db2243

// response (#1869)

Security: Send notfound inv items to the inv collector, Credit: Equilibrium #2156

zebra/zebra-network/src/peer/handshake.rs

Line 1108 in 5db2243

// responses to the inv collector. (#2156, #1768)

zebra/zebra-network/src/peer_set/set.rs

Line 737 in 5db2243

// TODO: reduce this log level after testing #2156 and #2726

Avoid an AwaitUtxo race condition when switching to the non-finalized state #5126

zebra/zebra-state/src/service.rs

Line 1028 in 5db2243

// See #5126 for details.

Treat TryRecvError::Closed in Inbound::poll_ready as a fatal error #1655

zebra/zebra-network/src/peer/connection.rs

Line 1206 in 5db2243

    
           // TODO: treat `TryRecvError::Closed` in `Inbound::poll_ready` as a fatal error (#1655)

Return errors from Connection methods to the Client #4734

zebra/zebra-network/src/peer/client/tests.rs

Line 379 in 5db2243

// TODO: In the context of #4734:

Cleanup and fill out TransactionHash #38

zebra/zebra-state/src/service/check/utxo.rs

Line 238 in 5db2243

// TODO: check coinbase transaction remaining value (#338, #1162)

GCP: Do not allow digests without tags in Artifact Registry #5933

zebra/.github/workflows/delete-gcp-resources.yml

Line 208 in 5db2243

    
           continue-on-error: true # TODO: remove after fixig https://github.com/ZcashFoundation/zebra/issues/5933

cleanup(cryptography): Remove unused shielded key and address code #5476

zebra/zebra-chain/src/sprout/keys.rs

Line 3 in 5db2243

//! Unused key types are not implemented, see PR #5476.

zebra/zebra-chain/src/orchard/keys.rs

Line 3 in 5db2243

//! Unused key types are not implemented, see PR #5476.

zebra/zebra-chain/src/sapling/keys.rs

Line 3 in 5db2243

//! Unused key types are not implemented, see PR #5476.

Enforce zebra_test::command timeouts regardless of command output #1140

zebra/zebra-test/src/command.rs

Line 222 in 5db2243

/// Only checked when the command outputs each new line (#1140).

zebra/zebra-test/src/command.rs

Line 809 in 5db2243

/// the child (#1140).

zebra/zebra-test/tests/command.rs

Line 74 in 5db2243

// TODO: create a similar test that pauses after output (#1140)

zebra/zebra-test/tests/command.rs

Line 101 in 5db2243

/// This test fails due to bugs in TestDirExt, see #1140 for details.

zebra/zebra-test/tests/command.rs

Line 130 in 5db2243

// TODO: create a similar test that pauses after output (#1140)

zebra/zebra-test/tests/command.rs

Line 157 in 5db2243

/// This test fails due to bugs in TestDirExt, see #1140 for details.

zebra/zebra-test/tests/command.rs

Line 445 in 5db2243

/// TODO: test the failure regex on timeouts with no output (#1140)

Cache unsolicited Peers responses until needed #3110

zebra/zebra-network/src/constants.rs

Line 191 in 5db2243

    
           /// In #3110, we changed the fanout to 1, to make sure we actually use cached address responses.

Do we need to validate nSpendsSapling, nOutputsSapling, and nActionsOrchard limits in V5 transactions? #2379

zebra/zebra-consensus/src/transaction.rs

Line 717 in 5db2243

// - shielded input and output limits? (#2379)

Stop double-spends by checking nullifiers in the finalized state #2230

zebra/zebra-state/src/service/finalized_state/tests/prop.rs

Line 38 in 5db2243

// TODO: check that the nullifiers were correctly inserted (#2230)

Fix slowness when syncing near the tip #3375

zebra/zebrad/src/components/sync/progress.rs

Line 32 in 5db2243

    
           /// TODO: replace with `MAX_CLOSE_TO_TIP_BLOCKS` after fixing slow syncing near tip (#3375)

zebra/zebrad/src/components/sync/progress.rs

Line 91 in 5db2243

// after fixing slow syncing near tip (#3375)

zebra/zebrad/src/components/sync/progress.rs

Line 173 in 5db2243

// TODO: warn after fixing slow syncing near tip (#3375)

Use DateTime32 for block times and lock times #2211

zebra/zebra-chain/src/transaction/lock_time.rs

Line 74 in 5db2243

// TODO: replace Utc.timestamp with DateTime32 (#2211)

zebra/zebra-chain/src/transaction/lock_time.rs

Line 88 in 5db2243

// TODO: replace Utc.timestamp with DateTime32 (#2211)

Refactor RPC methods to use HexData and an error constructor #5548

zebra/zebra-rpc/src/methods.rs

Line 377 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 480 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 508 in 5db2243

// use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 564 in 5db2243

// - use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 754 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 766 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 841 in 5db2243

// use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 923 in 5db2243

// - use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 1041 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods.rs

Line 1106 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods/get_block_template_rpcs.rs

Line 395 in 5db2243

// TODO: use a generic error constructor (#5548)

zebra/zebra-rpc/src/methods/get_block_template_rpcs.rs

Line 430 in 5db2243

// TODO: use a generic error constructor (#5548)

Fully anonymise timestamps in version messages when broadcasting transactions #3300

zebra/zebra-network/src/isolated/tests/vectors.rs

Line 152 in 5db2243

    
           // SECURITY TODO: check if the timestamp field can be zeroed, to remove another distinguisher (#3300)

zebra/zebra-network/src/isolated/tests/vectors.rs

Line 168 in 5db2243

// SECURITY TODO: should this just be zeroed anyway? (#3300)

zebra/zebra-network/src/isolated.rs

Line 36 in 5db2243

    
           /// SECURITY TODO: check if the timestamp field can be zeroed, to remove another distinguisher (#3300)

Process inv with mixed item types #1768

zebra/zebra-network/src/peer/handshake.rs

Line 1108 in 5db2243

// responses to the inv collector. (#2156, #1768)

Make MetaAddr.addr a private field #2357

zebra/zebra-network/src/meta_addr.rs

Line 141 in 5db2243

// canonical SocketAddr (#2357)

Deploy more Zebra or zcashd instances on testnet #1222

zebra/zebrad/tests/acceptance.rs

Line 1009 in 5db2243

// but they were removed because the testnet is unreliable (#1222).

Use a bellman::Proof instead of a byte array in Groth16Proof #3179

zebra/zebra-consensus/src/primitives/groth16.rs

Line 349 in 5db2243

// This validates the 𝜋_{ZKJoinSplit} element. In #3179 we plan to validate

zebra/zebra-chain/src/sprout/joinsplit.rs

Line 174 in 5db2243

// [`groth16::Item::try_from`]. In #3179 we plan to validate here instead.

zebra/zebra-chain/src/sapling/output.rs

Line 175 in 5db2243

// [`groth16::Item::try_from`]. In #3179 we plan to validate here instead.

zebra/zebra-chain/src/sapling/spend.rs

Line 224 in 5db2243

// [`groth16::Item::try_from`]. In #3179 we plan to validate here instead.

zebra/zebra-chain/src/transaction/serialize.rs

Line 233 in 5db2243

// [`groth16::Item::try_from`]. In #3179 we plan to validate here instead.

zebra/zebra-chain/src/transaction/serialize.rs

Line 262 in 5db2243

// [`groth16::Item::try_from`]. In #3179 we plan to validate here instead.

Use Poll::map in the download streams #2693

zebra/zebrad/src/components/sync/downloads.rs

Line 240 in 5db2243

// TODO: this would be cleaner with poll_map (#2693)

zebra/zebrad/src/components/inbound/downloads.rs

Line 141 in 5db2243

// TODO: this would be cleaner with poll_map (#2693)

zebra/zebrad/src/components/mempool/downloads.rs

Line 186 in 5db2243

// TODO: this would be cleaner with poll_map (#2693)

Make sure lists are unique in zebra-network Requests and Responses #2244

zebra/zebrad/src/components/inbound/tests/real_peer_set.rs

Line 240 in 5db2243

// In #2244 we will fix this by replacing response Vecs with HashSets.

zebra/zebrad/src/components/inbound/tests/real_peer_set.rs

Line 276 in 5db2243

// In #2244 we will fix this by replacing response Vecs with HashSets.

zebra/zebrad/src/components/inbound/tests/real_peer_set.rs

Line 392 in 5db2243

// In #2244 we will fix this by replacing response Vecs with HashSets.

zebra/zebra-network/src/protocol/internal/response.rs

Line 35 in 5db2243

    
           // TODO: make this into a HashMap<SocketAddr, MetaAddr> - a unique list of peer addresses (#2244)

zebra/zebra-network/src/protocol/internal/response.rs

Line 42 in 5db2243

// TODO: make this into an IndexMap - an ordered unique list of hashes (#2244)

zebra/zebra-network/src/protocol/internal/response.rs

Line 49 in 5db2243

// TODO: make this into an IndexMap - an ordered unique list of headers (#2244)

zebra/zebra-network/src/protocol/internal/response.rs

Line 59 in 5db2243

// TODO: make this into a HashSet - a unique list (#2244)

zebra/zebra-network/src/protocol/internal/response.rs

Line 69 in 5db2243

    
           // TODO: make this into a HashMap<block::Hash, InventoryResponse<Arc<Block>, ()>> - a unique list (#2244)

zebra/zebra-network/src/protocol/internal/response.rs

Line 76 in 5db2243

    
           // TODO: make this into a HashMap<UnminedTxId, InventoryResponse<UnminedTx, ()>> - a unique list (#2244)

zebra/zebra-network/src/protocol/internal/request.rs

Line 116 in 5db2243

// TODO: make this into an IndexMap - an ordered unique list of hashes (#2244)

zebra/zebra-network/src/protocol/internal/request.rs

Line 134 in 5db2243

// TODO: make this into an IndexMap - an ordered unique list of hashes (#2244)

TestChild::is_running can return true after a panic log #1781

zebra/zebrad/tests/acceptance.rs

Line 2004 in 5db2243

// See #1781.

zebra/zebra-test/src/command.rs

Line 483 in 5db2243

/// processes that have panicked. See #1781.

zebra/zebra-test/src/command.rs

Line 917 in 5db2243

/// have panicked. See #1781.

Add mempool metrics related to memory usage #2869

zebra/zebrad/src/components/mempool/storage.rs

Line 646 in 5db2243

// TODO: make it more accurate #2869

Move finalized state database and filesystem operations to blocking threads to avoid hangs #2188

zebra/zebra-state/src/service/finalized_state/disk_db.rs

Line 50 in 5db2243

    
           /// This bug might be fixed by moving database operations to blocking threads (#2188),

zebra/zebra-state/src/service/finalized_state/disk_db.rs

Line 438 in 5db2243

// TODO: move opening the database to a blocking thread (#2188)

zebra/zebra-state/src/service/finalized_state/disk_db.rs

Line 661 in 5db2243

/// move shutting down the database to a blocking thread (#2188)

Add a helper method for reading fixed-size byte arrays #10

zebra/zebra-chain/src/work/difficulty.rs

Line 295 in 5db2243

.field(&format_args!("{:#010x}", self.0))

Zebra should tell users about disk space requirements #1623

zebra/zebra-state/src/service/finalized_state/disk_db.rs

Line 459 in 5db2243

// TODO: provide a different hint if the disk is full, see #1623

Add hashAuthDataRoot to Block network messages, and semantically verify it #2336

zebra/zebra-state/src/service/check/utxo.rs

Line 68 in 5db2243

// during contextual validation (#2336).

Add peer count data to TimestampData::update trace #66

zebra/zebra-chain/src/chain_tip/network_chain_tip_height_estimator.rs

Line 117 in 5db2243

// (See https://docs.rs/time/0.1.44/src/time/duration.rs.html#166-173)

Add utxo_by_transparent_addr_loc index to the finalized state #3953

zebra/zebra-state/src/service/finalized_state/disk_format/transparent.rs

Line 544 in 5db2243

// TODO: serialize the index into a smaller number of bytes (#3953)

zebra/zebra-state/src/service/finalized_state/disk_format/transparent.rs

Line 545 in 5db2243

// serialize the index in big-endian order (#3953)

Security: Limit reconnection rate to individual peers #2275

zebra/zebra-network/src/peer_set/candidate_set/tests/prop.rs

Line 56 in 5db2243

/// TODO: after PR #2275 merges, add a similar proptest,

Implicit conversion of the minimum difficulty threshold to the compact representation #1277

zebra/zebra-chain/src/work/difficulty.rs

Line 400 in 5db2243

// ZIP 208 don't specify this conversion either.) See #1277 for details.

Activate Mempool storage based on a list of recent syncer obtain/extend lengths #2592

zebra/zebrad/src/components/sync/recent_sync_lengths.rs

Line 64 in 5db2243

// https://github.com/ZcashFoundation/zebra/issues/2592#issuecomment-897304684

getblocks, headers, getheaders, tx, getdata, notfound #44

zebra/zebra-network/src/peer_set/inventory_registry.rs

Line 180 in 5db2243

    
           // Based on Option::map from https://doc.rust-lang.org/src/core/option.rs.html#844

Security: Running Zebra nodes should eventually stop trying to contact peers that always fail #1865

zebra/zebra-network/src/peer_set/candidate_set.rs

Line 448 in 5db2243

// - Zebra should ignore peers that are older than 3 weeks (part of #1865)

Re-enable sync_large_checkpoints_testnet test #1791

zebra/zebrad/tests/acceptance.rs

Line 1010 in 5db2243

// We should re-add them after we have more testnet instances (#1791).

feat(rpc): Log unrecognized RPC requests #3860

zebra/zebrad/tests/common/failure_messages.rs

Line 31 in 5db2243

// Missing RPCs in zebrad logs (this log is from PR #3860)

Security: Limit the number of outbound peer connections when dialing new peers #1850

zebra/zebra-network/src/peer_set/limit.rs

Line 124 in 5db2243

    
           // TODO: This channel will be bounded by the connection limit (#1850, #1851, #2902).

Restore default target output path on Windows CI builds #3005

zebra/.github/workflows/continous-integration-os.yml

Line 105 in 5db2243

# or remove this workaround once the build is more efficient (#3005).

zebra/.github/workflows/continous-integration-os.yml

Line 116 in 5db2243

# TODO: Remove this workaround once the build is more efficient (#3005).

Detect when an older state version is present, and delete it #1213

zebra/zebra-state/src/config.rs

Line 32 in 5db2243

    
           /// Old state versions are [not automatically deleted](https://github.com/ZcashFoundation/zebra/issues/1213).

Add hints to port conflict and lock file panics #1535

zebra/zebrad/tests/acceptance.rs

Line 1804 in 5db2243

    
           /// The second node will panic with the Zcash listener conflict hint added in #1535.

zebra/zebrad/tests/acceptance.rs

Line 1832 in 5db2243

/// conflict hint added in #1535.

zebra/zebrad/tests/acceptance.rs

Line 1861 in 5db2243

/// conflict hint added in #1535.

zebra/zebrad/tests/acceptance.rs

Line 1926 in 5db2243

    
           /// The second node will panic with the Zcash state conflict hint added in #1535.

Store transactions in a separate database index #3151

zebra/zebra-state/src/service/finalized_state/zebra_db/block/tests/snapshot.rs

Line 256 in 5db2243

// TODO: split out block snapshots into their own function (#3151)

zebra/zebra-state/src/service/finalized_state/zebra_db/block/tests/snapshot.rs

Line 319 in 5db2243

// TODO: split out transaction snapshots into their own function (#3151)

zebra/zebra-state/src/service/finalized_state/zebra_db/block/tests/snapshot.rs

Line 353 in 5db2243

// TODO: snapshot TransactionLocations without Some (#3151)

Off-by-one error in testnet minimum difficulty block gap specification #1276

zebra/zebra-chain/src/work/difficulty/tests/vectors.rs

Line 460 in 5db2243

// threshold, but ZIP-205 and ZIP-208 are ambiguous. See bug #1276.

Make Zebra send notfound, and use received notfound to finish requests #2726

zebra/zebra-network/src/peer_set/set.rs

Line 737 in 5db2243

// TODO: reduce this log level after testing #2156 and #2726

Un-reject transactions which depend on newly created outputs #2694

zebra/zebrad/src/components/mempool/downloads.rs

Line 75 in 5db2243

/// [#2694](https://github.com/ZcashFoundation/zebra/issues/2694)

Safe Preallocation for (most) Vec<T> #1880

zebra/zebrad/src/components/sync.rs

Line 72 in 5db2243

    
           /// [`super::inbound::downloads::MAX_INBOUND_CONCURRENCY`] and #1880 for details.

zebra/zebrad/src/components/inbound/downloads.rs

Line 44 in 5db2243

/// (See #1880 for more details.)

zebra/zebrad/src/components/mempool/downloads.rs

Line 90 in 5db2243

/// (See #1880 for more details.)

Refactor zebra-network Bitcoin to Zebra protocol translation layer #1515

zebra/zebra-network/src/peer/connection.rs

Line 8 in 5db2243

    
           //! It should be refactored into a cleaner set of request/response pairs (#1515).

Fix an incorrect assertion when the block locator is at the tip #2789

zebra/zebra-state/src/service/tests.rs

Line 164 in 5db2243

// tests bug #2789

Upgrade to latest tokio #2200

zebra/zebra-state/src/service/chain_tip.rs

Line 612 in 5db2243

// once we upgrade to tokio 1.0 (#2200)

Add a size estimator to the message encoder and serializers #1774

zebra/zebra-network/src/protocol/external/codec.rs

Line 190 in 5db2243

/// See #1774.

NU5 Tracking Epic #1841

zebra/zebra-state/src/tests/setup.rs

Line 41 in 5db2243

    
           // TODO: This argument can be removed and just use Nu5 after we have an activation height #1841

zebra/zebra-state/src/tests/setup.rs

Line 55 in 5db2243

// TODO: update this to Nu5 after we have a height #1841

Make sure Zebra uses poll_ready and Buffer reservations correctly #1593

zebra/zebrad/src/components/inbound.rs

Line 305 in 5db2243

// See #1593 for details.

zebra/zebra-consensus/src/checkpoint.rs

Line 990 in 5db2243

// `poll_ready` has a corresponding `call`. See #1593.

Security: Use canonical SocketAddrs to avoid duplicate peer connections, Feature: Send local listener to peers #2276

zebra/zebra-network/src/meta_addr/arbitrary.rs

Line 100 in 5db2243

/// fields. (After PR #2276 merges.)

Ignore and sanitize untrusted MetaAddr peer services #2324

zebra/zebra-network/src/meta_addr.rs

Line 160 in 5db2243

// split gossiped and handshake services? (#2324)

zebra/zebra-network/src/meta_addr.rs

Line 617 in 5db2243

// consider sanitizing untrusted services to NODE_NETWORK (#2324)

zebra/zebra-network/src/meta_addr.rs

Line 681 in 5db2243

// TODO: create a "services implemented by Zebra" constant (#2324)

zebra/zebra-network/src/meta_addr.rs

Line 684 in 5db2243

// TODO: split untrusted and direct services (#2324)

zebra/zebra-network/src/meta_addr.rs

Line 933 in 5db2243

// TODO: order services by usefulness, not bit pattern values (#2324)

Limit the initial crawler demand for outbound peers, based on seed peer connections #2902

zebra/zebra-network/src/peer_set/limit.rs

Line 124 in 5db2243

    
           // TODO: This channel will be bounded by the connection limit (#1850, #1851, #2902).

Validate coinbase miner subsidy and miner fees #1162

zebra/zebra-state/src/service/check/utxo.rs

Line 238 in 5db2243

// TODO: check coinbase transaction remaining value (#338, #1162)

Fix tasks spawned by tower_batch::Batch only being alive during one test execution #2390

zebra/zebra-consensus/src/primitives/groth16/tests.rs

Line 73 in 5db2243

    
           // Use separate verifiers so shared batch tasks aren't killed when the test ends (#2390)

zebra/zebra-consensus/src/primitives/groth16/tests.rs

Line 178 in 5db2243

    
           // Use separate verifiers so shared batch tasks aren't killed when the test ends (#2390).

zebra/zebra-consensus/src/primitives/groth16/tests.rs

Line 440 in 5db2243

    
           // Use separate verifiers so shared batch tasks aren't killed when the test ends (#2390).

zebra/zebra-consensus/src/primitives/halo2/tests.rs

Line 151 in 5db2243

    
           // Use separate verifier so shared batch tasks aren't killed when the test ends (#2390)

zebra/zebra-consensus/src/primitives/halo2/tests.rs

Line 218 in 5db2243

    
           // Use separate verifier so shared batch tasks aren't killed when the test ends (#2390)

teor2345 · 2023-04-03T00:28:38Z

Note that some of these tasks have been closed as "wontfix", which is different from actually being fixed.

We might want to keep references to "wontfix" tasks in the code for context, but maybe we could change "TODO" to "WONTFIX".

arya2 requested a review from a team as a code owner March 16, 2023 17:56

arya2 self-assigned this Mar 16, 2023

arya2 requested review from upbqdn and removed request for a team March 16, 2023 17:56

github-actions bot added C-enhancement Category: This is an improvement C-feature Category: New features labels Mar 16, 2023

arya2 force-pushed the find-todos-with-closed-issues branch from 1b79bdb to 1d551b3 Compare March 16, 2023 17:59

arya2 changed the title ~~change(script): Add binary for finding refs to closed issues.~~ change(script): Add binary for finding references to closed issues. Mar 16, 2023

arya2 changed the title ~~change(script): Add binary for finding references to closed issues.~~ feat(script): Add binary for finding references to closed issues. Mar 17, 2023

arya2 added C-cleanup Category: This is a cleanup P-Medium ⚡ A-script Area: Script handling C-audit Category: Issues arising from audit findings labels Mar 17, 2023

arya2 commented Mar 17, 2023

View reviewed changes

zebra-utils/src/bin/search-issue-refs/main.rs Outdated Show resolved Hide resolved

upbqdn previously approved these changes Mar 17, 2023

View reviewed changes

arya2 dismissed upbqdn’s stale review via b6439dd March 20, 2023 22:15

arya2 requested a review from upbqdn March 20, 2023 22:20

arya2 added 10 commits March 21, 2023 14:11

adds search-todos to zebra-utils

e79d575

updates example output

d844a9d

updates example usage.

a546706

search for full issue url refs too

6bbd499

renames search-todos to search-issue-refs

ea9da45

Makes url in doc comment a hyperlink

5d5c14e

fixes spelling

7d2ab03

adds tokio to run a bit quicker

c412492

Update zebra-utils/src/bin/search-issue-refs/main.rs

2a27a7d

replaces if let .. else with let .. else

58a1c6c

arya2 force-pushed the find-todos-with-closed-issues branch from 0012243 to 30c5d87 Compare March 21, 2023 18:11

mpguerra mentioned this pull request Mar 21, 2023

Epic: Improvements from Zebra Audit #6277

Closed

36 tasks

adds location of references triggering github api calls that fail

9e99b69

teor2345 changed the title ~~feat(script): Add binary for finding references to closed issues.~~ feat(script): Add binary for finding references to closed issues Mar 21, 2023

arya2 added 3 commits March 27, 2023 04:49

Applies suggestions from code review

f111125

Add remote file refs

f792fb5

Update example output

d7fb9a4

mpguerra requested review from oxarbitrage and removed request for upbqdn March 27, 2023 12:09

arya2 requested review from upbqdn and removed request for upbqdn March 27, 2023 16:52

oxarbitrage previously approved these changes Mar 27, 2023

View reviewed changes

zebra-utils/src/bin/search-issue-refs/main.rs Outdated Show resolved Hide resolved

zebra-utils/src/bin/search-issue-refs/main.rs Outdated Show resolved Hide resolved

zebra-utils/src/bin/search-issue-refs/main.rs Show resolved Hide resolved

arya2 dismissed oxarbitrage’s stale review via f111125 March 27, 2023 20:49

arya2 requested a review from oxarbitrage March 27, 2023 21:03

oxarbitrage reviewed Mar 27, 2023

View reviewed changes

zebra-utils/src/bin/search-issue-refs/main.rs Outdated Show resolved Hide resolved

Update zebra-utils/src/bin/search-issue-refs/main.rs

f2eee16

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

arya2 requested a review from oxarbitrage March 27, 2023 21:38

oxarbitrage reviewed Mar 27, 2023

View reviewed changes

zebra-utils/src/bin/search-issue-refs/main.rs Show resolved Hide resolved

oxarbitrage approved these changes Mar 27, 2023

View reviewed changes

mergify bot added a commit that referenced this pull request Mar 27, 2023

Merge of #6347

f649392

mergify bot mentioned this pull request Mar 27, 2023

merge queue: embarking main (9a3c88c) and [#6411 + #6347 + #6413 + #6414 + #6412] together #6419

Closed

mergify bot merged commit aa1e407 into main Mar 28, 2023

mergify bot deleted the find-todos-with-closed-issues branch March 28, 2023 01:40

mpguerra mentioned this pull request Apr 11, 2023

Tracking: TODOs with closed tasks #6281

Closed

4 tasks

arya2 mentioned this pull request Apr 18, 2023

v1.0.0-rc.7 release #6542

Merged

36 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(script): Add binary for finding references to closed issues #6347

feat(script): Add binary for finding references to closed issues #6347

arya2 commented Mar 16, 2023 •

edited

Loading

codecov bot commented Mar 16, 2023 •

edited

Loading

upbqdn left a comment

mpguerra commented Mar 23, 2023

oxarbitrage left a comment

mpguerra commented Mar 28, 2023

mpguerra commented Mar 31, 2023 •

edited

Loading

arya2 commented Mar 31, 2023 •

edited

Loading

teor2345 commented Apr 3, 2023

feat(script): Add binary for finding references to closed issues #6347

feat(script): Add binary for finding references to closed issues #6347

Conversation

arya2 commented Mar 16, 2023 • edited Loading

Motivation

Solution

Review

Reviewer Checklist

codecov bot commented Mar 16, 2023 • edited Loading

Codecov Report

upbqdn left a comment

Choose a reason for hiding this comment

mpguerra commented Mar 23, 2023

oxarbitrage left a comment

Choose a reason for hiding this comment

mpguerra commented Mar 28, 2023

mpguerra commented Mar 31, 2023 • edited Loading

arya2 commented Mar 31, 2023 • edited Loading

References to Closed Issues

see more:

teor2345 commented Apr 3, 2023

arya2 commented Mar 16, 2023 •

edited

Loading

codecov bot commented Mar 16, 2023 •

edited

Loading

mpguerra commented Mar 31, 2023 •

edited

Loading

arya2 commented Mar 31, 2023 •

edited

Loading