Optimization: More memory-efficient scan state and staged ledger hashing

[tizoc]

Explain your changes:

• Hashing of the scan state and staged ledger performs a lot of unnecessary intermediary allocations by appending strings together before feeding them to the hashing context. This version avoids such concatenation and instead feeds strings directly to the hashing context while retaining the same ordering. Avoiding such unnecessary allocations relieves the garbage collector from all that pressure and also solves some >1GB allocation spikes.

Explain how you tested your changes:

• Patched nodes were launched in a mixed environment (patched and unpatched nodes) in a private cluster. Also a bunch of randomly generated scan states were hashed with both new and old versions to ensure that the result is the same.

Checklist:

• Dependency versions are unchanged
  • Notify Velocity team if dependencies must change in CI
• Modified the current draft of release notes with details on what is completed or incomplete within this project
• Document code purpose, how to use it
  • Mention expected invariants, implicit constraints
• Tests were added for the new behavior
  • Document test purpose, significance of failures
  • Test names should reflect their purpose
• All tests pass (CI will check this if you didn't)
• Serialized types are in stable-versioned modules
• Does this close issues? List them

[bkase]

Looks good to me!

[deepthiskumar]

!ci-build-me

[deepthiskumar]

!approved-for-mainnet

[deepthiskumar]

The version linter is failing due to the PR branch not being in this repo. We are working to create another pipeline. The PR doesn't update any types and the hashes before and after were tested.