backport: merge bitcoin#19326, #20464, #23451, #20852, #20355, #22050, #23115, #23409, #18468, #23413, partial bitcoin#21560, #23438 (auxiliary backports: part 11)

[kwvg]

Additional information

• bitcoin#19326 was partially backported as part of dash#4164 to the exclusion of 2a2182c, 77c5073.
  The excluded commits have been backported, marking the pull request as fully merged.

• bitcoin#18468 was partially backported as part of dash#4036 to the exclusion of 2676aea.
  The excluded commit has been backported, marking the pull request as fully merged.

• bitcoin#21560 does not include any TorV3 nodes.

• When backporting bitcoin#20464, llmq/dkgsessionhandler.cpp needs a reinterpret_cast because CHashWriter accepts a const char* while CDataStream's data() returns an unsigned const char*.

  Using reinterpret_cast is acceptable because CHashWriter::write() simply casts the const char* to unsigned const char*, reversing the previous cast (source), making this exercise solely for the purpose of fitting the pointer through the argument.

  A C-style cast wasn't used because the linter took objection to it (build) and only it, but seemed to be fine with every other C-style cast.

• Dependency for backport: merge bitcoin#27479, #27230, #25251, partial #22934, #23383, #24792, #26691, #27445 (secp256k1 update) #5575

  • bitcoin#27479, the primary motivation for dash#5575 uses features added to Span (such as the removal of MakeSpan and the introduction of BytePtr)
  • Another backport that was primarily required as bitcoin#22570, which does away with the serialization code for CSubNet, which utilizes Span but would be incompatible with bitcoin#23413, at least on macOS (build)

    • This in turn pulled in a bunch of networking-related backports and only the minimum set required to backport bitcoin#22570 has been included.

      This has the effect of making bitcoin#17428 vestigial as the code that actually integrates it with existing logic has been excluded from the backport as it relies on larger networking backports which are far beyond the scope of this PR.

  • This in turn required backporting the contributions made before it, which revealed merge conflicts, which then needed to be added into the fold of backports included in the pull request in order to ensure the least amount of conflicts.

Warnings

• The greater adoption of Span, removal of MakeSpan and introduction of BytePtr involved changes made to taproot and segregated witness code. This pull request may complicate Taproot backporting should that ever be on the roadmap.

• BytePtr is later renamed to AsBytePtr due to a naming conflict with the iOS SDK (see discussion for bitcoin#24989) but in the interest of minimizing the set of changes needed to merge in dash#5575 and the fact that Dash does not target i(Pad)OS, this is considered acceptable for now.

[knst]

as discussed in slack, please, use Span class here instead std::vector for scriptPubKey's + Hash160

Here's possible implementation: knst@a534b5f

[PastaPastaPasta]

as discussed in slack, please, use Span class here instead std::vector for scriptPubKey's + Hash160

Here's possible implementation: knst@a534b5f

This isn't needed? The purpose of using span is you can implicitly construct one from basically anything; seems fine to me to keep it as is

[knst]

This isn't needed? The purpose of using span is you can implicitly construct one from basically anything; seems fine to me to keep it as is

These 2 commits are extra context for you @PastaPastaPasta
knst@5d91e7e
knst@7d80ffb

so, this original dash's code will cause a compilation error with a new Span implementation (after all backports from this batch in this PR):

uint160 hashBytes(Hash160(prevout.scriptPubKey.begin()+1, prevout.scriptPubKey.end()-1));

There's approach in this PR (kwvg's solution):

-uint160 hashBytes(Hash160(prevout.scriptPubKey.begin()+1, prevout.scriptPubKey.end()-1));
+uint160 hashBytes(Hash160(std::vector<unsigned char>(prevout.scriptPubKey.begin()+1, prevout.scriptPubKey.end()-1)));

my patch:

-uint160 hashBytes(Hash160(prevout.scriptPubKey.begin()+1, prevout.scriptPubKey.end()-1));
+uint160 hashBytes{Hash160(Span{prevout.scriptPubKey.data()+1, prevout.scriptPubKey.size() - 2})};

my solution uses Span instead std::vector and it helps to avoid extra memory allocation, because Span do now copy data, do not own it, it is just container with pointer to data + its length (same as std::span and std::string_view), but std::vector<...> will cause memory allocation + memory copy + memory deallocation when its data no need more.

[PastaPastaPasta]

@knst thanks for the context! I didn't see that current code was allocating a vector.

[PastaPastaPasta]

utACK for merging via merge commit

[UdjinM6]

utACK

[knst]

LGTM