TCP P2P optimization

[shargon]

• VersionPayload now have a enum for services
• The command in header now is one byte instead of 12 byte string
• P2P have auto compression mechanism
• MagicBytes is sent in VersionPayload
• CRC was removed

[shargon]

Some compression samples:

Block 1945085 (Biggest):

• Original block size: 3.002.776 bytes
• Message size: 230.947 bytes
• Compression ratio: 13.002
• Space savings: 92.31%

Block 1318368 (Big one):

• Original block size: 512.428 bytes
• Message size: 503.367 bytes
• Compression ratio: 1.018
• Space savings: 1.77%

Block 4 (with only 1 tx):

• Original block size: 686 bytes
• Message size: 691 bytes
• Not compressed

[vncoelho]

Congratulations, @shargon!
Count with me for a deep review and revision.
I am going to wait for your signals for when to start to review.
What are the expected deadlines?

I need to focus on some other minor things during this week.

[canesin]

@shargon any opinion on https://facebook.github.io/zstd/#small-data ? Would it help here ?

[vncoelho]

@canesin, great to see you here, my friend.
I took a look and it is interesting. It uses learning techniques for optimizing the compression.
On the other hand, I am thinking about how to improve hashes compression? There might be no outstanding patterns. ahueahuea
But I believe it is worth investigating, Canesin.
What do you think, @igormcoelho?

@shargon, I think that this is the link for the C# recommended implementation: https://github.com/bp74/Zstandard.Net

[shargon]

@canesin maybe we can use different flags for different compression mechanisms, and allow more in the future. And maybe if zstandard is better for small data, we can change it on real time, and choose the best one for our packet

[vncoelho]

The compression details goes in the header, @shargon?
In this sense, those who receive it can easily identify which mechanism was used, right? I will take some time to ready the proposal as soon as possible.

[shargon]

@vncoelho yes, now we have flags for each message, and we can use it for compression, encryption, or other things

[vncoelho]

@shargon, these github drafts are good, but not practical... I will mark it as ready to review.

[vncoelho]

I am starting to check the code, @shargon.

It looks like to be a nice job, congratulations.

[shargon]

The connection is lost when we receive verack message :S

[shargon]

Now, is working well in my laptlop, could anyone more test it please?

[erikzhang]

@shargon @belane When can I merge it?

[shargon]

I think that we can add this improve too: shargon#11

We thought that GetBlocksPayload could be improved, what do you think?

[shargon]

My problem was a configuration problem, so i think that we can merge it, but please, review this shargon#11 for incorporate to this pr or don't

[erikzhang]

shargon#11 is good. But I can't modify it.

[shargon]

I invited you to the repo, and I merged it, please, modify whatever you want.