Calculate and expose more individual peer info through peers endpoint

[hidenori-shinohara]

Description

Resolves #3041

This PR calculates and exposes more metrics for each peer through the peers endpoint. The new output can be slightly verbose. If compact=true, the new output will be the same as the current output.

Checklist

• Reviewed the contributing document
• Rebased on top of master (no merge commits)
• Ran clang-format v8.0.0 (via make format or the Visual Studio extension)
• Compiles
• Ran all tests
• If change impacts performance, include supporting evidence per the performance document

[marta-lokhova]

Why do we use all-time max? Not sure it's helpful, as this data decays overtime and provides less insight into what's going on.

[hidenori-shinohara]

Updated to show P99s from the last 300-second window

[hidenori-shinohara]

Here's a sample output for one peer when compact=false:

         {                                                                                                            
            "address" : "54.173.54.200:11625",
            "async_read" : 149,    
            "async_write" : 511,              
            "byte_read" : 353760,            
            "byte_write" : 315188,            
            "duplicate_fetch_message_recv" : 0,
            "duplicate_flood_message_recv" : 547,
            "elapsed" : 18,                    
            "flow_control" : {                    
               "local_capacity" : {
                  "flood" : 200, 
                  "reading" : 201                 
               },                                 
               "outbound_queue_delay_scp_p99" : 0,
               "outbound_queue_delay_txs_p99" : 0,
               "peer_capacity" : 198,        
               "state" : "enabled"
            },                                     
            "id" : "sdf_1",                        
            "latency" : 22,        
            "message_delay_in_async_write_p99" : 0,
            "message_delay_in_write_queue_p99" : 92,
            "message_drop" : 0,
            "message_read" : 748,                  
            "message_write" : 705,                 
            "olver" : 20,                                                                                             
            "unique_fetch_message_recv" : 0,
            "unique_flood_message_recv" : 0,
            "ver" : "stellar-core 18.5.0 (d387c6a710322135ac076804490af22c4587b96d)"                                  
         }

[marta-lokhova]

Looks good to me overall! One thing I realized is that reporting p99 for the timers maybe isn't as useful as, say, p75, so perhaps we should report that instead.

[marta-lokhova]

tiny nit: maybe better to rename timer to aggregateTimer for clarity

[hidenori-shinohara]

Fixed!

[hidenori-shinohara]

I will squash the commits tomorrow

[hidenori-shinohara]

I just finished squashing the commits.

[MonsieurNicolas]

r+ 0064a59