Add PTF tests for MBR-based rate limiting

[ccascone]

We have observed issues in production deployments where BESS-UPF performs excessive dropping. This PR adds the following PTF tests to verify the following properties for both uplink and downlink scenarios:

• Conforming traffic (below MBR) should not experience drops
• Non-conforming traffic (above the MBR) should be policed to the MBR

Moreover, when running tests, we noticed that sometimes TRex is not able to achieve or sustain the desired TX rate for the duration of the test, invalidating results. While we don't know the root cause, this PR adds a new utility function trex_util.start_and_monitor_port_stats that immediately fails the test in such cases by:

• Monitoring TX/RX rates in real-time
• Raising an exception if the TX rate drops below a configurable threshold

TODO:

• Add sanity check assertion to make sure Trex is generating traffic at a high enough rate
• Add test for non-conforming sources (i.e., above app MBR, should experience drops)
• Add similar tests for session QERs will do in subsequent PR

Sample output:

Running tests...
----------------------------------------------------------------------
  mbr.DlAppMbrConformingTest ... 
Testing app MBR 10.0 Mbps...
 TX rate with Ethernet overhead: 10.0 Mbps (101.0%)
 1.13s: TX 8.2 Mbps (823.0 pps) --> RX 8.4 Mbps (~0.0% pkt loss) *
 2.06s: TX 10.0 Mbps (776.0 pps) --> RX 10.3 Mbps (~0.0% pkt loss)
 3.06s: TX 10.0 Mbps (887.3 pps) --> RX 10.2 Mbps (~0.1% pkt loss)
Testing app MBR 50.0 Mbps...
 TX rate with Ethernet overhead: 50.0 Mbps (101.0%)
 1.13s: TX 40.8 Mbps (4.1 Kpps) --> RX 41.8 Mbps (~0.0% pkt loss) *
 2.06s: TX 49.9 Mbps (3.9 Kpps) --> RX 51.2 Mbps (~0.0% pkt loss)
 3.07s: TX 50.1 Mbps (4.5 Kpps) --> RX 51.4 Mbps (~0.0% pkt loss)
...

Running tests...
----------------------------------------------------------------------
  mbr.UlAppMbrNonConformingTest ... 
Testing app MBR 10.0 Mbps...
 TX rate with Ethernet+GTPU overhead: 20.7 Mbps (103.6%)
 1.11s: TX 17.1 Mbps (1.7 Kpps) --> RX 8.4 Mbps (~49.5% pkt loss) *
 2.07s: TX 20.5 Mbps (1.6 Kpps) --> RX 10.0 Mbps (~50.0% pkt loss)
 3.06s: TX 20.8 Mbps (1.8 Kpps) --> RX 10.1 Mbps (~50.0% pkt loss)
Testing app MBR 50.0 Mbps...
 TX rate with Ethernet+GTPU overhead: 103.6 Mbps (103.6%)
 1.12s: TX 85.6 Mbps (8.3 Kpps) --> RX 42.2 Mbps (~49.5% pkt loss) *
 2.06s: TX 103.4 Mbps (7.9 Kpps) --> RX 50.4 Mbps (~50.0% pkt loss)
 3.06s: TX 103.5 Mbps (8.9 Kpps) --> RX 50.5 Mbps (~50.0% pkt loss)
...

[amarsri28]

there is an issue found where metering could happen at absurd rate due to memory corruption. try to test with given stress test and in production as well with provided fix - #528 #529

[ccascone]

Before, we were not checking the response from BESS, silently dropping errors such as failed QER insertions.