Calculate and display Jitter

[fujiapple852]

Trippy current calculates the StDev column as:

pub fn stddev_ms(&self) -> f64 {
    if self.total_recv > 1 {
        (self.m2 / (self.total_recv - 1) as f64).sqrt()
    } else {
        0_f64
    }
}

Where, for each successful Probe (for a given hop ttl), the total_recv, mean and m2 values are updated as follows:

let dur = probe.duration();
let dur_ms = dur.as_secs_f64() * 1000_f64;
hop.total_recv += 1;
hop.mean += (dur_ms - hop.mean) / hop.total_recv as f64;
hop.m2 += (dur_ms - hop.mean) * (dur_ms - hop.mean);

We would like to add support for a Jitter columns similar to those provided by mtr (see https://networkengineering.stackexchange.com/questions/5288/what-is-interarrival-jitter-in-mtr)

[trkelly23]

I am taking this issue on my existing fork. I will be working towards a PR for review by EOW.

[trkelly23]

Here's a proposal for the feature after taking a look at the code. The calculation requires at least 2 round trips to be completed for each hop. The millis Duration value from consecutive hops (HOPn - HOPn-1) results in jitter duration.
Calc: Jitter = RTTn - RTTn-1
RTT = Round trip time

1. Add --jitter -j option for the command line. Starts up in jitter display mode.
2. Headings will have a "J" prepended in the display.
3. Add values to any reporting like csv.
4. Might need jitter mode to Mode enum tui-jitter.
5. Stretch - When tui display is running pressing "j" will switch to jitter mode. This comes from having used mtr in the passed.

Am I overthinking it should I scale it back?

[fujiapple852]

@trkelly23 thanks for picking this up!

Add --jitter -j option for the command line. Starts up in jitter display mode.
Headings will have a "J" prepended in the display.

Which headings would be prefixed? I had imagined these jitter related values as additional columns in the hops table, is that what you are proposing?

A mockup of what I envisioned:

╭Hops───────────────────────────────────────────────────────────────────────────────────────────╮
│#   Host          Loss% Snt   Recv  Last  Avg   Best  Wrst  StDev  Jttr  Javg  jmax  Jint  Sts │
│2   10.76.194.68  0.0%  118   118   0.8   1.3   0.5   22.0  2.7    xxx   xxx   xxx   xxx   🟢  │

Is the proposal that these columns would be calculated and/or shown only if the --jitter (-j) flag is specified? While I think that is fine, what do you think about doing #757 first or along with this change so that the selection of columns to show can be more flexible?

Add values to any reporting like csv.

Yes that makes sense, we should add to all report modes.

Might need jitter mode to Mode enum tui-jitter.

Right, similar to --tui-as-mode we may want a --tui-jitter-mode or similar to control how jitter is displayed in the Tui. It could be as simple as a bool to turn jitter display on/off or an enum to give finer control of which columns are shown. Just to be clear, I don't think a new --mode tui-jitter is needed here however.

When tui display is running pressing "j" will switch to jitter mode. This comes from having used mtr in the passed.

Yes that would be useful. It shouldn't be to difficult to add as there are several other such things in the Tui already, such as pressing z to toggle the AS display mode. I can always help you with this part if needed.

The calculation requires at least 2 round trips to be completed for each hop. The millis Duration value from consecutive hops (HOPn - HOPn-1) results in jitter duration.
Calc: Jitter = RTTn - RTTn-1
RTT = Round trip time

I just had a look in mtr and, whilst we don't need to restrict ourselves to doing things the same way, it does serve as a useful reference point.

It allows specifying these additional (non-default) columns:

Jttr: Current Jitter
Javg: Jitter Mean/Avg.
Jmax: Worst Jitter
Jint: Interarrival Jitter

It would be useful to look at how these are calculated and aim to emulate the logic here. From a quick glance they seem more complex than simply RTTn - RTTn-1 though I confess statistics is not my area of expertise!

[trkelly23]

I have been working on the calculations for Jitter and wrote up a little test file with tests to validate. After checking mtr code and description of jitter per the spec RFC-3550 I was able to create a spreadsheet to calculate the values and code to calculate. The Jitter struct is for test purposes and these fields would go into Hop. Note the cumulative average calculation I'm using may have uncovered a bug in the hop.mean calculation.

Please, take a look at the calculation approach.

jitter.txt
The above file changed to .txt .rs not allowed.

[fujiapple852]

Note the cumulative average calculation I'm using may have uncovered a bug in the hop.mean calculation

Can you give some more details on the potential bug?

Aside from the fix I did in commit 4e90da7 (April 2022) the logic seems to have been the same since the very beginning. I don't recall what prompted me to fix that, I suspect I was doing a side-by-side comparison with MTR and was correcting for a divergence between the output.

I can't remember exactly, but I believe this may have been my translation of Welford's online algorithm for computing variance on a stream of data in a single pass (note the use of m2 in that algorithm and my original code).

Please, take a look at the calculation approach

I've had a look but I may not be understand correctly. Parsing at the code and paraphrasing, it seems the calculation you have are:

Javg:

Javg = ((Javg * total_recv) + dur_ms) / total_recv

Jint:

Jint = abs(dur_ms - Javg)

Jmax:

Jmax = max(Jmax, dur_ms)

The last one seems more like an overall max/worst calculation, rather than a mac/worst jitter calculation?

I don't see the basic Jttr calculation, unless that is what value is supported represent? That seems to be simply using dur_ms which doesn't seem right if so?

How do these calculations compare to the logic in MTR? Maybe best to impl these on a branch and do a side-by-side comparison?

[trkelly23]

I put a PR up after taking a look at Welford's calculation and a few stats sites. I see now the avg can be calculated if you have the total number in the dataset, the current avg & the new value. Nice much less processing and overhead.
I also verified the Jitter average (javg) is cumulative according to mtr.
As you said best to move the conversation to a PR.

Thanks for all the feedback and happy holidays! 🎅

[c-git]

Thanks and happy holidays to you as well.

I haven't had a chance to look at the code yet but let me know if you are doing any other calculations that are not "streaming friendly" and I can take a look as I still remember a lot of those I learned in school of which mean is one (the one you already found).