Incorrect "TCP upload sum" for tcp_nup, rrul, rrul_be tests #182
Comments
|
Update: dtaht (in #178 (comment)) said of the ping times, "I have seen this statistic be wildly wrong in the summary but correct in the graph." Related? |
|
Rich Brown <notifications@github.com> writes:
Update: dtaht (in
#178 (comment))
said of the ping times, "I have seen this statistic be wildly wrong in
the summary but correct in the graph."
Yes, this can be the case because of the way the summary values are
computed. However, it *should* be the case that what the summary outputs
is computed from the aggregate statistics that netperf outputs at the
end of the test where possible.
However, it seems there was a bug causing all this per-series metadata
to be cleared out before the file was saved. Not good! Will post a fix...
|
tohojo
added a commit
that referenced
this issue
Sep 13, 2019
The call to results.create_series() also clears out the series metadata. Since this happened after the metadata was saved, the end result was empty SERIES_META in the data files. This in turn led to odd results for bandwidth summary statistics, and of course the absence of all this metadata from the test results, as reported by Rich Brown. Fixes #182 (for new data files). [Backport of ff46520 from master] Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
|
Sorry to re-open, but the new version (Flent 1.9.9-git-ff46520) seems to fix the upload values for the Average - the individual streams sum to the displayed sum. But the Median values still don't look right. Here are two runs of the same command: |
|
Rich Brown <notifications@github.com> writes:
Sorry to re-open, but the new version (Flent 1.9.9-git-ff46520) seems
to fix the upload values for the Average - the individual streams sum
to the displayed sum.
But the Median values still don't look right. Here are two runs of the
same command:
Ah, no, the median has the same issue, but there is no per-test output
we can use, so we need to calculate it. And since the summary output
calculates a per-series median, the sum doesn't match up (which is not
too surprising; the median of the total throughput is not necessarily
the sum of the medians of each individual flow).
We could remove the median from the calculated data series if that is
less confusing?
|
|
Ah, you're right. Neither the sum nor the average of the medians is meaningful in any predictive way, so there's no use in showing it. I suppose that median of all samples might someday be interesting, but Flent doesn't compute it now. Yes - removing the sum and average of the medians, or displaying them as "-" would indicate that we know there isn't a value there. The docs can explain in greater depth why they're not shown. Thanks. |
tohojo
added a commit
that referenced
this issue
Sep 14, 2019
The median of sums is not really a meaningful value, and showing it was causing confusion. So just don't emit the median if the data series comes from a computed runner. Fixes #182 (again). Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
|
Confirmed. I like |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Using Flent 1.3.0 on macOS 10.12.6:
I notice that the "TCP upload sum" value shown in the upload portion of tests (tcp_nup, rrul, rrul_be) doesn't match the sum of the individual flows. In the first example below, the displayed sum (0.28 Mbps) is considerably lower than the sum of the individual flows (0.77 Mbps - which matches the expected speed of my 7Mbps/768kbps DSL link).
I attach the flent.gz and the log files from the first run shown below. (Earlier rrul and rrul_be test runs follow.) What other information could I gather? Thanks.
tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.log
tcp_nup-2019-09-12T074946.261029.flent-fremont_bufferbloat_net.flent.gz
The text was updated successfully, but these errors were encountered: