Potential issues with "95th Percentile Response Latencies (ms)" Panel Data

[SeamusGrafana]

I will prefix this by saying I am not sure if this is an actual issue or by design.

However, in the "95th Percentile Response Latencies (ms)" Panel, recently it shows a value of 1000 for all Endpoints, which while it appears to be correct, does not feel expected or realistic.

I suspect this is to do with a change added here;

#75

To test this, I used the latest Image Tag on Docker (which I believe relates to 0.3) and then against 0.2;

From around ~10:30 to 11:00am on that Graph, its using the latest image tag and has a consistent value of 100 for all endpoints.

At ~11:00am, I switched the Image Tag to 0.2 and restarted everything, and from then the values drop right down, and appear to vary quite a bit;

I suspect the spiky graph feature that was added is what is causing the latency to always show a value of 1000, this is fine, but it does not feel as realistic as before.

Am wondering if there is a change that could be made here to make this look a bit more "real" or similar.

If its expected and not an issue, then feel free to close this.

Thanks!

[hedss]

@SeamusGrafana Sooo sorry it's taken this long, FE have had a very busy 8 weeks or so.
So, the reasoning for this is multi-faceted, but we ideally want some pretty long latency times to make it very obvious in traces what's happening.
What does need to get fixed is that atm the bucket maxes out at 1000ms, which is not enough to accurately show these latencies. I'll bump the bucket sizes up so we get more realistic graphs.
And thank you!