Fleet version:
4.76.0
💥 Actual behavior
In a load test with ~95K hosts and ~300K software titles, we're still seeing issues when calling the endpoint with some query parameters after applying fixes for #34677
=== Performance Test Results: All teams (global stats) ===
Description Average Worst Results
----------- ------- ----- -------
Page 0, DESC order 579ms 905ms 20 items <-- optimized
Page 0, ASC order 1.291s 2.259s 20 items <-- partiallly optimized
Page 1000, DESC order 490ms 765ms 20 items <-- optimized
100 per_page 445ms 556ms 100 items <-- optimized
With CVE scores 439ms 491ms 20 items <-- optimized
Order by name, page 0 7.499s 7.896s 20 items
Order by name, page 1000 8.985s 9.294s 20 items
Vulnerable only 5.865s 6.059s 20 items
Search 'chrome' 14.158s 16.473s 20 items
Known exploit filter 19.708s 20.482s 20 items
Min CVSS score 7.0 33.679s 34.382s 20 items
Max CVSS score 8.0 39.132s 40.768s 20 items
CVSS range 7.0-9.0 42.431s 44.177s 20 items
What we don't know is how many other pages and commonly used endpoints by customers have performance issues (>5s latency). We don't know the extent of the real problem here.
🛠️ To fix
Improve /software/titles endpoint performance; timebox TBD at estimation. (See comment.)
🧑💻 Steps to reproduce
- Spin up load test with >300K software and ~100K hosts.
- Run script to check performance: https://gist.github.com/getvictor/0ef1da8bf1b85a4a8a6d3ee94c254e08
🕯️ More info (optional)
The fix includes a small behavior change. The default primary sort of /software/titles remains host_counts, but the secondary sort is now software_title_id and not name. This was necessary to optimize the endpoint. This means that if you have 1 host in your fleet, the software page will not show the software titles ordered by name anymore. For large fleets, this does not matter since all titles have different host counts.
Fleet version:
4.76.0
💥 Actual behavior
In a load test with ~95K hosts and ~300K software titles, we're still seeing issues when calling the endpoint with some query parameters after applying fixes for #34677
What we don't know is how many other pages and commonly used endpoints by customers have performance issues (>5s latency). We don't know the extent of the real problem here.
🛠️ To fix
Improve
/software/titlesendpoint performance; timebox TBD at estimation. (See comment.)🧑💻 Steps to reproduce
🕯️ More info (optional)
The fix includes a small behavior change. The default primary sort of
/software/titlesremainshost_counts, but the secondary sort is nowsoftware_title_idand notname. This was necessary to optimize the endpoint. This means that if you have 1 host in your fleet, the software page will not show the software titles ordered by name anymore. For large fleets, this does not matter since all titles have different host counts.