Skip to content

Optimize software/versions queries. (#35670)#36354

Merged
getvictor merged 1 commit intorc-minor-fleet-v4.77.0from
victor-software-versions-cherry-pick
Nov 26, 2025
Merged

Optimize software/versions queries. (#35670)#36354
getvictor merged 1 commit intorc-minor-fleet-v4.77.0from
victor-software-versions-cherry-pick

Conversation

@getvictor
Copy link
Copy Markdown
Member

Related issue: Resolves #34677 and Resolves #35349

Loadtest results:

Description               Average  Worst    Results
-----------               -------  -----    -------
Page 0, DESC order        441ms    506ms    20 items
Page 0, ASC order         1.099s   1.8s     20 items
Page 1000, DESC order     484ms    641ms    20 items
100 per_page              426ms    450ms    100 items
With CVE scores           467ms    630ms    20 items
Order by name, page 0     7.589s   7.812s   20 items
Order by name, page 1000  9.103s   9.656s   20 items
Vulnerable only           6.098s   6.34s    20 items
Search 'chrome'           14.305s  14.868s  20 items
Known exploit filter      20.253s  21.238s  20 items
Min CVSS score 7.0        33.743s  35.169s  20 items
Max CVSS score 8.0        39.825s  41.83s   20 items
CVSS range 7.0-9.0        42.556s  43.267s  20 items

Follow-up issue: #35799

Checklist for submitter

  • Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
    See Changes
    files
    for more information.

Testing

Database migrations

  • Checked schema for all modified table for columns that will auto-update timestamps during migration.

Summary by CodeRabbit

  • Refactor
  • Improved software listing and counting performance via database index and query optimizations, resulting in faster retrieval and reduced load times for software lists across team and global views.
  • Chores
  • Added a migration to apply the index changes and updated migration tracking.

(cherry picked from commit 3724166)

<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34677 and Resolves #35349

Loadtest results:
```text
Description               Average  Worst    Results
-----------               -------  -----    -------
Page 0, DESC order        441ms    506ms    20 items
Page 0, ASC order         1.099s   1.8s     20 items
Page 1000, DESC order     484ms    641ms    20 items
100 per_page              426ms    450ms    100 items
With CVE scores           467ms    630ms    20 items
Order by name, page 0     7.589s   7.812s   20 items
Order by name, page 1000  9.103s   9.656s   20 items
Vulnerable only           6.098s   6.34s    20 items
Search 'chrome'           14.305s  14.868s  20 items
Known exploit filter      20.253s  21.238s  20 items
Min CVSS score 7.0        33.743s  35.169s  20 items
Max CVSS score 8.0        39.825s  41.83s   20 items
CVSS range 7.0-9.0        42.556s  43.267s  20 items
```

Follow-up issue: #35799

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Improved software listing and counting performance via database index
and query optimizations, resulting in faster retrieval and reduced load
times for software lists across team and global views.
* **Chores**
* Added a migration to apply the index changes and updated migration
tracking.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

(cherry picked from commit 3724166)
@codecov
Copy link
Copy Markdown

codecov Bot commented Nov 26, 2025

Codecov Report

❌ Patch coverage is 86.63366% with 27 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (rc-minor-fleet-v4.77.0@c2876ac). Learn more about missing BASE report.

Files with missing lines Patch % Lines
server/datastore/mysql/software.go 88.39% 19 Missing and 2 partials ⚠️
.../20251117020000_OptimizeSoftwareHostCountsIndex.go 66.66% 4 Missing and 2 partials ⚠️
Additional details and impacted files
@@                    Coverage Diff                    @@
##             rc-minor-fleet-v4.77.0   #36354   +/-   ##
=========================================================
  Coverage                          ?   66.16%           
=========================================================
  Files                             ?     2088           
  Lines                             ?   177827           
  Branches                          ?     7372           
=========================================================
  Hits                              ?   117664           
  Misses                            ?    49332           
  Partials                          ?    10831           
Flag Coverage Δ
backend 67.88% <86.63%> (?)
frontend 53.52% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@getvictor getvictor marked this pull request as ready for review November 26, 2025 18:39
@getvictor getvictor requested review from a team as code owners November 26, 2025 18:39
@getvictor getvictor merged commit e6ff16f into rc-minor-fleet-v4.77.0 Nov 26, 2025
38 checks passed
@getvictor getvictor deleted the victor-software-versions-cherry-pick branch November 26, 2025 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants