-
Notifications
You must be signed in to change notification settings - Fork 267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ui: add caching to vulnerability reports (PROJQUAY-7250) #2900
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Putting the code aside for a moment, how good is, from a design standpoint, to move what is essentially a backend process to the frontend and then asking the frontend to keep data? We already cache (or do have the ability to cache) security reports on the API level, so the UI should just render that data. The securityworker doesn't need to do anything particular, since data is pulled from the memcached instance. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2900 +/- ##
=======================================
Coverage 70.89% 70.90%
=======================================
Files 436 436
Lines 40780 40780
Branches 5345 5345
=======================================
+ Hits 28913 28914 +1
- Misses 10175 10177 +2
+ Partials 1692 1689 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
@ibazulic The response of the security manifest endpoint can be very large in size. As described in the initial comment, when loading a tag list with sufficiently old image content that carries a lot of vulnerabilities, the individual security vulnerability reports can be very large in size, and they would be fetched whenever you navigate back and forth in the UI, even though the data is exactly the same. Even if the vulnerability report is cached on the server side, it is still always transferred to the client, needlessly. Users have reported slowness and total response traffic north of 40MB per tag list page, each report being 1~2MB worth of JSON. |
This moves the API calls to the
secscan
endpoint of Quay to react-query and also adds appropriate caching. This improves load time and resource utilization on the client and server side, since reports can be quite large (>1MB JSON) and there are many of them when loading a tag list view page with lots of manifest lists / OCI indexes, where a secscan API call is issued for every manifest child. The latter are now only issued when someone actually expands the manifest list row in the tag list table to view the child manifests.Using react-query allowed to clean up the code quite a bit and stop leveraging atoms for caching. It even introduced transparent and staggering retry (up to 3 times) which is useful in case the
secscan
endpoint was overloaded.This also changes the empty state view of a manifest detail page on the Packages and Security screen to simply have a message that this manifest didn't contain any content our scanner could recognize instead of still rendering the donut chart and package tables.
PS: There are still too many API calls going to render a tag list view due to a naive approach I took with #2469 - something I will fix in the next PR