Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ui: add caching to vulnerability reports (PROJQUAY-7250) #2900

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

dmesser
Copy link
Contributor

@dmesser dmesser commented May 22, 2024

This moves the API calls to the secscan endpoint of Quay to react-query and also adds appropriate caching. This improves load time and resource utilization on the client and server side, since reports can be quite large (>1MB JSON) and there are many of them when loading a tag list view page with lots of manifest lists / OCI indexes, where a secscan API call is issued for every manifest child. The latter are now only issued when someone actually expands the manifest list row in the tag list table to view the child manifests.

Using react-query allowed to clean up the code quite a bit and stop leveraging atoms for caching. It even introduced transparent and staggering retry (up to 3 times) which is useful in case the secscan endpoint was overloaded.

This also changes the empty state view of a manifest detail page on the Packages and Security screen to simply have a message that this manifest didn't contain any content our scanner could recognize instead of still rendering the donut chart and package tables.

PS: There are still too many API calls going to render a tag list view due to a naive approach I took with #2469 - something I will fix in the next PR

Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
Signed-off-by: dmesser <dmesser@redhat.com>
@ibazulic
Copy link
Member

Putting the code aside for a moment, how good is, from a design standpoint, to move what is essentially a backend process to the frontend and then asking the frontend to keep data? We already cache (or do have the ability to cache) security reports on the API level, so the UI should just render that data. The securityworker doesn't need to do anything particular, since data is pulled from the memcached instance.

Copy link

codecov bot commented May 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.90%. Comparing base (3166c5f) to head (a9b8bda).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2900   +/-   ##
=======================================
  Coverage   70.89%   70.90%           
=======================================
  Files         436      436           
  Lines       40780    40780           
  Branches     5345     5345           
=======================================
+ Hits        28913    28914    +1     
- Misses      10175    10177    +2     
+ Partials     1692     1689    -3     
Flag Coverage Δ
unit 70.90% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dmesser
Copy link
Contributor Author

dmesser commented May 27, 2024

@ibazulic The response of the security manifest endpoint can be very large in size. As described in the initial comment, when loading a tag list with sufficiently old image content that carries a lot of vulnerabilities, the individual security vulnerability reports can be very large in size, and they would be fetched whenever you navigate back and forth in the UI, even though the data is exactly the same. Even if the vulnerability report is cached on the server side, it is still always transferred to the client, needlessly. Users have reported slowness and total response traffic north of 40MB per tag list page, each report being 1~2MB worth of JSON.
While this PR does not address that in itself, it at least avoids loading that many bytes again and also reloading the individual vuln report for a manifest from the server when navigating from the tag list to the tag detail page. The caching is also rather a by-product of using react-query instead of directly embedding Axios calls into the component codes, helping with decluttering the code and avoiding props drilling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants