Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of 3 api endpoints by about 312% (for O3 data) #4813

Merged
merged 1 commit into from
Sep 23, 2022

Conversation

kraih
Copy link
Member

@kraih kraih commented Sep 22, 2022

/api/v1/products, /api/v1/machines, and /api/v1/test_suites share the same backend code for fetching data. The current implementation makes thousands of select queries to fetch settings. That is very inefficient, so this patch switches to a join/collapse based prefetch approach. This is slightly more efficient than a plain prefetch, because it avoids the creation of DateTime objects. See https://metacpan.org/pod/DBIx::Class::ResultSet#collapse for more information about the collapse feature.

Progress: https://progress.opensuse.org/issues/94667

@codecov
Copy link

codecov bot commented Sep 22, 2022

Codecov Report

Merging #4813 (4e30b65) into master (58b5d19) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 4e30b65 differs from pull request most recent head 3c2a4b7. Consider uploading reports for the commit 3c2a4b7 to get more accurate results

@@           Coverage Diff           @@
##           master    #4813   +/-   ##
=======================================
  Coverage   98.09%   98.09%           
=======================================
  Files         377      377           
  Lines       34869    34870    +1     
=======================================
+ Hits        34205    34206    +1     
  Misses        664      664           
Impacted Files Coverage Δ
lib/OpenQA/WebAPI/Controller/API/V1/Table.pm 97.45% <100.00%> (+0.02%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Contributor

@Martchus Martchus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks generally very good, I have only one remark (see inline comment) that might be worthwhile to look into.

lib/OpenQA/WebAPI/Controller/API/V1/Table.pm Outdated Show resolved Hide resolved
`/api/v1/products`, `/api/v1/machines`, and `/api/v1/test_suites` share
the same backend code for fetching data. The current implementation
makes thousands of select queries to fetch settings. That is
very inefficient, so this patch switches to a join/collapse based
prefetch approach. This is slightly more efficient than a plain
prefetch, because it avoids the creation of DateTime objects. See
https://metacpan.org/pod/DBIx::Class::ResultSet#collapse for more
information about the collapse feature.

Progress: https://progress.opensuse.org/issues/94667
@kraih kraih merged commit f232561 into master Sep 23, 2022
@okurz okurz deleted the k/faster_tables branch September 23, 2022 08:58
@perlpunk
Copy link
Contributor

So the only thing I missed in my original suggestion was the order 🤦

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants