Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Bodhi web UI unusable for large updates after 2.2.0 update #951
After latest Bodhi update, the sheer number of taskatron test results seems to cause most web browsers to hang. Try any larger update, such as https://bodhi.fedoraproject.org/updates/FEDORA-2016-78efe5595f -- Firefox locks up here after clicking on this link and Chrome as well.
Firefox fills all available real memory and swap space after opening https://bodhi.fedoraproject.org/updates/FEDORA-2016-78efe5595f. I killed the process at 8 GB and counting .. .
Thanks for the bug report! A quick fix would probably be to limit the number of taskotron results that are rendered in the template to some arbitrary but small-enough number. I think we could get a fix like that in soon so that the pages at least render.
Something perhaps a little more sohpisticated would be to sort the tasks so that the failing ones sort highest, limit them to some arbitary number, and if there are more than our limit we can just link to the full taskotron output (something like this https://taskotron.fedoraproject.org/artifacts/all/8ac4e78e-7e6a-11e6-bab6-525400120b80/task_output/ ?)
What do you think?
There are multiple issues, I think. I'll try to describe them in detail, sorry for a long post.
First, tasks IDs look like
So, for the moment, I'd probably only display results with one level under
Second, all our tasks publish results per koji build. Some of them also publish per-bodhi-update results (like depcheck), but the new ones don't (rpmgrill, abicheck) and we'd like to move away from per-bodhi-update results (once bodhi is ready) even for the old ones. We've had big issues in keeping those per-update results in sync with bodhi state (when results are edited) and figured it would be much easier to aggregate the results on bodhi side (which always knows the current state of the update, and doesn't need to do slow network queries). So, bodhi should aggregate per-build results into an overall result for multi-build updates. (Well, it's not strictly necessary, but then updates with >5 builds will get enormous list of results that will be tedious to read through).
So, I imagine this could be again solved with tree-like structure. The top item would be an overall depcheck result, and if you expanded that, you would see depcheck results for individual builds. It could even show an "incomplete" icon next to the overall result, if bodhi detected there are still results missing for certain builds.
Overall I imagine it could be like this:
Of course, if you can then sort this list first according to outcome importance (FAIL > NEEDS_INSPECTION > PASS), and then according to task name, that would be also helpful. But sorting is probably the last thing to do here, first we need to trim the current extra long list of individual results into some shorter, easier to read structure (e.g. like the one proposed above).
Sounds reasonable? I understand this is not trivial to implement. If you need any help with resultsdb queries or even support for some new queries (we're working on resultsdb v2, now is the time to ask for compatibility-breaking features), we'll try to do our best.
@kparal I'll just clean up some terminology, that might lead to false expectancies:
This is not technically true, as we don't really store/provide definitive relations between results, on the Taskotron/ResultsDB side. So there is no way to say "give me all 'subresults' of a this result with ID X". What you could do, though (and what @kparal probably meant) is that you could do a query like this:
Absolutely. This IMHO also goes a bit with what you said about the tasks - that each and every task can store the results in a different way. Bodhi should be the place where the results get aggregated to "can/can not be pushed". This of course means, that Bodhi should know which of the results are "blocking", but this is a policy isuue, basically. What I wanted to say here is, that we strive to keep ResultsDB as dumb as possible, so even if the "decision" would not be made in Bodhi, some kind of middleware would have to be created to "aggregate" the results for (istead of) it. ResultsDB will happily provide the data, but it won't ever hold/provide any meaning for it.
One more thing that I wanted to mention - ResultsDB v2 will have an additional endpoint (created with Bodhi in mind, to be honest), that will provide "latest results" for all relevant testcases & filter. What it means is that instead of grabbing a list of testcases first, and then querying ResultsDB multiple times, you'll be able to do
Hope that helps. And as @kparal said - we are about to roll out new version, so if you want any functionality, now is the time to ask.
I wanted to make updates visible in Bodhi as quickly as possible, so I've worked out a "dirty hack" for this problem and have it deployed to staging right now:
Basically, if there are more than 16 builds in an update (this one had 60-something IIRC) Bodhi begins to filter out all passing test results to reduce the number of elements in the page. You will notice that the update still takes a long while to load, but it should load eventually without going too crazy on RAM. The number 16 is completely arbitrary (but a pleasing power of 2) - I just played around with different numbers of updates until my browser seemed near the limit. It also inserts a note at the top of the results table to indicate that the results are being filtered.
I plan to submit a PR with this change today and will hopefully get a 2.2.3 release out with it soon if we are allowed to do so during the infrastructure freeze. I don't intend for this patch to be the long term fix to this problem, but just a band-aid so the pages at least load. If we find that 16 is still too many we can adjust it later.