Displaying the case list for large projects is slow

[holtgrewe]

Describe the bug
It is very slow to open the list of all cases for many projects.

To Reproduce
Steps to reproduce the behavior:

1. Open the case list for a project with many cases.
2. See the page load for many seconds.

Expected behavior
The page list should be faster or we have to add pagination to make it faster.

Screenshots
N/A

Additional context

• Displaying the project overview for large cohorts is slow #303
• Perform further performance optimizations in case list #307 (follow-up)
• Make annotation display in case overview list scale to large projects #308
• Make project-wide quality control scale up to large cohorts #309

[holtgrewe]

Root Cause Analysis

• look at all queries generated by project overview
• there are 10k+ annotated variants which explains the problem (all of them are fetched, chrome developer tools shows 46MB of data)
• previously, we only had one web server thread, now there are more so things can actually load in the background
• after the tab with the annotated variant has loaded, the tab actually gets overloaded
• further, the QC data for such big projects is huge (e.g., chrome developer tools show 70MB of data for QC)

[holtgrewe]

Resolution Proposal

• only load quality control and annotation table when the tab is actually opened
• refuse to display variant annoation as HTML for more than 100 variants
• load quality control figures only when user explicitely clicks button
• document this to the user as a known issue

Note that this is only a mitigation, the problem will be corrected properly in follow-up issues.

Affected Components

• varfish-server

Affected Modules/Files

• variants modules with urls, views, templates

Required Architectural Changes
N/A

Resolution Sketch

• use more lazy loading of tabs
• add button to load QC explicitely

[holtgrewe]

Whoops, flipped #303 and #304 here. Moving text to #303.