Large loan performance is very poor

[grantfitzsimmons]

https://herbrbge11723-edge.test.specifysystems.org/specify/view/loan/860/

Expected behavior
Large loans (such as the one linked above) should load much quicker and should not take 5 minutes to load. The loan return dialog is much faster than the form is to load, but that could be due to a smaller number of records.

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to loan 860 on a copy of herb_rbge's database (linked above)
2. Let the page load
3. See that it can take several minutes for all loan preparations to appear

[grantfitzsimmons]

I feel like @realVinayak would fix this quickly

[grantfitzsimmons]

This is still something Lesley at RBGE is having issues with.

[grantfitzsimmons]

Reported By: Robyn at RBGE

We have now upgraded Lesley’s machine to one that is the most powerful RBGE issues (they are often used by those working on bioinformatics/sequencing work).

Unfortunately she still seems to be experiencing issues with getting loans to load without Specify crashing.

I have attached a crash report for loan 0255 (469 preparations). She has tried opening this loan on several occasions.

She has also had issues with loan 00860 (635), this triggers the ‘Aw snap’ error in chrome.

I have tried opening both loans (both browse in forms and own tabs) and running reports, and have had no issues with any of these records. I was running these on my laptop using Firefox.

Any thoughts on what may be causing these issues, as it is preventing Lesley being able to complete work that is core to her job.

This loan has 635 loan preparations, and each of these makes a number of requests to return the catalog number, prep type, current determination, and counts.

This number of requests being made all at once results in a flurry of type errors! This should be handled somehow to prevent this from occuring.

https://herbrbge9323-v791-prerelease.test.specifysystems.org/specify/view/loan/860/

Specify 7 Crash Report Oct 6.txt

@CarolineDenis @acwhite211 Is there any way we could load these preparations in chunks so that it doesn't overwhelm the system and cause issues with too many requests and running out of memory?

It uses a great deal of memory on my system as well as theirs. We need to handle these cases.

[CarolineDenis]

Temporary solution:
Change the Loan Preparations to sub view form and not to table form

[CarolineDenis]

For future development:
Only fetch the records displayed on the screen in a form table.
Add a fetch more on scroll down

[maxpatiiuk]

Virtual scrolling library (#1761) would likely be a perfect choice in here - each row is the same height. Virtual scroll library will not render the component (and thus not make network request) until necessary

[emenslin]

Can recreate in edge (7.9.6), potentially also related to #3001 as the fields that take the longest to show up are the relationship fields

[specifysoftware]

This issue has been mentioned on Specify Community Forum. There might be relevant details there:

https://discourse.specifysoftware.org/t/specify-6-eol-questions-attachment-mapping-files-and-repeat-label-printing/3871/2