Request form Cannot Load Holdings that have many items

[maxkadel]

Expected behavior

Orangelight is able to display the request page, no matter how many holdings a given object has

Actual behavior

When an object has many holdings, it takes Bibdata too long to return all the holdings, making the Orangelight request time out because bibdata's call to Alma for this large amount of data fails.

Steps to replicate

• Go to https://catalog.princeton.edu/catalog/9933643713506421
• (for extra inspectability open the network tab in your browser's developer tools)
• Click the 'Request' button - it will wait for awhile, then you will either see the generic error page in the browser, or it will finally load (but after a fairly long wait)

You can also see the speed of the blocking request at https://bibdata.princeton.edu/bibliographic/9933643713506421/holdings/22727480400006421/availability.json

Impact of this bug

Patrons cannot request items when an object has many holdings.

Honeybadger link and code snippet, if applicable

https://app.honeybadger.io/projects/54399/faults/86853263

bibdata.rb  19  items_by_mfhd(...)
[PROJECT_ROOT]/app/models/concerns/requests/bibdata.rb:19:in `items_by_mfhd'

def items_by_mfhd(system_id, mfhd_id)
    # response = bibdata_conn.get "/availability?mfhd=#{mfhd_id}"
    response = bibdata_conn.get "/bibliographic/#{system_id}/holdings/#{mfhd_id}/availability.json"
    parse_response(response)
end

Implementation notes, if any

• It's possible we need to add pagination to the bibdata requests / responses, to handle objects with so many items
  Issue description and ideas

[maxkadel]

Note that the request to https://bibdata.princeton.edu/bibliographic/9933643713506421/holdings/22727480400006421/availability.json takes over 1 minute to respond:

[kevinreiss]

There are two options for managing holdings with many item records.

1. Implement pagination or lazy loading on the bibdata availability response.
2. Build the form with the item level data for Alma holdings stored in the solr document (we already do this for SCSB items). For real-time status checking we could either check only items patrons select for requesting or lazy load the availability data on top of the already built form after the intial form load happens with the indexed data. Example record from Alma with a large amount of item data: https://catalog.princeton.edu/catalog/9933643713506421/raw.

[kevinreiss]

Workaround for now is to request the item directly in Alma and SCSB (if at ReCAP). Marie WC or Joan Martine can make the requests.

[kevinreiss]

another example URL https://catalog.princeton.edu/catalog/995639693506421

[kevinreiss]

To get a sense of the scale of the issue we are facing here asked for some fresh data on holdings with many items:

https://github.com/PrincetonUniversityLibrary/alma-config/issues/456

Update: Approximately 2,700 holdings have greater than 100 items. 68 greater than 1000, 202 greater than 500. 500 items is roughly the point the current form struggles to load data.

[kevinreiss]

An interim solution for these records could simply be to look at the count of individual items that are listed in the holdings_1display "items" key. This data is added to the records from the Alma publishing process. If we judge that 500 items is the maximum amount the current form architecture can support we could have the request page at least give the user the same fallback behavior we provide for a record with no item data and allow them to manually type the volume info they want and submit the request to circulation staff so they can manually place the request the user in the appropriate staff interface (either Alma or SCSB).

For example if you check the raw solr_xml for https://catalog.princeton.edu/catalog/995639693506421, in https://catalog.princeton.edu/catalog/995639693506421/raw you'll see there are over a thousand items attached to a holding on the record.

[kevinreiss]

Discussion document related to this ticket: https://docs.google.com/document/d/1Qnmc6TKD4OwtRr_Fer9F2-SJW54Vi41CZaOAxZYRtZ0/edit#heading=h.quvh73610qly