Update "Downloading Multiple Records Using Query Strings"

[charlottejmc]

I am opening a space to work on updating the lesson Downloading Multiple Records Using Query Strings in light of the recent changes to the Old Bailey's API.

Although the old version of the website will still be accessible until August 2024 at https://www.dhi.ac.uk/oldbaileyonline, we want to update lessons which are affected by this change so they remain usable in the future.

This lesson poses an issue because the URLs for the search page have changed, so the examples in the Downloading the Search Results Page section don't work anymore.

This lesson has currently been translated to ES and PT, so it will need to be updated in those languages too.

[charlottejmc]

I've gone through the first part of the lesson noting down each line where I can determine that a change will be needed. Often (but not always) I was able to suggest that change myself. However, it became gradually more complex to identify the problematic lines and the necessary changes from line 704 onward.

• Line 107. Change link to new Advanced Search form: https://www.oldbaileyonline.org/search/advanced?div_type=trialAccount&text=mulatto%2A%20negro%2A&year_gte=1700&year_lte=1750
• Line 121. Change the image under “The Advanced Search on OBO” to show the new interface
• Line 122. Remove “Calculate total” because new website doesn’t let us do this (I think)
• Line 142. Change the URL to https://www.oldbaileyonline.org/search/advanced?div_type=trialAccount&text=mulatto%2A%20negro%2A&year_gte=1700&year_lte=1750#results
  • Do we want to ask readers to click on Next here already to generate the &from=10 component?
• Code block 151 - 162. Update the division of the URL components to match the new URL
  • https://www.oldbaileyonline.org/search/advanced
  • ?div_type=trialAccount
  • &text=mulatto%2A%20negro%2A
  • &year_gte=1700
  • &year_lte=1750
  • #results (?)
• Line 165. Review how many “important pieces of information” there are (6 instead of 12?) and whether we keep the #results. This paragraph will have to be updated to explain what these new components refer to.
• Line 184. This is not a “+” sign anymore, but a series of symbols
• Line 187 - 190. Adjust components for new URL (perhaps a discussion about what _gte and _lte might mean here? The rest of the discussion is still relevant.)
• Line 205 - 209. To get these components, I have to click on Next to get results 11-13: then, my URL looks like this:
  • https://www.oldbaileyonline.org/search/advanced?div_type=trialAccount&from=10&text=mulatto%2A%20negro%2A&year_gte=1700&year_lte=1750#results so it has added &from=10
• Line 220. Change “We also know that by changing the “start” value in the URL we can sequentially call each search results page, and ultimately retrieve all of the trial documents from them.” -> to change “from” value instead
• Line 236. Change start to from
• Line 255. Update to new URL, which includes &from=0
• Line 258. Change start to from
• Code block 265 - 291. Update needed to adjust to the new URL
  • Needs more technical expertise than I have
• Code block 304 - 310. Update needed to adjust to the new URL
• Check that the code blocks at Lines 328 / 339 / 350 / 369 / 388 still work for this URL
• Line 427. Check that we are still using toMonth (because this URL doesn’t show months)
• Code block 441. Update the code to match the changes made in Code Block 265 - 291 above
• Code Block 483 - 489. Check it still works with new URL
• Code Block 530 - 571. Again, Update the code to match the changes made in Code Block 441 above
• Line 589 - 653. Our new URL does not have a * so this could probably be removed, BUT we have % signs now so they might still pose the same issue, in which case we slightly adjust the paragraph at Line 591.
• Line 667. I searched all previous lessons for print.jsp and did not see it – where was this taught previously?
• Line 673. This URL is now wrong. The link to the trial is https://www.oldbaileyonline.org/record/t17800628-33 but I’m not sure how to find the ‘print’ version
• Line 697. Update link to current version
  • Find the way the links are formatted in the new HTML code of the downloaded files
• Line 702 - 704. Similar update

This seems to be an indicator that this lesson will need quite an involved revision and perhaps some external help.

[anisa-hawes]

Super work, @charlottejmc. Thank you for adding these thorough notes, and for going as far as you have done.

I'll identify an external person who can support us with these revisions.

[anisa-hawes]

Adding to this that a reader has written to let us know that Downloading Web Pages with Python is also affected by this change to the Old Bailey Online's URL structure.