Create clean/ca/los_angeles_pd.py

[gguerrei]

• Agency Slug: ca_los_angeles_pd
• Module: clean/ca/los_angeles_pd.py
• URL: https://www.lapdonline.org/senate-bill-1421-senate-bill-16-sb-16/

[gguerrei]

MAIN VISION FOR LA:

LA has two systems for getting to the next request page:

1. From main index page -> second index page that holds all child pages (links displayed go directly to nextrequest)
2. From main index page -> second index page that holds most current child pages + more index pages divided by year

In both cases, all URLs come structured (displayed directly on the HTML, no need for reconstruction)

Need two distinct approaches to tackle this:

1. For the first one, straightforward: once on the second index page, write a function that gets child pages URLs and stores them in a list. (THIS PART IS DONE)
2. For the second one, more convoluted: will likely require running a second get_index_page_urls while at the same time running the get_child_scrape. Will need to vet which third index page URLs have already been scraped. The challenge here is understanding order of operations, and knowing where each subsequent link should be stored.

Brainstorming the second method:

• For the sake of discussion, let's redefine some terms:
• Main index page - holds all topics covered by the various public records laws (https://www.lapdonline.org/senate-bill-1421-senate-bill-16-sb-16/)
• Second index page - Officer Involved Shooting (OIS) page. Accessible by clicking the link directly from main index page (https://www.lapdonline.org/sb-1421-ois-main-page/)
• Third index page - yearly pages accessible from the second index page (https://www.lapdonline.org/sb-1421-ois-2023/)
• Child pages - links directly to next request. Accessible from second index page for current year, accessible for other years from respective third index pages. (https://lacity.nextrequest.com/documents?folder_filter=F007-23)

Now that we understand the terms, let's move on to the plan of attack.
The scraping needs to be broken down into two parts:

PART 1: main index page scrape -> second index page scrape

from there, two possible paths:

PART 2A: child page scrape for current year
PART 2B: third index page scrape -> child page scrape

To achieve that, here are some goals:

• Figure out how to write the conditional to run one function instead of the other (or figure out a way to make this selection manual without making the experience worse for the user + fits with other scrapers lexicon and workflow)
• Write a checker function for future scrapes that checks if third index pages were already scraped (since they occupy previous years, no new information will be added to the pages)