Best practices for pulling 10 years of employment-discrimination dockets against Fortune 500 companies via the RECAP v4 API

[AbiramiRathina]

Hi Free Law Project team, and hello to anyone in the community who's solved a similar query before. I'm putting together a research dataset and I'd love a sanity check on my approach before I start hammering the API.
What I'm trying to do
For each company on the Fortune 500 list, I want to retrieve every federal civil docket from RECAP where the company appears as a defendant in an employment-discrimination case filed in the last 10 years (filed on or after 2016-05-15). I'm scoping "employment discrimination" broadly to the civil-rights / employment Nature of Suit (NOS) family rather than only DEI-framed suits — specifically:

442 – Civil Rights: Employment (Title VII, ADEA, Equal Pay Act)
445 – Americans with Disabilities Act – Employment
440 – Other Civil Rights (catch-all; will be filtered/reviewed downstream)

(Source: PACER Civil Nature of Suit Code Descriptions, Rev. 12/22.)
The final output is a per-company table of dockets (case name, court, docket number, date filed, NOS, party block, link to the CourtListener docket).
Proposed pipeline

Ingest the Fortune 500 list (name + common aliases / former names / known subsidiaries) into a normalized CSV. I'll dedupe and lowercase, and keep an aliases column so e.g. "Alphabet Inc." also matches "Google LLC."
For each (company, alias) issue a RECAP search query against /api/rest/v4/search/?type=r with:

party_name=
nature_of_suit=442 (and repeat for 445, 440)
filed_after=2016-05-15
filed_before=2026-05-15
order_by=dateFiled desc

Follow the cursor (next URL in each response) until exhausted — v4 supports deep cursor pagination, so I won't hit the old page-100 ceiling.
Post-filter to keep only rows where the matched party is on the defendant side (the parties block inside each docket returns party_types, which I'll check for Defendant).
Persist to Parquet, plus a thin SQLite index for joins.
Throttle: stay well under the per-minute / per-hour cap; back off on 429s.

Questions I'd love guidance on

Is type=r on /api/rest/v4/search/ the right entry point for this, or would I be better served hitting the database-backed /api/rest/v4/dockets/ endpoint and joining /api/rest/v4/parties/ (with filter_nested_results=True) myself? My intuition is that search is better for fuzzy party-name matching, but the docket endpoint gives stricter, paginated filtering. Which does the team recommend for a bulk research crawl like this?
Party-name matching semantics. When I pass party_name=Walmart Inc., how is that tokenized? Does it do a phrase match, a match on the analyzed field, or something else? I want to make sure I'm not silently missing "Wal-Mart Stores, Inc." or "Walmart, Inc." (no period). Is there a recommended way to OR a list of aliases in one query, or should I just fire one request per alias?
NOS filtering. Is nature_of_suit exposed as a direct filter on type=r, or do I need to use the advanced-operators string (e.g. q=natureOfSuit:442)? If the latter, can I OR codes in a single query (natureOfSuit:(442 OR 445 OR 440))?
Defendant-side filter. Is there a server-side way to constrain party_name matches to a particular party_type (Defendant), or is post-filtering on the returned parties array the expected pattern?
Date filter. Is filed_after / filed_before the right param pair on type=r, or should I be using dateFiled__gte / dateFiled__lte against the docket endpoint?
Rate limits as of May 2026. I saw the recent change to default throttles — for a one-time bulk pull like this (probably ~500 companies × 3 NOS codes × aliases), is there a recommended pacing, and is a membership the right route? Happy to support FLP either way.
Result-count caveat. The docs note that type=r uses a cardinality aggregation with ±6% error above 2,000 results. For per-company counts that should be fine (counts will be small), but I want to confirm that the docket list itself is exact and only the count field is approximate.
Bulk data alternative. Would you actually steer me to the bulk-data dumps for a 10-year retrospective like this instead of the live API? If so, which file(s)?

Sketch of the code (Python)
This is the rough shape — I'd love a code review before I run it at scale. Token auth, cursor pagination, retry-with-backoff on 429.
python"""
recap_fortune500_employment_discrim.py
Pulls federal civil dockets where Fortune 500 companies appear as parties
in NOS 440 / 442 / 445 cases filed in the last 10 years.
"""

import csv
import json
import os
import time
from pathlib import Path
from urllib.parse import urlencode

import requests

API_ROOT = "https://www.courtlistener.com/api/rest/v4"
SEARCH = f"{API_ROOT}/search/"
TOKEN = os.environ["COURTLISTENER_TOKEN"]

HEADERS = {
"Authorization": f"Token {TOKEN}",
"User-Agent": "research-crawler/1.0 (contact: arathina@seas.upenn.edu)",
}

NOS_CODES = ["442", "445", "440"] # employment + civil rights
FILED_AFTER = "2016-05-15"
FILED_BEFORE = "2026-05-15"

OUT_DIR = Path("./out")
OUT_DIR.mkdir(exist_ok=True)

def load_fortune500(csv_path: str) -> list[dict]:
"""CSV columns: rank, company, aliases (pipe-separated)."""
with open(csv_path, newline="") as f:
return list(csv.DictReader(f))

def search_recap(party_name: str, nos: str) -> list[dict]:
"""Yield every docket matching (party_name, nos) in the date window."""
params = {
"type": "r",
"party_name": party_name,
"nature_of_suit": nos,
"filed_after": FILED_AFTER,
"filed_before": FILED_BEFORE,
"order_by": "dateFiled desc",
}
url = f"{SEARCH}?{urlencode(params)}"
results: list[dict] = []

while url:
    resp = _get_with_backoff(url)
    payload = resp.json()
    results.extend(payload.get("results", []))
    url = payload.get("next")  # cursor URL, or None when exhausted
return results

def _get_with_backoff(url: str, max_tries: int = 6) -> requests.Response:
delay = 2.0
for attempt in range(max_tries):
r = requests.get(url, headers=HEADERS, timeout=60)
if r.status_code == 429:
wait = int(r.headers.get("Retry-After", delay))
time.sleep(wait)
delay *= 2
continue
r.raise_for_status()
# gentle pacing well under the per-minute cap
time.sleep(1.2)
return r
raise RuntimeError(f"Exhausted retries for {url}")

def is_defendant(docket: dict, alias: str) -> bool:
"""Check the nested parties block to confirm alias is a Defendant."""
alias_l = alias.lower()
for party in docket.get("parties", []) or []:
name = (party.get("name") or "").lower()
if alias_l not in name:
continue
types = {pt.get("name", "") for pt in party.get("party_types", []) or []}
if "Defendant" in types:
return True
return False

def main(fortune500_csv: str):
companies = load_fortune500(fortune500_csv)
sink = OUT_DIR / "dockets.jsonl"
with sink.open("w") as out:
for row in companies:
aliases = [row["company"]] + [
a.strip() for a in row.get("aliases", "").split("|") if a.strip()
]
for alias in aliases:
for nos in NOS_CODES:
for docket in search_recap(alias, nos):
if not is_defendant(docket, alias):
continue
record = {
"rank": row["rank"],
"company": row["company"],
"matched_alias": alias,
"nos": nos,
"case_name": docket.get("caseName"),
"court": docket.get("court_id") or docket.get("court"),
"docket_number": docket.get("docketNumber"),
"date_filed": docket.get("dateFiled"),
"absolute_url": docket.get("absolute_url"),
"docket_id": docket.get("docket_id") or docket.get("id"),
}
out.write(json.dumps(record) + "\n")

if name == "main":
main("fortune500.csv")
A couple of specific things I'm unsure are right in the code above

I'm assuming party_name, nature_of_suit, filed_after, filed_before all coexist on type=r. The docs show party_name and court examples, but I haven't seen all four combined in one example — please correct me if any of these need to be moved into the q= advanced-syntax string instead.
The parties/party_types shape I check in is_defendant is based on the v4 nested-results docs. If the search result row doesn't actually include the full parties block (vs. only when filter_nested_results=True on the parties endpoint), I'd need to do a second hop per docket — please flag if that's the case.
For pagination I'm trusting the next URL exclusively (no manual cursor= handling). Confirming that's the recommended approach.
I'd rather make one query per (alias, NOS) than batch via q=, because the explicit filters seem more predictable than full-text search. If batching via q=natureOfSuit:(440 OR 442 OR 445) is actually preferred for crawl efficiency, I'll switch.

Why I'm asking before crawling
I want to be a good citizen of the API: minimize wasted requests, make sure my filters are doing what I think they're doing on the server (not silently fanning out), and avoid re-running the whole thing because I mis-specified the NOS field name. Thanks in advance — and if there's a tag/label on the repo that's better for "API usage" questions than Discussions, happy to move it.

[AbiramiRathina]

Hi Team,

Waiting on your response for my question

[jmann1234]

Hey @AbiramiRathina - the entity resolution piece you're wrestling with (Walmart vs. Wal-Mart Stores vs. Walmart Inc.) is why I built CaseVine.ai. Tt's a REST API that handles the alias matching and returns all federal cases by company name with severity scoring. No PACER account needed. Happy to give you free credits to test it against your Fortune 500 list - casevine.ai