Skip to content

[Data Source] Build Durham County BOCC agenda, minutes, and vote scraper #167

Description

@ThatXliner

Summary

Add Durham County Board of Commissioners (BOCC) meetings as the county-level half of the Durham local-information pilot.

Official sources

The document-heavy County archive is separate from the City OnBase source and should be tested independently.

Scope

  • Discover the CivicPlus/archive pagination request; do not crawl thousands of unrelated documents.
  • Enumerate regular/work/special sessions, public hearings, cancellations, and amendments.
  • Parse agenda/minutes PDFs into agenda items, consent items, hearings, motions, outcomes, and named roll calls where present.
  • Preserve bilingual documents without mixing Spanish into English extraction.
  • Match official video records deterministically by meeting date.
  • Map through a Durham County adapter to the local-government contract.
  • Add caching/checksums, source-version tags, polite concurrency, OCR diagnostics, and idempotent upserts.
  • Test a regular session, work session, cancellation, and amended document.

Acceptance criteria

  • Backfill 12 months with agenda items and final actions under the same meeting.
  • Replaced PDFs update rather than duplicate.
  • Every record cites the official County page/document; one parse failure cannot abort the run.

Outreach Tracker

  • Track: Data Source
  • Stage: Researching
  • Playbook: A — no outreach unless a stable archive feed cannot be identified
  • Time sensitivity: Medium
  • Next action: identify archive pagination and create 2026 fixtures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: businessFunding, legal, partnerships, opsarea: scrapersapps/scraper (data source scrapers)enhancementNew feature or requestsourceRelated to news/information sources e.g. scrapers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions