Skip to content

uapublius/updb-importers

Repository files navigation

UPDB Importers

  • Hosts the UPDB postgres database

  • Contains source for importers that read, normalize, and insert reports into the database.

    • Parse dates using manual processing and chrono-node
    • Parse locations into {city, district, country, water, other}
      • Detect country codes from list
      • Detect province/state/district from list
      • Detect water bodies from list
    • Detect media attachments in reports
    • Detect references in reports
    • Clean reports: detect and use old encodings, normalize html & plaintext line breaks and spaces, replace invisible/unusable ASCII/unicode characters.
  • Houses config file for serving postgREST API from database.

Data

The data to import is generated by the updb-scrapers project.

API

Postgrest serves a RESTful API of the database. See server/postgrest.conf.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages