Skip to content

Latest commit

 

History

History
71 lines (52 loc) · 2.69 KB

populatepostgres.rst

File metadata and controls

71 lines (52 loc) · 2.69 KB
Raw

populate postgres

Populate PostgreSQL for the first time

Socorro supports multiple products, each of which may contain multiple versions.

  • A product is a global product name, such as Firefox, Thunderbird, Fennec, etc.
  • A version is a revision of a particular product, such as Firefox 3.6.6 or Firefox 3.6.5
  • A branch is the indicator for the Gecko platform used in a Mozilla product / version. If your crash reporting project does not have a need for branch support, just enter “1.0” as the branch number for your product / version.

Customize CSV files

Socorro comes with a set of CSV files you can customize and use to bootstrap your database.

Set up environment :: make virtualenv . socorro-virtualenv/bin/activate export PYTHONPATH=.

Load the Socorro schema :: ./socorro/external/postgresql/setupdb_app.py --database_name=breakpad

Customize CSVs in tools/dataload/, at minimum you probably need to bump the dates and build IDs in :: raw_adu.csv reports.csv releases_raw.csv

You will probably want to change "WaterWolf" to your own product name and version history, if you are setting this up for production.

See databasetablesbysource-chapter for a complete explanation of each table.

Run backfill function to populate matviews

Socorro depends upon materialized views which run nightly, to display graphs and show reports such as "Top Crash By Signature".

IMPORTANT NOTE - many reports use the reports_clean_done() stored procedure to check that reports exist for the last UTC hour of the day being processed, as a way to catch problems. If your crash volume does not guarantee one crash per hour, you may want to modify this function in socorro/sql/schema.sql and reload the schema :

./socorro/external/postgresql/setupdb_app.py --database_name=breakpad --dropdb

ALSO - the backfill procedure ignores any data over 30 days old. Make sure you've adjusted the dates in the CSV files appropriately, or change these funtions in the schema.sql and reload the schema as above.

Normally this is run for the previous day by cron_daily_matviews.sh but you can simply run the backfill_matviews() function to bootstrap the system.

This is normally run by the import.sh, so take a look in there if you need to make adjustments.

There also needs to be at least one featured version, which is controlled by setting "featured_version" column to "true" for one or more rows in the product_version table. The import script will go ahead and set all imported versions as featured.

After modifying CSV files, use the import script to load the data :: ./tools/dataload/import.sh