Permalink
Switch branches/tags
Nothing to show
Commits on May 22, 2014
  1. Updated acs2011_5yr scripts to create sequences with geoid field and …

    leehach committed May 22, 2014
    …populate field on table load.
  2. sql_insert_into_tables now adds geoid value on table load. This appro…

    leehach committed May 22, 2014
    …ach is *much* faster than updating after load.
Commits on May 15, 2014
Commits on Dec 3, 2013
  1. Scripts for spatial support

    leehach committed Dec 3, 2013
Commits on Nov 23, 2013
  1. Add scripts to download TIGER/Line water features from Census and use…

    leehach committed Nov 23, 2013
    … them to clip Census tracts to shoreline.
Commits on Nov 19, 2013
  1. Minor addition to README.

    leehach committed Nov 19, 2013
Commits on Nov 11, 2013
  1. Added geoid field to all sequence (estimate and moe) tables. Geoid

    leehach committed Nov 11, 2013
    is emerging as the de facto global unique identifier to be used
    to join Census data. It is currently used to link TIGER/Line
    geographic data to geoheader file in various Census data products,
    but does not appear in the sequence files, thus requiring the use
    of (stusab, logrecno) for joins among tables. While geoid is added
    primarily for intended support of geographic data with PostGIS,
    its use as a single field unique key makes it worthwhile to include
    in any event.
    
    Geoid has been added to CREATE statement for all sequence tables.
    In order to support existing installation, a function
    sql_add_geoid_to_storage_tables() has been added to perform a one-
    time update via ALTER TABLE. All "subject table" views have now
    been modified to include geoid, so if views are recreated in an
    existing installation, sql_add_geoid_to_storage_tables() must
    have been run first!
    
    Geoid is created with a UNIQUE index, which will slow down bulk
    loads, but will increase JOIN efficiency.
    
    At the moment, geoid is *not* populated when data is first copied
    from the staging tables to final storage. A function
    sql_update_geoid_storage_tables() is provided to write values to
    geoid based on matching records in geoheader. This function can
    take a considerably long time to run, even for a single table,
    and the function includes input parameters to restrict by seq
    or estimate/moe.
  2. Added explicit field list to INSERT statement generated by sql_parse_…

    leehach committed Nov 11, 2013
    …tmp_geoheader(). Preparatory for adding geometry support, in which case implicit column ordering can no longer be assumed.
  3. Added omnibus import function which creates staging tables, imports d…

    leehach committed Nov 11, 2013
    …ata, transfers data to final storage, and cleans up (drops) staging tables. Documentation added to README.
  4. Merge branch 'import_log'

    leehach committed Nov 11, 2013
  5. Stubbed comprehensive import/insert script, need to merge with master…

    leehach committed Nov 11, 2013
    … to use functions added there.
Commits on Nov 10, 2013
  1. Added optional parameters to sql_drop_import_tables() and sql_create_…

    leehach committed Nov 10, 2013
    …import_tables() to restrict drop/create by sequence or by estimate/MOE.
  2. Added import_log table and sql_populate_import_log() function to keep…

    leehach committed Nov 10, 2013
    … track of files successfully imported.
  3. Updated meta-script README.

    leehach committed Nov 10, 2013
  4. Added several maintenance functions to DROP, TRUNCATE, or set autovac…

    leehach committed Nov 10, 2013
    …cum on sequence storage tables.
  5. Added explict column list to INSERTs in preparation for adding geomet…

    leehach committed Nov 10, 2013
    …ry columns (which would screw up implicit column list since raw Census files lack geometry).
Commits on Nov 9, 2013
  1. Fixed determination of first field in sequence to account for acs2012…

    leehach committed Nov 9, 2013
    …_1yr not beginning at seq_position=1 for some sequences (?).
  2. Use UNLOGGED staging tables for Postgres>=9.1. Fixed determination of…

    leehach committed Nov 9, 2013
    … first field in sequence to account for acs2012_1yr not beginning at seq_position=1 for some sequences (?).
  3. All SQL-creating functions in meta-scripts no longer return (sometime…

    leehach committed Nov 9, 2013
    …s lengthy) SQL statement if exec=TRUE. SQL statement only returned if exec=FALSE.
Commits on Apr 2, 2013
  1. Import scripts had hard-coded filename base (e.g. 20115, for ACS 2011…

    leehach committed Apr 2, 2013
    … 5-year). Added getter function to define filename base per schema, and adjusted meta-scripts to read filename base when dynamically creating DDL.
Commits on Feb 21, 2013
  1. Replaced idiosyncratic filepath with <census_upload_root> placeholder…

    leehach committed Feb 21, 2013
    … for ACS 2011; added README.md to ACS 2011 folder
Commits on Feb 17, 2013
  1. Simplified main README, moved instructions on use of meta-scripts to …

    leehach committed Feb 17, 2013
    …the meta-scripts folder
Commits on Jul 23, 2012
  1. Added state-specific import scripts to acs2010_5yr; Cleanup of delete…

    unknown unknown
    unknown authored and unknown committed Jul 23, 2012
    …d files