Skip to content
This repository has been archived by the owner on Jun 5, 2023. It is now read-only.
/ group-two-automation Public archive

discovering, documenting, and assisting the automation of SGID data

License

Notifications You must be signed in to change notification settings

agrc/group-two-automation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Identify all internal SGID-related data processes

Click to see dataset's that are updated
Address Points
  • Zach
  • Greg
Broadband
  • Zach
  • Bert
Boundaries
  • Mike
  • Datasets
    • Municipalities
    • County
    • Voting/Precinct
    • Tax
Roads
  • Michael
  • Greg
Parcels
  • Rick
  • Datasets
    • Normal Parcels
    • LIR
Imagery
  • Rick
  • Datasets
    • Indices
    • Raster.utah.gov
Zone Boundaries
  • Dave
  • Datasets
    • PSAPS
    • Fire
    • EMS
    • Law
    • City Codes
Water
  • Zach
  • Datasets
    • NHD Springs
    • NHD Lakes
    • NHD Streams

Document and review processes

Click to see all discovered processes
  1. Address Points

    1. Existing scripts are written, 1 per participating county
    2. Incomplete because data is not received from
      • Daggett
      • Unitah
      • San Juan
      • Grand
      • Garfield
      • Sanpete
  2. Broadband

    1. needs documentation but automation is not the best fit
  3. Boundaries

    1. Municipal/County Boundaries
      1. PDF or jpg provied by LGO after county submission
      2. Outline is created with COGO tools in state plane and added to muni_modifications
      3. (muni/county)_modification polygon is manually used to adjust official boundary feature class and muni carto feature class
      4. blog post created quarterly
  4. Precints (Vista Ballot Areas)

    1. Mostly shapefiles received from counties
    2. ETL manually into our schema
    3. Validated on overlap with political districts and issues fixed
  5. PLSS Fabric

    1. BLM appends mini fabrics
    2. Counties and surveyors can submit tie sheets for corner adjustments
    3. Uses mostly built in fabric tools from vendor
    4. Exported to PLSS GCDB format and appends attributes manually
  6. Tax Nomenclature

    1. Terri C from tax provides individual geodatabases
    2. Manually create new tax areas using tax process doc
    3. Manual and scripted review happens based on checklist with exceptions
    4. Create prior years and publish new years
    5. Data can be updated quarterly
    6. Blog post is created yearly
  7. Roads

    1. Data is requested from counties based on a schedule
    2. Data is edited in UTRANS
    3. Scripts are run to ETL data
    4. Change detection is run
    5. Custom tool helps reconcile differences in provided vs new data
    6. Monthly blog post
  8. Parcels

    1. Existing scripts ETL data into our schema. One per county.
  9. Sales Tax Areas need to align with municipal and county boundaries

    1. schema and attribute checks
  10. Health stuff is loaded/edited adhoc when reached out to by stewards

    1. email sent with instructions
  11. PSAPS

    1. Only certain dispatch centers are supported
      1. box elder
      2. vernal dispatch
        1. Daggett, Duchesne, Uintah
      3. salt lake communication center (TOC)
      4. weber/morgan
      5. cedar dispatch 2. Iron co
        1. cedar also dispatches DPS on highways, etc
      6. st george dispatch
      7. san juan
      8. beaver
      9. miller
  12. Address Points

    1. ETL address points manually
      1. Sevier, Wayne, and Piute Counties they send David the address points directly
    2. geocodes address points and inspects less than 100% matches
    3. fixes issues manually 2. look at associated street, use street view, look at address ranges, if there is a segment
  13. Roads

    1. Inspectes segments with geocodes scores < 100%
    2. Edits in spillman and Utrans with a note
    3. tool to segment to segment connectivity and create a node splitting the segment recalculate address ranges
    4. dangles as exceptions, vb to pro?
  14. Schools

    1. Spreadsheet from office of ed
    2. Geocoded and ETL to our schema manually
    3. Validate addresses and possibly move point
    4. should update quarterly
    5. addresses are sometimes wrong and need moved
  15. EMS Response Zones

    1. Come from Department of Health Bureau of EMS.
    2. They license EMS agencies and the license describes the area of responsibility.
    3. Polygons are created or modified based on the descriptions.
  16. Care Centers

    1. Spreadsheet from office of ed
    2. Geocoded and ETL to our schema manually
    3. Addresses are validated manually
  17. Enterprise Zones

    1. Kelleigh sends shapefiles in emails
    2. Align boundaries with muni boundaries manually splitting where they cross
  18. Zip Codes

    1. Align boundary with street segments manually
    2. Updated after talking to cities and jurisdictions
    3. Can change when code gets subdivided
  19. Address Grids

    1. Align to boundaries manually
  20. Water   1. Download data from USGS

    1. Scripts exist to ETL
      1. state boundary intersections
      2. split for in/outside of utah
      3. add the ismajor and issubmerged attributes

Decide which processes are best to automate

  1. David is requesting help with dispatch data sets
    1. Change detected for UTRANS and local data streets and address points in PSAP areas
    2. attribute and geometry differences same with the tool that greg has already created
    3. can use pro to edit data
    4. when streets move boudaries sometimes should follow the street
  2. County Municipal boundaries
    1. would like to automate modification edits to live feature class
      1. currently using the trace tool
  3. Vista Ballot Area Updates
    1. The process happens around 29 times a year plus the extras from the organized counties
    2. Might happen a lot more in 2020 with redistricting
    3. would like to automate proposed edits to live propogation (forklift?)
    4. tools to help validate data, topology checks, visual checks, validate vista id's
    5. can be built as a pro addin
  4. GCDB
    1. There are currently publishing tools to go from the fabric to individual feature classes
    2. The process could be made smoother with an attribute ETL after the first step
  5. Tool to align segments and points
    1. There seem to be a lot of use cases to be able to select segments from different feature classes and make them coincident
    2. There is also a need for when digitizing points that they align with a nearby address point

Plan an implementation for automatable processes

Determine data dependencies

https://agrc.github.io/group-two-automation/ graph

  1. green dash geometry/attributes are derived
  2. red dash used for quality control
  3. black data aligns to geometries

About

discovering, documenting, and assisting the automation of SGID data

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Languages