Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Peformance optimization to BoundarySet resource #24

Closed
wants to merge 41 commits into from

3 participants

@rhymeswithcycle

I was running into some performance issues with boundary-set/ URLs -- they were doing the equivalent of SELECT * FROM boundaryservice_boundary, which meant asking PostGIS for a lot of rather hefty polygons.

This changes that to a SELECT id, slug FROM boundaryservice_boundary, which in my use case helped performance quite a bit.

rhymeswithcycle added some commits
@rhymeswithcycle rhymeswithcycle Switch to setuptools b25525c
@rhymeswithcycle rhymeswithcycle Performance optimization: when accessing boundary-set resource(s), do…
…n't SELECT polygons from the DB
a2012d3
@rhymeswithcycle rhymeswithcycle A few light model revisions eed47ac
@rhymeswithcycle rhymeswithcycle Store geodata in WGS84 lat/lon 3171a64
@rhymeswithcycle rhymeswithcycle Experimental rewrite of the API code
- no longer uses tastypie (though it's still a dependency for some utilities)
- geospatial fields are no longer included in standard responses -- they're a separate request
- much faster performances in some cases (e.g. /boundary-set/)
- different URL structure: boundaries are /boundary/federal-electoral-districts/outremont/
- support KML and WKT output
- et cetera!

Models and the loader and slightly changed (and so incompatible) but most of the code
is the same.
db2cdaa
@rhymeswithcycle rhymeswithcycle Shapefile loader: make transactional and don't reload by default e152729
@rhymeswithcycle rhymeswithcycle Typo ff94ccb
@rhymeswithcycle rhymeswithcycle Fix mistake with passing around the database name 0f14301
@rhymeswithcycle rhymeswithcycle Change definition of 'intersects' to ST_Covers OR ST_Overlaps; add 't…
…ouches' query
c79e066
@rhymeswithcycle rhymeswithcycle Allow shape output on /boundary/ queries 42518bb
@rhymeswithcycle rhymeswithcycle Output tweaks eeafc3e
@rhymeswithcycle rhymeswithcycle Limit total number of resources returned in geo list queries 136ff81
@rhymeswithcycle rhymeswithcycle Remove kind_first and display_name fields 0681cf2
@rhymeswithcycle rhymeswithcycle Remove count and metadata_fields from BoundarySet; rename href to sou…
…rce_url
0a3eb23
@rhymeswithcycle rhymeswithcycle Change definitions format d80bffe
@rhymeswithcycle rhymeswithcycle Use plural names for resource types in URLs 50f2acf
@rhymeswithcycle rhymeswithcycle Keep shapefiles at data/shapefiles by default 97b10a6
@rhymeswithcycle rhymeswithcycle Raise error when /boundaries/set-slug URL points to a nonexistent set 7a25ace
@rhymeswithcycle rhymeswithcycle Improve over-limit error message in geo list view df9265d
@rhymeswithcycle rhymeswithcycle Boundaries, not Boundarys 89d2dd9
@rhymeswithcycle rhymeswithcycle Add related-URLs support cd75c31
@rhymeswithcycle rhymeswithcycle Switch to django-jsonfield instead of including our own JSONField code a7db990
@rhymeswithcycle rhymeswithcycle Tweaks: maximum results for geo queries, parameter names 54779c6
@rhymeswithcycle rhymeswithcycle Add titlecase to repository 249c423
@rhymeswithcycle rhymeswithcycle Tweak titlecase to deal better with accented characters and phrases i…
…ncluding digits
f9817d1
@rhymeswithcycle rhymeswithcycle Improve Unicode handling; add functions for common string cleanup tasks 04eeb30
@rhymeswithcycle rhymeswithcycle Expose licence_url on boundary-sets b13c049
@rhymeswithcycle rhymeswithcycle Add preliminary version of HTML API browser e4cbb64
@rhymeswithcycle rhymeswithcycle Continue work on API-browser interface 6c7fff8
@jamesturk

any chance of this getting merged in? was having the same problem (hitting the list of ZCTAs would bring down Postgres)

@onyxfish

Hey guys, sorry I lagged in merging this pull request. It should have been a simple one, but now it seems to have gotten messy with a lot of unrelated commits stacked on top. I'm going to cherry-pick out the changes and commit just those. I've tested this locally and it works, but please feel free to give it the once over and see if I did anything stupid.

@onyxfish onyxfish closed this in f28caae
@jamesturk

did this commit get cherry-picked? this is the one I was talking about before the other commits got piled on w/ this pull request

opennorth@a2012d3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 17, 2011
  1. @rhymeswithcycle

    Switch to setuptools

    rhymeswithcycle authored
  2. @rhymeswithcycle

    Performance optimization: when accessing boundary-set resource(s), do…

    rhymeswithcycle authored
    …n't SELECT polygons from the DB
Commits on Dec 24, 2011
  1. @rhymeswithcycle
  2. @rhymeswithcycle
Commits on Dec 25, 2011
  1. @rhymeswithcycle

    Experimental rewrite of the API code

    rhymeswithcycle authored
    - no longer uses tastypie (though it's still a dependency for some utilities)
    - geospatial fields are no longer included in standard responses -- they're a separate request
    - much faster performances in some cases (e.g. /boundary-set/)
    - different URL structure: boundaries are /boundary/federal-electoral-districts/outremont/
    - support KML and WKT output
    - et cetera!
    
    Models and the loader and slightly changed (and so incompatible) but most of the code
    is the same.
  2. @rhymeswithcycle
  3. @rhymeswithcycle

    Typo

    rhymeswithcycle authored
  4. @rhymeswithcycle
Commits on Jan 9, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle
  3. @rhymeswithcycle

    Output tweaks

    rhymeswithcycle authored
  4. @rhymeswithcycle
  5. @rhymeswithcycle
  6. @rhymeswithcycle
  7. @rhymeswithcycle
  8. @rhymeswithcycle
Commits on Jan 11, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle
  3. @rhymeswithcycle
  4. @rhymeswithcycle
Commits on Jan 17, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle
Commits on Jan 20, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle
  3. @rhymeswithcycle
  4. @rhymeswithcycle
  5. @rhymeswithcycle
  6. @rhymeswithcycle
Commits on Jan 23, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle
Commits on Jan 28, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle
  3. @rhymeswithcycle
  4. @rhymeswithcycle
Commits on Jan 29, 2012
  1. @rhymeswithcycle
  2. @rhymeswithcycle

    Add initial migration

    rhymeswithcycle authored
  3. @rhymeswithcycle
  4. @rhymeswithcycle
  5. @rhymeswithcycle

    New README

    rhymeswithcycle authored
  6. @rhymeswithcycle
  7. @rhymeswithcycle

    Readme update

    rhymeswithcycle authored
Something went wrong with that request. Please try again.