Allow late-binding of datasources #622

Closed
artemp opened this Issue Oct 11, 2011 · 4 comments

2 participants

@artemp
Mapnik member

Currently when you create a Datasource it immediately (in the constructor) checks to see whether the file/database/table/etc is available.

For cascadenik we now have a compile option which instantiates mapnik objects (including datasources) and then serializes them out to XML. This obviously is a bit annoying if the production DB/data/etc need to be available.

Proposal:

  • Change the constructor signature of datasources to be datasource(const mapnik::parameters& params, bool bind=true). It will still check for the right parameters (eg. table parameter for postgis datasource, file parameter for shape, etc) but NOT attempt to verify that they exist or are accessible.
  • Add a public void bind() method to the datasources which does the same work the constructor used to do - checking files exist, connecting to DBs, sorting out extents, etc. If it's called more than once it no-ops.
  • bind() would be called by the constructor when bind=true (the default), so existing behaviour doesn't change.
  • envelope(), features(), and features_at_point() would all check for bound-ness and call bind() if they're not.
  • the bind() method and the bind constructor parameter would be exposed through the python bindings.
@artemp
Mapnik member

[gravitystorm] This would massively help with large-raster data sources. At the moment if a raster is greater than 512*512 ( http://trac.mapnik.org/browser/trunk/plugins/input/raster/raster_datasource.cpp#L135 ) there is significant overhead in pre-processing - leading to >15 minutes mapnik-startup times when using >10,000 raster layers. Deferring this processing to when the raster is actually needed would be great.

@artemp
Mapnik member

[rcoup] Plan is to port to 0.7.x branch after committed to trunk - the datasource stuff doesn't appear to have changed that much so it shouldn't be too hard.

@artemp
Mapnik member

[springmeyer] applied to trunk in r2291 and backported to 0.7.2 in r2293

@artemp artemp closed this Oct 11, 2011
@springmeyer
Mapnik member

see reworking of this at #962

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment