apply sql (e.g. ExecuteSQL) prior to Collection iteration

[bekozi]

Is it possible to apply a limiting SQL statement prior to iterating over a fiona Collection?

[sgillies]

Can you be more specific? Do you mean an attribute filter or something else?

[bekozi]

Below is a code example. It amounts to an attribute filter (there may be a better way to do it). Fiona is perfect for my application except for iteration over large shapefiles when only a small subset of the geometries are actually needed.

from osgeo import ogr
from shapely import wkb

shp_path = 'foo.shp'
select_ugid = [1,2,3]
ds = ogr.Open(shp_path)
try:
    lyr = ds.GetLayerByIndex(0)
    lyr.ResetReading()
    if select_ugid is not None:
        lyr_name = lyr.GetName()
        ## format where statement different for singletons
        if len(select_ugid) == 1:
            sql_where = 'UGID = {0}'.format(select_ugid[0])
        else:
            sql_where = 'UGID IN {0}'.format(tuple(select_ugid))
        sql = 'SELECT * FROM {0} WHERE {1}'.format(lyr_name,sql_where)
        features = ds.ExecuteSQL(sql)
    else:
        features = lyr

    geoms = [None]*len(features)
    for idx,feature in enumerate(features):
        attrs = feature.items()
        attrs.update({'geom':wkb.loads(feature.geometry().ExportToWkb())})
        geoms[idx] = attrs
finally:
    ds.Destroy()

[sgillies]

Understood. According to http://www.gdal.org/ogr/drv_shapefile.html unless you've made an attribute index for UGID, your code above is functionally equivalent to this:

import fiona

def select_ugid(f):
    return f['properties']['UGID'] in [1, 2, 3]

with fiona.open('foo.shp') as c:
    features = filter(select_ugid, c)

Because of that and because I don't want to add any SQL to Fiona interfaces, I won't expose OGR's ExecuteSQL in Fiona.

[bekozi]

Thanks, Sean. This is very helpful! I went ahead and closed the ticket...